There are many ways to program online experiments that interface with sites such as Amazon Mechanical Turk (AMT) or Prolific. One standard approach is to create a website that is hosted on your own server and send participants from AMT or Prolific over to a site on that server. Once you have a server set up appropriately, you have complete control over how data is read from the server or is saved to the server. This can lead to very powerful and flexible online experiments. However, the disadvantage is the development time needed to set up a server in the first place. If you lack certain coding skills, this setup can be quite daunting.
In this tutorial, I show how to code online experiments that can be run locally on AMT. There is no need to set up your own server — AMT itself will store the data generated by the experiment. This significantly cuts down development time as you can focus on writing code for the logic of the experiment and not have to worry about the logic of communicating with a server. We have used this approach in a number of experiments and I thought it would be good to share the basic steps needed to get started.
The coding in this tutorial is based on a combination of JavaScript, jQuery (a JavaScript library), and basic HTML and CSS. If you have some basic familiarity with JavaScript, you should be good to go. You will also need to set up an account with AMT as a requester. If you need a more basic introduction, there is a nice tutorial by Timothy Brady at UCSD. Just keep in mind that the code in that tutorial will not work with the most recent version of AMT unless you use the <crowd-form> wrapper (see Example 1 below).
Basic experiment
The code below shows a simple example to get started. Save this code locally to your computer and make you sure you use a “html” extension in your filename (e.g. “codeexperiment1.html”). When you double-click on this file, you should be able to run the code in your default browser.
<!-- Code listing 1 -->
<script src="https://assets.crowd.aws/crowd-html-elements.js"></script>
<crowd-form answer-format="flatten-objects">
<h4>Estimate the height of the Eiffel tower (in feet):
<input type="number" name="estimate">
</h4>
<!-- Prevent crowd-form from creating its own button -->
<crowd-button form-action="submit">Submit</crowd-button>
</crowd-form>
Lines 2, 3, 9 and 10 are required lines in the code to make sure you can submit data to AMT. All the HTML code and JavaScript that you develop should be placed between tags <crowd-form>
and </crowd-form>
. In this example, we develop a simple input form that asks for numeric input. Pressing the submit button will submit the participant’s numeric estimate to the AMT server. AMT will create a data field “estimate” corresponding to the name we gave to the input form element on line 5. This code can already be used in AMT as a HIT!
Posting code on AMT
How do we use this code on AMT? Once you have an account as a requester, start a new project. You can pick any of the provided templates, but I typically pick the “other” template. Click on create project. Now, in the first tab, fill out the properties of the HIT (project name, title, etc). Once you are done with that, the second tab, “design layout” is the place where you can insert the HTML code. Remove the existing code that is there (associated with the template), and just paste your code. Now you are good to go! Of course, you might want to test out your code before you make your HIT live and become responsible for payments. AMT has provided a sandbox environment that allows you test your code as a requester as well as a worker. This is convenient if you are not entirely sure if your HIT will work correctly.
Types of user input
Besides numeric input, there are a number of ways to let the user interact with your experiment using standard HTML elements such as checkboxes, radio buttons, and text input. However, be mindful that some standard html input types are not compatible with the AWS crowd-form wrapper and will lead to errors. For example, for buttons, you will have to use the <crowd-button>
form element. For more information on what HTML elements are provided by the crowd-form wrapper, please consult these AWS pages. Some of these crowd-form HTML elements are not particularly attractive or useful and you should feel free to use your own mix of standard HTML and crowd-form HTML elements.
Multiple screens
In the next code example, I will show you how to present the participant a sequence of screens where each screen corresponds to different events in the experiment such as acquiring consent, showing instructions, showing experimental stimuli, asking for feedback, etc. The simplest way to cycle through different screens is to associate each screen with its own HTML section (as defined by the <div> tags) and to change the visibility of each individual screen using jQuery commands.
<!-- Code listing 2 -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.0/jquery.min.js"></script>
<script src="https://assets.crowd.aws/crowd-html-elements.js"></script>
<style>
body {
font-family: Arial, Helvetica; font-size: 12pt;
-webkit-user-select: none; /* Safari */
-moz-user-select: none; /* Firefox */
-ms-user-select: none; /* IE10+/Edge */
user-select: none; /* Standard */
}
span.nobr { white-space: nowrap; }
.grayscreen {
border: 2px solid gray; background: #EEEEEE;
padding-top: 20px; padding-bottom: 20px; padding-left: 50px; padding-right: 50px;
width: 800px; margin: 0 auto;
display: none;
}
.buttonplacement {
width: 120px; padding: 5px;
margin-left: auto; margin-right: 20px; margin-top: 30px;
}
</style>
<crowd-form answer-format="flatten-objects">
<!-- Consent page -->
<div class="grayscreen" id="consentDiv">
<h3>Participant Information</h3>
<p>Welcome to the experiment! Are you ready to participate?</p>
<div class="buttonplacement">
<crowd-button id="consentProceed" onClick="consented()">Proceed</crowd-button>
</div>
</div>
<!-- Estimation page -->
<div class="grayscreen" id="estimationDiv">
<h4>Estimate the height of the Eiffel tower (in feet):
<input type="number" name="estimate">
</h4>
<div class="buttonplacement">
<crowd-button id="estimateProceed" onClick="finishedestimate()">Proceed</crowd-button>
</div>
</div>
<div class="grayscreen" id="submitDiv">
<h4>You have reached the end of the experiment. Press the button to submit your HIT.</h4>
<div class="buttonplacement">
<crowd-button form-action="submit" id="submitbutton">Submit</crowd-button>
</div>
</div>
<script>
function consented() {
$( '#consentDiv').hide();
$( '#estimationDiv').show();
}
function finishedestimate() {
$( '#estimationDiv').hide();
$( '#submitDiv').show();
}
// This is the start of the program -- we reveal the consent/information page
$( '#consentDiv').show();
// Needed to prevent enter key press from submitting the form
$(document).on("keydown", "form", function(event) {
return event.key != "Enter";
});
</script>
</crowd-form>
Lines 4-23 define CSS styles we can use to make the site look a little nicer. Lines 7-10 are used to prevent a cursor from showing up in the site and prevents the user from copying any text. Lines 13-18 define a CSS class we name grayscreen
. This will be used to define the styling of the basic screen for the experiment. On line 17, we define the default visibility for this section to be “none” such that any HTML section styled according to this class will (initially) be invisible. Try to comment out this line (by putting // in front it) and see what happens. Notice that all screens are now visible simultaneously and are placed in sequential fashion. The basic idea of switching between screen is to manipulate the visibility of these screens, which already “exist” in the background.
The HTML code on lines 26-49 sets up three screens, corresponding to a consent page, an experimental trial page, and a final page to submit the results. Note that the HTML is divided into sections defined by the <div> tags. For example, <div class="grayscreen" id="consentDiv">
tells the browser that we start a new section styled according to the grayscreen class and we can refer to this section by the id tag consentDiv
.
On the consent page, we have a button that when clicked will execute the JavaScript function consented()
. Note that this function is defined on lines 52-55. This function will first execute the jQuery command $( '#consentDiv').hide()
which hides the HTML section defined by the div tag with id consentDiv
. On the next line, the jQuery command $( '#estimationDiv').show()
will make the section with div tag estimationDiv
visible.
If you are not familiar with jQuery, consult one of the many online tutorials for jQuery. It is also really helpful to test your understanding of jQuery by trying out different jQuery commands in the console (right click on a site, and click “inspect” to bring up the console).
Finally, line 62 is where the execution of the code starts. This line simply makes the consent page visible, and everything else happens as a result of button presses.
Debugging
Before we add additional complexity to the program, it is important to understand how we can debug the code. The basic console will suffice for our debugging purposes. You can bring it up by right clicking on a webpage and clicking on “inspect”. Look here and here for some tutorials on debugging JavaScript code.
To keep things simple, in the code below, I have added a few extra lines, highlighted in the code, that help the debugging process.
<!-- Code listing 3 -->
<script>const doDebug = 1;</script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.0/jquery.min.js"></script>
<script src="https://assets.crowd.aws/crowd-html-elements.js"></script>
<style>
body {
font-family: Arial, Helvetica; font-size: 12pt;
-webkit-user-select: none; /* Safari */
-moz-user-select: none; /* Firefox */
-ms-user-select: none; /* IE10+/Edge */
user-select: none; /* Standard */
}
span.nobr { white-space: nowrap; }
.grayscreen {
border: 2px solid gray; background: #EEEEEE;
padding-top: 20px; padding-bottom: 20px; padding-left: 50px; padding-right: 50px;
width: 800px; margin: 0 auto;
display: none;
}
.buttonplacement {
width: 120px; padding: 5px;
margin-left: auto; margin-right: 20px; margin-top: 30px;
}
</style>
<crowd-form answer-format="flatten-objects">
<!-- Consent page -->
<div class="grayscreen" id="consentDiv">
<h3>Participant Information</h3>
<p>Welcome to the experiment! Are you ready to participate?</p>
<div class="buttonplacement">
<crowd-button id="consentProceed" onClick="consented()">Proceed</crowd-button>
</div>
</div>
<!-- Estimation page -->
<div class="grayscreen" id="estimationDiv">
<h4>Estimate the height of the Eiffel tower (in feet):
<input type="number" name="estimate">
</h4>
<div class="buttonplacement">
<crowd-button id="estimateProceed" onClick="finishedestimate()">Proceed</crowd-button>
</div>
</div>
<div class="grayscreen" id="submitDiv">
<h4>You have reached the end of the experiment. Press the button to submit your HIT.</h4>
<div class="buttonplacement">
<crowd-button form-action="submit" id="submitbutton">Submit</crowd-button>
</div>
</div>
<script>
function myconsolelog( str ) {
if (doDebug==1) console.log( str );
}
function consented() {
myconsolelog( 'Reached function consented...')
$( '#consentDiv').hide();
$( '#estimationDiv').show();
}
function finishedestimate() {
myconsolelog( 'Reached function finishedestimate...')
$( '#estimationDiv').hide();
$( '#submitDiv').show();
}
// This is the start of the program -- we reveal the consent/information page
$( '#consentDiv').show();
// Needed to prevent enter key press from submitting the form
$(document).on("keydown", "form", function(event) {
return event.key != "Enter";
});
</script>
</crowd-form>
Line 2 defines a variable that we can set to 1 if we want to see what the program is doing by bringing up the console. When the experiment is ready to be posted on AMT, we can set this flag to 0 to make sure that a participant cannot get any special insight into the operation of the program.
Lines 53-55 define a simple function that writes a string to the console but only if the doDebug
variable is set to 1.
Lines 57 and 62 are the actual lines we added to debug the program. For example, suppose we are not quite sure if the function consented()
is ever executed (perhaps nothing is happening). This line will print a message to the console to indicate that the function will indeed be executed. Adding these simple lines will greatly help to understand what is going right or wrong with your code. Note that you can also print values of variables to the console.
Another important debugging tool is to type in commands directly into the console. For example, if you are not quite sure if some variables are defined (correctly), you can type them into the console, and the console will print the current value. Similarly, if you are not quite sure how jQuery works, just try out different queries directly in the console.
Multiple experimental trials and saving internal data
In the next code, we show example code of an experiment that proceeds through multiple trials. On each trial, an image is shown and the task for the participant is to estimate the height of the building shown in the image.
The code highlights how to solve the problems of 1) randomizing the order of trials, 2) validating user input, and 3) storing the results from each individual trials and making sure that these results will get saved to AMT.
<!-- Code listing 4 -->
<script>const doDebug = 1;</script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.0/jquery.min.js"></script>
<script src="https://assets.crowd.aws/crowd-html-elements.js"></script>
<style>
body {
font-family: Arial, Helvetica; font-size: 12pt;
-webkit-user-select: none; /* Safari */
-moz-user-select: none; /* Firefox */
-ms-user-select: none; /* IE10+/Edge */
user-select: none; /* Standard */
}
span.nobr { white-space: nowrap; }
.grayscreen {
border: 2px solid gray; background: #EEEEEE;
padding-top: 20px; padding-bottom: 20px; padding-left: 50px; padding-right: 50px;
width: 600px; margin: 0 auto;
display: none;
}
.buttonplacement {
width: 120px; padding: 5px;
margin-left: auto; margin-right: 20px; margin-top: 30px;
}
#toptextLeft {
color: #4f4f4f;
padding-bottom: 20px;
}
img {
max-height: 400px;
height: 50%;
width: auto;
}
</style>
<crowd-form answer-format="flatten-objects">
<!-- Tag for placing hidden form values -->
<div id="addhere"></div>
<!-- Consent page -->
<div class="grayscreen" id="consentDiv">
<h3>Participant Information</h3>
<p>Welcome to the experiment! Are you ready to participate?</p>
<div class="buttonplacement"><crowd-button id="consentProceed" onClick="consented()">Proceed</crowd-button></div>
</div>
<!-- Estimation page -->
<div class="grayscreen" id="estimationDiv">
<div id="textbox"><p id="toptextLeft">TEST</p></div>
<div id="showimage"><img src="https://imsexp.s3.amazonaws.com/white.jpg" id="imagetrial"></div>
<h4>Estimate the height of this building (in feet):</h4>
<input type="number" name="estimate" id="estimate" oninput="validateanswer()">
<div class="buttonplacement"><crowd-button id="estimateProceed" onClick="doExperimentTrials( 'nexttrial' )">Proceed</crowd-button></div>
</div>
<div class="grayscreen" id="submitDiv">
<h4>You have reached the end of the experiment. Press the button to submit your HIT.</h4>
<div class="buttonplacement"><crowd-button form-action="submit" id="submitbutton">Submit</crowd-button></div>
</div>
<script>
// List of images and path (global variables)
var impath = 'https://steyvers.socsci.uci.edu/files/2022/03/';
var trialList = [ { 'Im':'building1.jpg', 'TrueAns':2717 },
{ 'Im':'building2.jpg', 'TrueAns':2227 },
{ 'Im':'building3.jpg', 'TrueAns':2073 },
{ 'Im':'building4.jpg', 'TrueAns':1966 } ];
function myconsolelog( str ) {
if (doDebug==1) console.log( str );
}
function validateanswer() { // if the answer is valid, enable the proceed button
estimate = $( '#estimate').val();
if (estimate > 0) $( '#estimateProceed').prop('disabled', false); else
$( '#estimateProceed').prop('disabled', true);
}
function shuffle(o){ /* Fisher-Yates shuffle */
for(var j, x, i = o.length; i; j = Math.floor(Math.random() * i), x = o[--i], o[i] = o[j], o[j] = x); return o;
}
function reorder(arr, index, n) { // Function to reorder n elements of arr[] according to index[]
var temp = [...Array(n)];
for (var i = 0; i < n; i++) temp[index[i]] = arr[i];
for (var i = 0; i < n; i++) arr[i] = temp[i];
}
function doExperimentTrials( whphase ) {
if (whphase=="start") { // initialize the experiment; randomize trials
myconsolelog( 'Starting doExperimentTrials...')
trialNow = 0; // we omit "var" in front of this variable and it becomes a global variable automatically
nTrialsTotal = trialList.length;
// Randomize order of trials
var Order = new Array( nTrialsTotal );
for (let i=0; i<nTrialsTotal; i++) Order[ i ] = i;
Order = shuffle( Order );
reorder( trialList, Order, nTrialsTotal );
// Show first image
imageNow = trialList[ trialNow ].Im;
trueAns = trialList[ trialNow ].TrueAns;
$('#imagetrial').attr( "src" , impath + imageNow ); // load the current image
$('#toptextLeft').text( 'Trial ' + (trialNow+1) + ' of ' + nTrialsTotal ); // show trial counter
$( '#estimateProceed').prop('disabled', true); // disable proceed button
// Show estimation page
$( '#estimationDiv').show();
} else if (whphase=="nexttrial") {
// Store the estimate from the last trial in a hidden form input along with trial information
var newElement = '<input type="hidden" name=image' + (trialNow+1) + ' value="' + imageNow + '">';
$('#addhere').append( newElement );
var newElement = '<input type="hidden" name=trueAns' + (trialNow+1) + ' value="' + trueAns + '">';
$('#addhere').append( newElement );
estimate = $( '#estimate').val();
var newElement = '<input type="hidden" name=estimate' + (trialNow+1) + ' value="' + estimate + '">';
$('#addhere').append( newElement );
myconsolelog( ' Estimate=' + estimate + ' True answer=' + trueAns);
// Increase trial Counter
trialNow++;
myconsolelog( 'doExperimentTrials. Trial ' + trialNow)
if (trialNow==nTrialsTotal) { // we have reached the end of the experiment
$( '#estimationDiv').hide();
$( '#submitDiv').show();
} else {
$( '#estimate').val(''); // reset the input from the last trial
$( '#estimateProceed').prop('disabled', true); // disable proceed button
imageNow = trialList[ trialNow ].Im; // Get the next image
trueAns = trialList[ trialNow ].TrueAns;
$('#imagetrial').attr( "src" , impath + imageNow );
$('#toptextLeft').text( 'Trial ' + (trialNow+1) + ' of ' + nTrialsTotal );
}
}
}
function consented() {
$( '#consentDiv').hide();
doExperimentTrials( "start" );
}
// This is the start of the program -- we reveal the consent/information page
$( '#consentDiv').show();
// Needed to prevent enter key press from submitting the form
$(document).on("keydown", "form", function(event) {
return event.key != "Enter";
});
</script>
</crowd-form>
The new function doExperimentTrials
on line 84 codes the logic of the experiment. It accepts a string value whphase
that signals what activity we want to execute. When whphase == "start"
, we start initializing the experiment by randomizing the order of the trials, setting up the screen, and revealing the first trial. When whphase=="nexttrial"
, we first store the participant’s response before proceeding to the next trial of the sequence. Of course, we can associate different functions for initializing the experimental sequence and going to the next trial. This is a matter of personal preference. I find it convenient to see all the code associated with the experimental trial sequence in one place so that it becomes easier to make changes in the code.
Lines 106-114 store the participant’s response for each trial in a hidden form element that will not be visible on the screen, but will be part of the information that gets saved by AMT. These hidden form elements are actually added to the html code under the section labeled addhere
. When you debug the JavaScript code, you can see that the html is changing. Open the elements tab (in the developer tools) and go to the html code under the addhere
section. You will see the new form elements are added after each response is made.
The main data structure to organize the information about individual trials (e.g., the names of images and answer key) is an array of objects on line 63. This is a convenient data structure. When we randomize the order of the array on lines 91-94, all the object fields that correspond to the same trial stay together. For your experiment, you can expand this data structure to contain any information related to a particular trial/image, such as the particular condition associated with the trial.
Generally, it is a good idea to validate the response from the participant and prevent the participant from entering invalid responses or allow the participant to skip a trial. In this experiment, the participant cannot proceed to the next trial without typing in some number. The validation works by attaching a callback function validateanswer()
on line 50 that responds to changes in the input. If the current input from the user is valid, the proceed button is enabled. Otherwise, it is disabled.
Preventing a worker from participating more than once
When posting a HIT with multiple assignments (e.g., a batch of 10 participants), AMT prevents a worker from doing multiple assignments within that HIT. However, once you post a new HIT (e.g., a new batch of 10 participants), AMT has no way of preventing a worker that participated previously in one HIT from participating in the new HIT. From the perspective of the worker, they cannot easily tell if the experiment is the same as what they have done before, and if the pay is appealing and the HIT is not too boring, some workers will be tempted to participate in the same type of HIT again.
To solve this problem, we need to create some code to prevent workers from doing the same experiment again. In the code below, we use an external database, based on google Firebase, to keep track of the number of times that a worker (based on their workerid) has participated previously. If we find out a worker has participated previously, we don’t allow the worker to proceed with the HIT.
With this approach, we deviated from the principle stated earlier of keeping things simple and not involving servers to keep track of information. However, once you have this code up and running, it will translate easily to new experiments. Also, while it does take a bit of work to set up the Firebase on your end (although it is not too bad), it will probably still be easier than setting up your own server to manage your experiment and data storage.
<!-- Code listing 5 -->
<script>const doDebug = 1;</script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.0/jquery.min.js"></script>
<script src="https://assets.crowd.aws/crowd-html-elements.js"></script>
<script src="https://www.gstatic.com/firebasejs/7.14.1/firebase-app.js"></script>
<script src="https://www.gstatic.com/firebasejs/7.14.1/firebase-database.js"></script>
<style>
body {
font-family: Arial, Helvetica; font-size: 12pt;
-webkit-user-select: none; /* Safari */
-moz-user-select: none; /* Firefox */
-ms-user-select: none; /* IE10+/Edge */
user-select: none; /* Standard */
}
span.nobr { white-space: nowrap; }
.grayscreen {
border: 2px solid gray; background: #EEEEEE;
padding-top: 20px; padding-bottom: 20px; padding-left: 50px; padding-right: 50px;
width: 600px; margin: 0 auto;
display: none;
}
.buttonplacement {
width: 120px; padding: 5px;
margin-left: auto; margin-right: 20px; margin-top: 30px;
}
#toptextLeft {
color: #4f4f4f;
padding-bottom: 20px;
}
img {
max-height: 400px;
height: 50%;
width: auto;
}
</style>
<crowd-form answer-format="flatten-objects">
<!-- Tag for placing hidden form values -->
<div id="addhere"></div>
<!-- Consent page -->
<div class="grayscreen" id="consentDiv">
<h3>Participant Information</h3>
<p>Welcome to the experiment! Are you ready to participate?</p>
<div class="buttonplacement"><crowd-button id="consentProceed" onClick="consented()">Proceed</crowd-button></div>
</div>
<!-- Estimation page -->
<div class="grayscreen" id="estimationDiv">
<div id="textbox"><p id="toptextLeft">TEST</p></div>
<div id="showimage"><img src="https://imsexp.s3.amazonaws.com/white.jpg" id="imagetrial"></div>
<h4>Estimate the height of this building (in feet):</h4>
<input type="number" name="estimate" id="estimate" oninput="validateanswer()">
<div class="buttonplacement"><crowd-button id="estimateProceed" onClick="doExperimentTrials( 'nexttrial' )">Proceed</crowd-button></div>
</div>
<div class="grayscreen" id="submitDiv">
<h4>You have reached the end of the experiment. Press the button to submit your HIT.</h4>
<div class="buttonplacement"><crowd-button form-action="submit" id="submitbutton">Submit</crowd-button></div>
</div>
<div class="grayscreen" id="disallowworker">
<h4>NOTE</h4><p>You have participated previously in this study. A precondition for this study is that participants have not engaged in this study before (in similar other HITS).</p>
</div>
<script>
// Firebase configuration
var firebaseConfig = {
apiKey: "INSERT_YOUR_INFORMATION",
authDomain: "INSERT_YOUR_INFORMATION",
databaseURL: "INSERT_YOUR_INFORMATION",
projectId: "INSERT_YOUR_INFORMATION",
storageBucket: "INSERT_YOUR_INFORMATION",
messagingSenderId: "INSERT_YOUR_INFORMATION",
appId: "INSERT_YOUR_INFORMATION"
};
firebase.initializeApp(firebaseConfig); // Initialize Firebase
var workerCount = -1; // -1 not yet determined; >= 1 number of times worker participated
var databaseRef = 'INSERT_YOUR_PATH_TO_COUNTS_NODE';
// List of images and path (global variables)
var impath = 'https://steyvers.socsci.uci.edu/files/2022/03/';
var trialList = [ { 'Im':'building1.jpg', 'TrueAns':2717 },
{ 'Im':'building2.jpg', 'TrueAns':2227 },
{ 'Im':'building3.jpg', 'TrueAns':2073 },
{ 'Im':'building4.jpg', 'TrueAns':1966 } ];
var waitForWorkerCount = function(callback) {
if (workerCount > -1) {
callback();
} else {
setTimeout(function() {
waitForWorkerCount( callback);
}, 100);
}
};
function getworkerID() {
// Extract the workerId; if no worker id can be found (e.g., because experiment is tested outside of AMT, the worker id is assigned "test")
workerId = "test"; assignmmentId = "test"; // Assign a temporary ID
const queryString = window.location.search; // example: "?assignment_id=33LKR6A5KEKWJYEXALGJHPIGL181TQ&auto_accept=true"
if ( !(queryString == null)) {
myconsolelog('Urlparams method; query='+queryString);
const urlParams = new URLSearchParams(queryString);
if ( !(urlParams == null)) {
workerId = urlParams.get('workerId');
assignmentId = urlParams.get('assignmentId');
if ( !(workerId == null)) {
myconsolelog( 'Managed to extract WorkerID = ' + workerId );
} else workerId = "test";
if ( !(assignmentId == null)) {
myconsolelog( 'Managed to extract AssignmentID = ' + assignmentId );
} else assignmentId = "test";
}
}
myconsolelog( 'Final WorkerID = ' + workerId );
myconsolelog( 'Final AssignmentID = ' + assignmentId );
}
function getWorkerCount() {
// Asynchronous Function
//firebase.database.enableLogging(true);
myconsolelog("Retrieving worker count...." );
var ref = firebase.database().ref(databaseRef+'/'+workerId);
ref.once("value", function(snapshot) {
workerCount = snapshot.val();
if (workerCount == null) workerCount = 0;
myconsolelog("Retrieved worker count: " + workerCount );
}, function (error) {
workerCount = 0;
myconsolelog("Error in retrieving count: " + error.code);
myconsolelog("Assuming worker count: " + workerCount );
});
}
function updateWorkerIDCounts() {
myconsolelog( 'Updating workerID database counts for ' + workerId );
// Increment a count
firebase
.database()
.ref(databaseRef)
.child( workerId )
.set(firebase.database.ServerValue.increment(1));
}
function myconsolelog( str ) {
if (doDebug==1) console.log( str );
}
function validateanswer() { // if the answer is valid, enable the proceed button
estimate = $( '#estimate').val();
if (estimate > 0) $( '#estimateProceed').prop('disabled', false); else
$( '#estimateProceed').prop('disabled', true);
}
function shuffle(o){ /* Fisher-Yates shuffle */
for(var j, x, i = o.length; i; j = Math.floor(Math.random() * i), x = o[--i], o[i] = o[j], o[j] = x); return o;
}
function reorder(arr, index, n) { // Function to reorder n elements of arr[] according to index[]
var temp = [...Array(n)];
for (var i = 0; i < n; i++) temp[index[i]] = arr[i];
for (var i = 0; i < n; i++) arr[i] = temp[i];
}
function doExperimentTrials( whphase ) {
if (whphase=="start") { // initialize the experiment; randomize trials
myconsolelog( 'Starting doExperimentTrials...')
trialNow = 0; // we omit "var" in front of this variable and it becomes a global variable automatically
nTrialsTotal = trialList.length;
// Randomize order of trials
var Order = new Array( nTrialsTotal );
for (let i=0; i<nTrialsTotal; i++) Order[ i ] = i;
Order = shuffle( Order );
reorder( trialList, Order, nTrialsTotal );
// Show first image
imageNow = trialList[ trialNow ].Im;
trueAns = trialList[ trialNow ].TrueAns;
$('#imagetrial').attr( "src" , impath + imageNow ); // load the current image
$('#toptextLeft').text( 'Trial ' + (trialNow+1) + ' of ' + nTrialsTotal ); // show trial counter
$( '#estimateProceed').prop('disabled', true); // disable proceed button
// Show estimation page
$( '#estimationDiv').show();
} else if (whphase=="nexttrial") {
// Store the estimate from the last trial in a hidden form input along with trial information
var newElement = '<input type="hidden" name=image' + (trialNow+1) + ' value="' + imageNow + '">';
$('#addhere').append( newElement );
var newElement = '<input type="hidden" name=trueAns' + (trialNow+1) + ' value="' + trueAns + '">';
$('#addhere').append( newElement );
estimate = $( '#estimate').val();
var newElement = '<input type="hidden" name=estimate' + (trialNow+1) + ' value="' + estimate + '">';
$('#addhere').append( newElement );
myconsolelog( ' Estimate=' + estimate + ' True answer=' + trueAns);
// Assume that the worker is now committed -- update the database for this worker
// After this point, the worker cannot do the experiment for a second time
updateWorkerIDCounts();
// Increase trial Counter
trialNow++;
myconsolelog( 'doExperimentTrials. Trial ' + trialNow)
if (trialNow==nTrialsTotal) { // we have reached the end of the experiment
$( '#estimationDiv').hide();
$( '#submitDiv').show();
} else {
$( '#estimate').val(''); // reset the input from the last trial
$( '#estimateProceed').prop('disabled', true); // disable proceed button
imageNow = trialList[ trialNow ].Im; // Get the next image
trueAns = trialList[ trialNow ].TrueAns;
$('#imagetrial').attr( "src" , impath + imageNow );
$('#toptextLeft').text( 'Trial ' + (trialNow+1) + ' of ' + nTrialsTotal );
}
}
}
function consented() {
$( '#consentDiv').hide();
doExperimentTrials( "start" );
}
// Needed to prevent enter key press from submitting the form
$(document).on("keydown", "form", function(event) {
return event.key != "Enter";
});
// This is the start of the program
getworkerID(); // Get information about worker
getWorkerCount();
waitForWorkerCount( function() {
if ( (workerCount > 0) && (workerId != 'test' )) {
// Disallow participant
$('#disallowworker').show();
} else
{
// Allow participant to start the experiment
$( '#consentDiv').show();
}
});
</script>
</crowd-form>
TBW
Balancing between-subject designs
TBW