Inteligibility task: audio and survey text plugin!

[gustavoestivalet]

Hello, guys,

How are you going? I am fine, and doing new and cool experiments with jsPsych!

Simple question: How can I play an audio in a survey text? Or put a survey text in an audio playin?

I mean, I want to make an inteligibility task. The participant has to listen an audio and write what s/he heard. Thus, I need a mix of audio-button-response/survey-text. I could do it using both plug-ins in a procedure, but I would like to have the space to write already in the onset of the audio, thus, participants can start to write even before the audio finishes, providing better precision for the writing RT.

Thank you very much for your attention and efforts!
Best regards,
Gustavo Estivalet

[becky-gilbert]

Hi @gustavoestivalet! Sorry for the super-slow response. 😦

If you don't want the audio and survey-text parts to go one after another, then the best way to do this is to create a new plugin. If it were me, I would probably start with the audio-keyboard-response plugin, and modify it to (1) add a questions parameter (like the survey-text plugin), (2) add a form element to the page's HTML that contains each question prompt and an associated input type="text" element, (3) add a submit button to the page's that validates the response and ends the trial, (4) remove the key choices parameter and keyboard listener, and (5) edit the trial's data so that it saves each question name and text input response. Here are the relevant bits of code that I would pull from the survey-text plugin:

Add a questions parameter to plugin.info:

jsPsych/plugins/jspsych-survey-text.js

Lines 20 to 63 in e917fab

	questions: {
	type: jsPsych.plugins.parameterType.COMPLEX,
	array: true,
	pretty_name: 'Questions',
	default: undefined,
	nested: {
	prompt: {
	type: jsPsych.plugins.parameterType.STRING,
	pretty_name: 'Prompt',
	default: undefined,
	description: 'Prompt for the subject to response'
	},
	placeholder: {
	type: jsPsych.plugins.parameterType.STRING,
	pretty_name: 'Placeholder',
	default: "",
	description: 'Placeholder text in the textfield.'
	},
	rows: {
	type: jsPsych.plugins.parameterType.INT,
	pretty_name: 'Rows',
	default: 1,
	description: 'The number of rows for the response text box.'
	},
	columns: {
	type: jsPsych.plugins.parameterType.INT,
	pretty_name: 'Columns',
	default: 40,
	description: 'The number of columns for the response text box.'
	},
	required: {
	type: jsPsych.plugins.parameterType.BOOL,
	pretty_name: 'Required',
	default: false,
	description: 'Require a response'
	},
	name: {
	type: jsPsych.plugins.parameterType.STRING,
	pretty_name: 'Question Name',
	default: '',
	description: 'Controls the name of data values associated with this question'
	}
	}
	},

The preamble, button_label and autocomplete parameters will probably be useful too. But note that the preamble parameter here is similar to the prompt parameter from the audio plugin, so you just want to keep one of the two.

jsPsych/plugins/jspsych-survey-text.js

Lines 64 to 82 in e917fab

	preamble: {
	type: jsPsych.plugins.parameterType.STRING,
	pretty_name: 'Preamble',
	default: null,
	description: 'HTML formatted string to display at the top of the page above all the questions.'
	},
	button_label: {
	type: jsPsych.plugins.parameterType.STRING,
	pretty_name: 'Button label',
	default: 'Continue',
	description: 'The text that appears on the button to finish the trial.'
	},
	autocomplete: {
	type: jsPsych.plugins.parameterType.BOOL,
	pretty_name: 'Allow autocomplete',
	default: false,
	description: "Setting this to true will enable browser auto-complete or auto-fill for the form."
	}
	}

In plugin.trial, set question defaults:

jsPsych/plugins/jspsych-survey-text.js

Lines 87 to 101 in e917fab

	for (var i = 0; i < trial.questions.length; i++) {
	if (typeof trial.questions[i].rows == 'undefined') {
	trial.questions[i].rows = 1;
	}
	}
	for (var i = 0; i < trial.questions.length; i++) {
	if (typeof trial.questions[i].columns == 'undefined') {
	trial.questions[i].columns = 40;
	}
	}
	for (var i = 0; i < trial.questions.length; i++) {
	if (typeof trial.questions[i].value == 'undefined') {
	trial.questions[i].value = "";
	}
	}

Create an HTML string variable that contains the form element, question prompt(s) and text input box(es):

jsPsych/plugins/jspsych-survey-text.js

Lines 109 to 142 in e917fab

	if (trial.autocomplete) {
	html += '<form id="jspsych-survey-text-form">';
	} else {
	html += '<form id="jspsych-survey-text-form" autocomplete="off">';
	}
	// generate question order
	var question_order = [];
	for(var i=0; i<trial.questions.length; i++){
	question_order.push(i);
	}
	if(trial.randomize_question_order){
	question_order = jsPsych.randomization.shuffle(question_order);
	}
	// add questions
	for (var i = 0; i < trial.questions.length; i++) {
	var question = trial.questions[question_order[i]];
	var question_index = question_order[i];
	html += '<div id="jspsych-survey-text-'+question_index+'" class="jspsych-survey-text-question" style="margin: 2em 0em;">';
	html += '<p class="jspsych-survey-text">' + question.prompt + '</p>';
	var autofocus = i == 0 ? "autofocus" : "";
	var req = question.required ? "required" : "";
	if(question.rows == 1){
	html += '<input type="text" id="input-'+question_index+'" name="#jspsych-survey-text-response-' + question_index + '" data-name="'+question.name+'" size="'+question.columns+'" '+autofocus+' '+req+' placeholder="'+question.placeholder+'"></input>';
	} else {
	html += '<textarea id="input-'+question_index+'" name="#jspsych-survey-text-response-' + question_index + '" data-name="'+question.name+'" cols="' + question.columns + '" rows="' + question.rows + '" '+autofocus+' '+req+' placeholder="'+question.placeholder+'"></textarea>';
	}
	html += '</div>';
	}
	// add submit button
	html += '<input type="submit" id="jspsych-survey-text-next" class="jspsych-btn jspsych-survey-text" value="'+trial.button_label+'"></input>';
	html += '</form>'

Focus on the first input text box when the page loads:

jsPsych/plugins/jspsych-survey-text.js

Line 146 in e917fab

display_element.querySelector('#input-'+question_order[0]).focus();

Create an event listener for the form's submit event that validates the response (i.e. produces a pop-up on the page if a question is required but nothing is entered), saves the question names and responses, and ends the trial. You might want to add other information to the trial data here, like the audio stimulus:

jsPsych/plugins/jspsych-survey-text.js

Lines 148 to 182 in e917fab

	display_element.querySelector('#jspsych-survey-text-form').addEventListener('submit', function(e) {
	e.preventDefault();
	// measure response time
	var endTime = performance.now();
	var response_time = endTime - startTime;
	// create object to hold responses
	var question_data = {};
	for(var index=0; index < trial.questions.length; index++){
	var id = "Q" + index;
	var q_element = document.querySelector('#jspsych-survey-text-'+index).querySelector('textarea, input');
	var val = q_element.value;
	var name = q_element.attributes['data-name'].value;
	if(name == ''){
	name = id;
	}
	var obje = {};
	obje[name] = val;
	Object.assign(question_data, obje);
	}
	// save data
	var trialdata = {
	rt: response_time,
	response: question_data
	};
	display_element.innerHTML = '';
	// next trial
	jsPsych.finishTrial(trialdata);
	});
	var startTime = performance.now();
	};

In the audio plugin that you're modifying, the form/question HTML would go inside of the setupTrial function and would be added to the page somewhere around here. Instead of these lines, you would just have display_element.innerHTML = html;. (You could keep the audio plugin's prompt parameter instead of the preamble, but you would still need to change this so that the prompt is added to your larger HTML string variable that also contains the form etc.):

jsPsych/plugins/jspsych-audio-keyboard-response.js

Lines 107 to 110 in e917fab

	// show prompt if there is one
	if (trial.prompt !== null) {
	display_element.innerHTML = trial.prompt;
	}

And you could delete the audio plugin's end_trial function (since this is now handled by the form's submit event handler) as well as the keyboard listener in these lines:

jsPsych/plugins/jspsych-audio-keyboard-response.js

Lines 136 to 206 in e917fab

	// function to end trial when it is time
	function end_trial() {
	// kill any remaining setTimeout handlers
	jsPsych.pluginAPI.clearAllTimeouts();
	// stop the audio file if it is playing
	// remove end event listeners if they exist
	if (context !== null) {
	audio.stop();
	} else {
	audio.pause();
	}
	audio.removeEventListener('ended', end_trial);
	audio.removeEventListener('ended', setup_keyboard_listener);
	// kill keyboard listeners
	jsPsych.pluginAPI.cancelAllKeyboardResponses();
	// gather the data to store for the trial
	var trial_data = {
	rt: response.rt,
	stimulus: trial.stimulus,
	response: response.key
	};
	// clear the display
	display_element.innerHTML = '';
	// move on to the next trial
	jsPsych.finishTrial(trial_data);
	}
	// function to handle responses by the subject
	function after_response(info) {
	// only record the first response
	if (response.key == null) {
	response = info;
	}
	if (trial.response_ends_trial) {
	end_trial();
	}
	};
	function setup_keyboard_listener() {
	// start the response listener
	if (context !== null) {
	jsPsych.pluginAPI.getKeyboardResponse({
	callback_function: after_response,
	valid_responses: trial.choices,
	rt_method: 'audio',
	persist: false,
	allow_held_key: false,
	audio_context: context,
	audio_context_start_time: startTime
	});
	} else {
	jsPsych.pluginAPI.getKeyboardResponse({
	callback_function: after_response,
	valid_responses: trial.choices,
	rt_method: 'performance',
	persist: false,
	allow_held_key: false
	});
	}
	}
	};

Sorry if this is a bit of a pain, but it's probably not as difficult as it seems - just a lot of copy/pasting and deleting code. I think we're planning to add an audio extension in a future release (see #1640) that would allow people to play audio on top of any existing plugin, which will make this much easier.