User Reference:StimulusPresentationTask: Difference between revisions

From BCI2000 Wiki
Jump to navigation Jump to search
No edit summary
 
(44 intermediate revisions by 2 users not shown)
Line 1: Line 1:
==Summary==
==Synopsis==
The main purpose of this task is to present a sequential series of either auditory or visual stimuli to the user of the BCI system. The sequence and nature of the stimuli can be defined by the investigator. In addition to stimulus delivery, the task can optionally be used in conjunction with BCI2000's P300 Signal Processing module (P3SignalProcessing.exe) to provide feedback to a selected stimulus in either a copy or a free mode.
The main purpose of this task is to present a sequential series of auditory and/or visual stimuli to the user of the BCI system.
 
The StimulusPresentationTask is suited to implement evoked response (ERP) paradigms, as well as to display tasks for selective activation experiments, such as SMR screening sessions.
 
The sequence and nature of the stimuli can be defined by the investigator. In addition to stimulus delivery, the task can optionally be used in conjunction with BCI2000's P300 Signal Processing module ([[User_Reference:P3TemporalFilter|P3TemporalFilter]]) to provide feedback to a selected stimulus in either a copy or a free mode.


==Functional Description==
==Functional Description==
Line 7: Line 11:


Stimuli are set up through a parameter defined by the application module. This implicitly defines the total number of stimuli as well as the details of each stimulus.
Stimuli are set up through a parameter defined by the application module. This implicitly defines the total number of stimuli as well as the details of each stimulus.
Each stimulus is defined by the following properties:
Each stimulus is defined by the following properties:
    
    
Line 12: Line 17:
#Icon file  
#Icon file  
#Audio file  
#Audio file  
#An optional individual Stimulus Duration ("on time")


In addition to stimuli that are part of the actual stimulation sequence, the <tt>FocusOn</tt>
In addition to stimuli that are part of the actual stimulation sequence, the [[#FocusOn|FocusOn]]
and <tt>Result</tt> parameters contain definitions for a stimulus that indicates what to focus on,
and [[#Result|Result]] parameters contain definitions for a stimulus that indicates what to focus on,
and a stimulus that presents the result. These stimuli are only used when the task is set to copy or free mode.  
and a stimulus that presents the result. These stimuli are only used when the task is set to copy or free mode.  


The following table contains an example definition of two stimuli; for clarity, the [[Parameter Definition|parameter definition line]] defining the associated parameter is also given:
The following table contains an example definition of two stimuli:


{|border="1"
{|border="1"
|              ||focuson              ||result            ||stimulus1         ||stimulus2
|              ||stimulus1 ||stimulus2
|-
|-
|caption      ||Please focus on      ||The result was    ||Donkey || &nbsp
|caption      ||Donkey || &nbsp;
|-
|-
|icon          ||icons\focus on.bmp  ||icons\result.bmp  ||icons\donkey.bmp  ||icons\elephant.bmp
|icon          ||images\donkey.bmp  ||images\elephant.bmp
|-
|-
|audio        ||wav\focus on.wav    || wav\result.wav  || wav\snicker.wav  ||wav\trumpet.wav
|audio        || sounds\snicker.wav  ||sounds\trumpet.wav
|}  
|}  


A blank entry for caption/icon/audio file is accepted, and defines that no presentation of the respective element takes place (e.g., see caption in stimulus2). The stimulus definition parameter does ''not'' contain a description on how the stimuli are presented.
For further details, see the [[#Stimuli|Stimuli]] parameter description.


"P3AV matrix Matrix= "
====Stimulus Codes====
  "{caption icon audio OnTime} " // row labels
When defining a stimulus sequence, stimuli are referred to an integer ID called ''stimulus code''.
  "{stimulus1 stimulus2} "      // column labels
The stimulus code associated with a stimulus corresponds to the column in which that stimulus is defined in the [[#Stimuli|Stimuli]] matrix parameter.
  "Donkey icons\\donkey.bmp  wav\\snicker.wav 10 " // stimulus1
  "%      icons\\elephant.bmp wav\\trumpet.wav 30 " // stimulus2


''Comments:''  In order to specify individual stimulus durations, a row labelled <tt>OnTime</tt> may be added to the <tt>Matrix</tt> parameter. In this case, the global <tt>OnTime</tt> parameter will be ignored, and a duration for each stimulus must be given in the <tt>OnTime</tt> row.
In the recorded data file, stimulus presentation is indicated by the [[#StimulusCode|StimulusCode]] state.
 
During presentation of a stimulus, this state is set to the associated stimulus code.
The stimulus properties might contain white spaces. A caption/icon/audio file will not be presented, if it is not defined (e.g., see caption in stimulus2). The stimulus definition parameter does ''not''  contain a description on how the stimuli are presented. Stimulus numbers start at 1. Relative paths are interpreted relative to the application module's executable location.
(In general, this not the application module's working directory at runtime but the path portion of the first argument to <tt>main()</tt>, interpreted relative to the runtime working directory.)


===Stimulus Sequence===
===Stimulus Sequence===


Stimuli are presented in a certain sequence. This sequence can either be deterministic, i.e., defined by the investigator, or random.  
Stimuli are presented in a certain sequence. This sequence can either be deterministic, i.e., defined by the investigator, or pseudo-random.  


====Deterministic Sequence:====
====Deterministic Sequence:====
Line 58: Line 60:


The investigator defines absolute stimulus frequencies for each stimulus, with the sum <math>N</math> of those values equaling the total number of stimulus presentations in the final sequence. The resulting random sequence is obtained by applying a random permutation to an arbitrary sequence that reproduces the given frequencies, with all <math>N!</math> index permutations being equally probable (Block Randomization).
The investigator defines absolute stimulus frequencies for each stimulus, with the sum <math>N</math> of those values equaling the total number of stimulus presentations in the final sequence. The resulting random sequence is obtained by applying a random permutation to an arbitrary sequence that reproduces the given frequencies, with all <math>N!</math> index permutations being equally probable (Block Randomization).
(Block randomization routines, as well as a pseudo random generator that is better than the one available via <tt>rand()</tt>, are available in the P3 Speller module.)


As an example:  
As an example:  
Line 68: Line 68:


Multiple sequences can be generated from the given frequencies. The investigator can define how many sequences are generated and presented.
Multiple sequences can be generated from the given frequencies. The investigator can define how many sequences are generated and presented.
====P3Speller Compatible====
Sequences are random, with each stimulus being presented ''NumberOfSequences'' times, in a block-randomized fashion. When in copy mode, each entry in the ''ToBeCopied'' parameter corresponds to a ''NumberOfSequences''-times repetition of all stimulus codes. In other ''InterpretMode''s, stimulus presentation is repeated indefinitely. This mode is intended as a compatibility mode for transitioning experiments from or to the [[User Reference:P3SpellerTask|P3SpellerTask]].


===Stimulus Delivery===
===Stimulus Delivery===


For any stimulus, delivery occurs simultaneously for caption, icon, and audio. (A caption, if defined, always appears in front on an icon.) (A computer can only execute commands in sequence, but the time difference between start of presentation of caption, icon, and audio, is negligible). ''Comment:''  A knowledgeable investigator has to understand the implications of audio files
For any stimulus, delivery occurs simultaneously for caption, icon, and audio.  
that are of unequal length!
When both caption and icon are defined, the caption appears overlaying the icon.


An investigator can specify:
An investigator can specify:
    
    
*Size and position of the target window (using the same scheme/parameters as used by the RJB task, Oddball paradigm, or P3 speller).  
*[[#WindowWidth|Size and position]] of the target window.  


*Width and height of caption and icon in percent of screen width/height. <br />(Using, e.g., the <tt>TImage::Stretch</tt> property for scaling.)
*[[#StimulusWidth|Width and height]] of caption and icon in percent of screen width/height, or that the icon should appear in its original pixel size.


*Whether captions, icons, or audio files will be presented (i.e., a global switch). There are no individual switches for each stimulus. However, individual captions/icons/wave files are not presented if they are not defined (i.e., their respective entries are blank).  
*Whether captions, icons, or audio files will be presented (i.e., a [[#IconSwitch|global switch]]). There are no individual switches for each stimulus. However, individual captions/icons/wave files are not presented if they are not defined (i.e., their respective entries are blank).


*The volume for audio playback as a float value between 0 and 1.  
*The [[#AudioVolume|volume for audio playback]] as a percentage of maximum volume.  


*Window background color in RGB.<br />(For convenience, RGB values may be entered in hexadecimal notation, e.g. <tt>0xff0000</tt> for red.)  
*[[#BackgroundColor|Window background color]] in RGB.<br />(For convenience, RGB values may be entered in hexadecimal notation, e.g. <tt>0xff0000</tt> for red.)  


*Caption color in RGB.
*[[#CaptionColor|Caption color]] in RGB.
   
   
*The duration during which a stimulus is presented in units of SampleBlocks.<br />(Playback of audio extending above the specified duration will be muted.)
*The [[#StimulusDuration|duration]] of stimulus presentation.<br />(Playback of audio extending above the specified duration will be muted.)
   
   
*The duration of an inter-stimulus interval following stimulus presentation in units of SampleBlocks.<br />(During the inter-stimulus interval, the screen is blank and audio is turned off.)  
*The duration of an [[#ISIMinDuration|inter-stimulus interval]] following stimulus presentation.<br />(During the inter-stimulus interval, the screen is blank and audio is turned off.)  


*A minimum and maximum time (in units of SampleBlocks) that will be added randomly to the inter-stimulus interval, with probability distributed uniformly over time intervals. If these variables are both set to 0, the actual inter-stimulus interval will always be exactly as defined above. If these variables are set to, for example, 0 and 3, inter-stimulus intervals will randomly be longer by between 0 and 3 units, with a probability of <math>1/4</math> for the occurrence of any one of the four time intervals possible. 
*Variance in inter-stimulus intervals, with probability distributed uniformly over the interval between minimum and maximum inter-stimulus interval.
 
*''Comment:'' A user can enter comments to the specific run in a string parameter.  


*For documentation purposes, a user can enter a [[#UserComment|comment]] to the specific run in a string parameter.


==Processing of Classification Results==
==Processing of Classification Results==


The task can be configured to interpret results communicated to it by the P3 Signal Processing module. These results represent a judgment of which of the stimuli was most likely selected. Handling of these results is identical to the P3 Spelling Task.  
The task can be configured to interpret results communicated to it by the [[User Reference:P3TemporalFilter|P3 Signal Processing]] module. These results represent a judgment of which of the stimuli was most likely selected. Handling of these results is identical to the [[User Reference:P3SpellerTask|P3 Spelling Task]].  


When it transmits a classification result, Signal Processing sets the state ''StimulusCodeRes'' to the stimulus code that was originally transmitted to it by the user application. For example, when Signal Processing sets ''StimulusCodeRes''  to 3, it transmits classification results for stimulus 3. In addition, it sets ''StimulusTypeRes''  to reflect the type of stimulus  
When it transmits a classification result, Signal Processing sets the state ''StimulusCodeRes'' to the stimulus code that was originally transmitted to it by the user application. For example, when Signal Processing sets ''StimulusCodeRes''  to 3, it transmits classification results for stimulus 3. In addition, it sets ''StimulusTypeRes''  to reflect the type of stimulus  
(0=non-target, 1=target) when the system is in copy mode. Signal Processing transmits the classication result as one number (i.e., the first control signal).
(0=non-target, 1=target) when the system is in copy mode. Signal Processing transmits the classification result as one number (i.e., the first control signal).


===Free Mode===
===Free Mode===
Line 119: Line 121:
Copy mode is similar to free mode. In copy mode, the investigator can define a list of stimuli to be copied (e.g., "3 5 4"). In this example, the user has to attend to stimulus 3 for the first sequence, 5 for the second sequence, etc.
Copy mode is similar to free mode. In copy mode, the investigator can define a list of stimuli to be copied (e.g., "3 5 4"). In this example, the user has to attend to stimulus 3 for the first sequence, 5 for the second sequence, etc.


In addition to presenting the result, the delivery of stimuli is preceded by a presentation that describes the stimulus to which the user must attend. This presentation uses the stimulus that is defined in the <tt>focuson</tt> column of the stimuli parameter. This presentation is followed by delivery of the desired target stimulus. As an example, the system might say "Please focus on" ... "yes," before it starts with the sequence of stimulus deliveries.
In addition to presenting the result, the delivery of stimuli is preceded by a presentation that describes the stimulus to which the user must attend. This presentation uses the stimulus that is defined in the [[#FocusOn|FocusOn]] parameter. This presentation is followed by delivery of the desired target stimulus. As an example, the system might say "Please focus on" ... "yes," before it starts with the sequence of stimulus deliveries.


Copy mode terminates (i.e., suspends the task) when the user is finished copying all stimuli specified by the investigator.
Copy mode terminates (i.e., suspends the task) when the user has finished copying all stimuli specified by the investigator.


==Parameters==
==Parameters==
 
{{ApplicationWindowParams}}
The following parameters configure this module:
 
*<tt>WinXpos, WinYpos, WinWidth, WinHeight</tt>: Size and position of the target window (using the same scheme/parameters as used by the RJB task, Oddball paradigm, or P3 speller). 
 
*<tt>WinBackgroundColor</tt>: Window background color in RGB.<br />(For convenience, RGB values may be entered in hexadecimal notation, e.g. <tt>0xff0000</tt> for red.) 
 
*<tt>StimulusWidth</tt>: Width of stimulus as a percent of screen width/height. Height is deduced based on the respective icon's aspect ratio. 
 
*<tt>AudioSwitch</tt>: Whether audio files will be presented (i.e., a global switch). There are no individual switches for each stimulus. However, wave files will not be presented for individual stimuli if they are not defined. The values can be: 0(off) and 1(on). 
 
*<tt>VideoSwitch</tt>: Whether video files will be presented (i.e., a global switch). There are no individual switches for each stimulus. However, icons files will not be presented for individual stimuli if they are not defined. The values can be: 0(off) and 1(on). 
 
*<tt>CaptionSwitch</tt>: Whether captions will be presented (i.e., a global switch). There are no individual switches for each stimulus. However, captions will not be presented for individual stimuli if they are not defined. The values can be: 0(off) and 1(on).
*<tt>AudioVolume</tt>: The volume for audio playback as a float value between 0 and 1
 
*<tt>CaptionColor</tt>: Color of stimulus caption text in hex (0x00BBGGRR).
 
*<tt>CaptionHeight</tt>: Height of stimulus caption text as a percent of screen height. 
 
*<tt>OnTime</tt> The duration during which a stimulus is presented (in units of SampleBlocks)<br />(Playback of audio extending above the specified duration will be muted.)<br />The duration, in seconds, is calculated by using <tt>OnTime</tt> multiplied by <tt>SampleBlockSize</tt> (samples transmitted per time) divided by <tt>SamplingRate</tt> (samples per one second). When individual stimulus durations are given in the <tt>Matrix</tt> parameter, the value of the <tt>OnTime</tt> parameter is ignored. 
 
*<tt>OffTime</tt>: The duration of an inter-stimulus interval following stimulus presentation (in units of SampleBlocks)<br />(During the inter-stimulus interval, the screen is blank and audio is turned off.) 
 
*<tt>MinInterTime, MaxInterTime</tt>: A minimum and maximum time (in units of SampleBlocks) that will be added randomly to the inter-stimulus interval, with probability distributed uniformly over time intervals. If these variables are both set to 0, the actual inter-stimulus interval will        always be exactly as defined above.  If these variables are set to, for example, 0 and 3, inter-stimulus intervals will randomly be longer by between 0 and 3 units, with a probability of <math>1/4</math> for the occurrence of any one of the four time intervals possible.
 
*<tt>PreSequenceTime</tt>: The duration (in SampleBlocks) before a sequence is presented. During this time the following sequence occurs: In Copy Mode: the "Please, focus on ..." message will be played followed by the actual stimulus on which the subject needs to focus. In Free Mode and No-Interpretation Mode, no stimulus will be presented. The screen will be blank.
 
*<tt>PostSequenceTime</tt>: The duration (in SampleBlocks) after a sequence is presented. During this time the following sequence occurs: In Copy Mode and Free Mode: "The result was ..." message will be played followed by the predicted stimulus. In No-Interpretation Mode: during this time nothing will be presented, just a plain blank screen.
 
*<tt>Matrix</tt>: Defines all the possible stimuli which can be presented on the screen. See description in the Stimulus Definition section.
 
*<tt>FocusOn</tt>: Defines a stimulus that is used to present the "please, focus on .." message.
 
*<tt>Result</tt>: Defines a stimulus that is used to present the "the result is .." message.
 
*<tt>Sequence</tt>: Sequence of stimuli that are presented to the user (deterministic type) / Frequencies for each stimulus (random sequence). Stimuli are defined as column indices in the  <tt>Matrix</tt> parameter.
 
*<tt>SequenceType</tt>: Specifies whether the sequence is deterministic (value 0) or random (value 1).
 
*<tt>NumberOfSeq:</tt> Number of sequences being presented to the user in a single run.
 
*<tt>InterpretMode</tt>: None (value 0, no interpretation)/Free (value 1)/Copy (value 2) mode. If the interpretation mode is set to none, the task does not rely on presence of  <tt>StimulusCodeRes</tt> and <tt>StimulusTypeRes</tt> states and thus can be used with other signal processing modules.
 
*<tt>ToBeCopied</tt>: A list of stimuli that user needs to copy (only in copy mode). 
 
*<tt>UserComment:</tt> A user can enter comments to the specific run in a string parameter.
 
The following conditions need to be satisfied:
 
*Sound card has to be present if wave files are defined and global audio switch is on.
 
*All the parameters have to be present.
 
*<tt>MinInterTime</tt> has to be less than <tt>MaxInterTime</tt>.
 
*<tt>PreSequenceTime</tt> and <tt>PostSequenceTime</tt> have to be at least 2 times larger than <tt>OnTime</tt>. This follows from the fact that during presequence and postsequence there will be 2 stimuli presented: one is for "please, focus on .."/"the result is .." and the other is for the actual name of the stimulus.
 
*The time defined by <tt>PostSequenceTime</tt> multiplied by <tt>SampleBlockSize</tt> has to be larger than the value defined by <tt>NumSamplesInERP</tt>.
 
 
==States==
 
The time line of stimulus delivery is encoded in state variables as defined in Table 1
[[Image:stimuluspresentation_states.png|center|thumb|400px|Encoding scheme for this task]]
 
==Timeline==
[[Image:stimuluspresentation_timeline.png|center|thumb|400px|Timeline]]
 
==Parameters==
{{ApplicationBaseParams}}
{{StimulusTaskParams}}
{{StimulusTaskParams}}


Line 203: Line 133:
*0 deterministic sequence mode: the sequence is explicitly defined in the ''Sequence'' parameter;  
*0 deterministic sequence mode: the sequence is explicitly defined in the ''Sequence'' parameter;  
*1 random sequence mode: the sequence is random, with pre-defined stimulus frequencies.
*1 random sequence mode: the sequence is random, with pre-defined stimulus frequencies.
*2 P3Speller compatible mode: the sequence is random, with each stimulus being presented exactly ''NumberOfSequences'' times, in a block-randomized fashion. In copy spelling mode, one such sequence is presented for each entry in the ''ToBeCopied'' parameter. In other ''InterpretMode''s, such sequences are repeated indefinitely; the current run must be stopped manually.


====Sequence====
====Sequence====
In deterministic mode, a list of stimulus codes defining the sequence of presentations.
In deterministic mode, a list of stimulus codes defining the sequence of presentations.
In random mode, a list of integer stimulus frequencies.  
In random mode, a list of integer stimulus frequencies.
In P3Speller compatible mode, this parameter is ignored.


====NumberOfSequences====
====NumberOfSequences====
Line 212: Line 144:


====ToBeCopied====
====ToBeCopied====
A list of stimulus codes defining a sequence in which stimuli are announced to be attended to.
A list of stimulus codes defining a sequence of attended stimuli.
This parameter is only used in "copy mode".
At the beginning of each presentation sequence, another entry from this list is announced as the attended stimulus (see [[#FocusOn|FocusOn]]).
 
This parameter is only used in [[#InterpretMode|copy mode]].


====UserComment====
====UserComment====
Line 223: Line 157:
For each stimulus, the following properties are defined by its row entries:
For each stimulus, the following properties are defined by its row entries:
*Caption: a text string, with its size and color depending on the ''CaptionHeight'' and ''CaptionColor'' parameters;
*Caption: a text string, with its size and color depending on the ''CaptionHeight'' and ''CaptionColor'' parameters;
*Icon: a graphic file (Windows BMP), with its size depending on the ''StimulusWidth'' parameter;
*Icon: a graphic file (a number of formats are supported through libav), with its size depending on the ''StimulusWidth'' parameter;
*Audio: an audio file (Windows WAV), with playback starting at the onset of the visual stimuli.
*AV: an AV file (video with or without audio, or an audio file), with playback starting at the onset of the visual stimuli.
For backward compatibility reasons, each of these three rows must be present.
A blank entry for caption/icon/AV is accepted, and defines that no presentation of the respective element takes place.
Note that for Windows .bmp files, the top left pixel is taken to be the transparent background color, so results may not be as expected. Try a different format, or add a 1-pixel frame in a different color to the image.
 
Additionally, a number of global stimulus parameters may be overridden with specific values for individual stimuli.
Additionally, a number of global stimulus parameters may be overridden with specific values for individual stimuli.
To do this, for each parameter to be individualized, add an additional row to the Stimuli matrix. The row label indicates the parameter to be changed, and has to be one of:
To do this, for each parameter to be individualized, add an additional row to the Stimuli matrix. The row label indicates the parameter to be changed, and has to be one of:
*StimulusDuration,  
*StimulusDuration,
*EarlyOffsetExpression,  
*ISIMinDuration, ISIMaxDuration,  
*ISIMinDuration, ISIMaxDuration,  
*StimulusWidth, CaptionHeight, CaptionColor, AudioVolume.
*StimulusWidth, CaptionHeight, CaptionColor, AudioVolume.
Line 234: Line 173:


====FocusOn====
====FocusOn====
In copy mode (see [[#InterpretMode|InterpretMode]]), the attended stimulus is presented prior to the stimulus sequence,
In copy mode (see [[#InterpretMode|InterpretMode]]), the attended stimulus is presented prior to the stimulus sequence ([[#PreSequenceTime|PreSequenceTime]]),
preceded with a special announcement stimulus.
preceded with a special announcement stimulus.
This stimulus' properties are defined by the ''FocusOn'' parameter, which is a matrix in the same format as the ''Stimuli'' parameter.
This stimulus' properties are defined by the ''FocusOn'' parameter, which is a matrix in the same format as the ''Stimuli'' parameter.
Usually, this matrix has a single column; when multiple columns are present, all stimuli are presented at the same time.
Usually, this matrix has a single column; when multiple columns are present, all stimuli are presented concurrently.
To control the duration of the "FocusOn" announcement and the attended stimulus independently of the global [[#StimulusDuration|StimulusDuration]] parameter, a ''StimulusDuration'' column may be present in the ''FocusOn'' matrix.


====Result====
====Result====
In copy and free modes (see [[#InterpretMode|InterpretMode]]), the classification result is presented, preceded with an announcement stimulus.
In copy and free modes (see [[#InterpretMode|InterpretMode]]), the classification result is presented following the sequence ([[#PostSequenceTime|PostSequenceTime]]). Presentation of the predicted stimulus is preceded with an announcement stimulus.
This stimulus' properties are defined by the ''Result'' parameter, which is a matrix in the same format as the ''Stimuli'' and ''FocusOn'' parameters.
This stimulus' properties are defined by the ''Result'' parameter, which is a matrix in the same format as the ''Stimuli'' and ''FocusOn'' parameters.
As for the "FocusOn" parameter, to control the duration of the "Result" announcement and the predicted stimulus independently of the global [[#StimulusDuration|StimulusDuration]] parameter, a ''StimulusDuration'' column may be present in the ''Result'' matrix.


====StimulusWidth====
====StimulusWidth====
For icon stimuli, stimulus width in percent of screen width. If this value is zero, all stimuli will be displayed unscaled, i.e. in their original pixel size.
For icon stimuli, stimulus width in percent of screen width.
Stimulus height is deduced from the stimulus' aspect ratio, which is always conserved.
If this parameter is zero, all stimuli will be displayed unscaled, i.e. at their original pixel size.
{{StimulusParamOverride}}
{{StimulusParamOverride}}


Line 259: Line 202:


====CaptionSwitch====
====CaptionSwitch====
A boolean parameter to switch the display of stimulus captions on or off.
A boolean parameter to globally switch presentation of stimulus captions on or off.
To present captions for individual stimuli only, remove captions for other stimuli from the [[#Stimuli|Stimuli]] matrix.


====IconSwitch====
====IconSwitch====
A boolean parameter to switch icon display on or off.
A boolean parameter to globally switch presentation of icon stimuli on or off.
To present icons for individual stimuli only, leave icon entries for other stimuli blank in the [[#Stimuli|Stimuli]] matrix.


====AudioSwitch====
====AudioSwitch====
A boolean parameter to switch audio files on or off.
A boolean parameter to globally switch presentation of audio stimuli on or off.
To present audio for individual stimuli only, remove audio entries for other stimuli from the [[#Stimuli|Stimuli]] matrix.
====AVSwitch====
A boolean parameter to globally switch presentation of AV stimuli on or off.
To present audio for individual stimuli only, remove AV entries for other stimuli from the [[#Stimuli|Stimuli]] matrix.


====AudioVolume====
====AudioVolume====
The volume for audio playback, in percent of maximum volume.
The volume for audio playback, in percent of maximum volume.
{{StimulusParamOverride}}
{{StimulusParamOverride}}
====AVPlayToEnd====
If yes, AV stimuli will be played as long as they last; if no, AV stimuli will be aborted after the specified stimulus duration has expired.


==States==
==States==
Line 283: Line 235:


==See also==
==See also==
[[P3SpellerTask]]
[[User Reference:P3TemporalFilter]], [[User Reference:P3SpellerTask]], [[Programming Reference:StimulusTask Class]]
 
[[Category:Filters]][[Category:User Application]][[Category:Specification]]

Latest revision as of 16:44, 29 March 2023

Synopsis

The main purpose of this task is to present a sequential series of auditory and/or visual stimuli to the user of the BCI system.

The StimulusPresentationTask is suited to implement evoked response (ERP) paradigms, as well as to display tasks for selective activation experiments, such as SMR screening sessions.

The sequence and nature of the stimuli can be defined by the investigator. In addition to stimulus delivery, the task can optionally be used in conjunction with BCI2000's P300 Signal Processing module (P3TemporalFilter) to provide feedback to a selected stimulus in either a copy or a free mode.

Functional Description

Stimulus Definition

Stimuli are set up through a parameter defined by the application module. This implicitly defines the total number of stimuli as well as the details of each stimulus.

Each stimulus is defined by the following properties:

  1. Caption
  2. Icon file
  3. Audio file

In addition to stimuli that are part of the actual stimulation sequence, the FocusOn and Result parameters contain definitions for a stimulus that indicates what to focus on, and a stimulus that presents the result. These stimuli are only used when the task is set to copy or free mode.

The following table contains an example definition of two stimuli:

stimulus1 stimulus2
caption Donkey  
icon images\donkey.bmp images\elephant.bmp
audio sounds\snicker.wav sounds\trumpet.wav

A blank entry for caption/icon/audio file is accepted, and defines that no presentation of the respective element takes place (e.g., see caption in stimulus2). The stimulus definition parameter does not contain a description on how the stimuli are presented. For further details, see the Stimuli parameter description.

Stimulus Codes

When defining a stimulus sequence, stimuli are referred to an integer ID called stimulus code. The stimulus code associated with a stimulus corresponds to the column in which that stimulus is defined in the Stimuli matrix parameter.

In the recorded data file, stimulus presentation is indicated by the StimulusCode state. During presentation of a stimulus, this state is set to the associated stimulus code.

Stimulus Sequence

Stimuli are presented in a certain sequence. This sequence can either be deterministic, i.e., defined by the investigator, or pseudo-random.

Deterministic Sequence:

The investigator defines the order by entering a list of stimulus IDs to be presented. As an example:

1 5 3 4 2

defines a sequence in which stimulus 1 is presented first, followed by stimulus 5, etc.

Random Sequence:

The investigator defines absolute stimulus frequencies for each stimulus, with the sum of those values equaling the total number of stimulus presentations in the final sequence. The resulting random sequence is obtained by applying a random permutation to an arbitrary sequence that reproduces the given frequencies, with all index permutations being equally probable (Block Randomization).

As an example:

6 2 3

defines a sequence of 11 stimulus presentations with stimulus 1 being presented 6 times, stimulus 2, 2 times, and stimulus 3, 3 times. The resulting sequence will be a permutation of , and the probability for to equal will be .

Multiple sequences can be generated from the given frequencies. The investigator can define how many sequences are generated and presented.

P3Speller Compatible

Sequences are random, with each stimulus being presented NumberOfSequences times, in a block-randomized fashion. When in copy mode, each entry in the ToBeCopied parameter corresponds to a NumberOfSequences-times repetition of all stimulus codes. In other InterpretModes, stimulus presentation is repeated indefinitely. This mode is intended as a compatibility mode for transitioning experiments from or to the P3SpellerTask.

Stimulus Delivery

For any stimulus, delivery occurs simultaneously for caption, icon, and audio. When both caption and icon are defined, the caption appears overlaying the icon.

An investigator can specify:

  • Width and height of caption and icon in percent of screen width/height, or that the icon should appear in its original pixel size.
  • Whether captions, icons, or audio files will be presented (i.e., a global switch). There are no individual switches for each stimulus. However, individual captions/icons/wave files are not presented if they are not defined (i.e., their respective entries are blank).
  • Window background color in RGB.
    (For convenience, RGB values may be entered in hexadecimal notation, e.g. 0xff0000 for red.)
  • The duration of stimulus presentation.
    (Playback of audio extending above the specified duration will be muted.)
  • The duration of an inter-stimulus interval following stimulus presentation.
    (During the inter-stimulus interval, the screen is blank and audio is turned off.)
  • Variance in inter-stimulus intervals, with probability distributed uniformly over the interval between minimum and maximum inter-stimulus interval.
  • For documentation purposes, a user can enter a comment to the specific run in a string parameter.

Processing of Classification Results

The task can be configured to interpret results communicated to it by the P3 Signal Processing module. These results represent a judgment of which of the stimuli was most likely selected. Handling of these results is identical to the P3 Spelling Task.

When it transmits a classification result, Signal Processing sets the state StimulusCodeRes to the stimulus code that was originally transmitted to it by the user application. For example, when Signal Processing sets StimulusCodeRes to 3, it transmits classification results for stimulus 3. In addition, it sets StimulusTypeRes to reflect the type of stimulus (0=non-target, 1=target) when the system is in copy mode. Signal Processing transmits the classification result as one number (i.e., the first control signal).

Free Mode

The task can be configured to operate in free mode. In this case, the sequence of stimulus deliveries is followed by a time period, in which the Signal Processing classification result is presented. The final classification result is the stimulus with the highest classification result.

In order to present this result, the system uses the stimulus defined in the result column of the stimuli parameter. This presentation is followed by delivery of the determined stimulus. In other words, after a sequence of stimulus deliveries, the system might play a .wav file that says: "the result is," followed by a .wav file that says "yes." (assuming "yes" represents the stimulus that produced the highest classification result).

Finally, the task sends this result to the operator module as an ASCII text message so that it appears in a log window.

Free mode does not terminate until the investigator suspends operation.

Copy Mode

Copy mode is similar to free mode. In copy mode, the investigator can define a list of stimuli to be copied (e.g., "3 5 4"). In this example, the user has to attend to stimulus 3 for the first sequence, 5 for the second sequence, etc.

In addition to presenting the result, the delivery of stimuli is preceded by a presentation that describes the stimulus to which the user must attend. This presentation uses the stimulus that is defined in the FocusOn parameter. This presentation is followed by delivery of the desired target stimulus. As an example, the system might say "Please focus on" ... "yes," before it starts with the sequence of stimulus deliveries.

Copy mode terminates (i.e., suspends the task) when the user has finished copying all stimuli specified by the investigator.

Parameters

WindowWidth, WindowHeight

The width and height of the subject-visible application window, in pixels.

WindowLeft, WindowTop

The screen position of the application window's top left corner, in pixels.

WindowBackgroundColor

The window's background color, given as an RGB value. For convenience, RGB values may be entered in hexadecimal notation, e.g. 0xff0000 for red.

PreRunDuration

The duration of the pause preceding the first sequence. Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.

PostRunDuration

Duration of the pause following last sequence. Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.

PreSequenceDuration

Duration of the pause preceding sequences (or sets of intensifications). Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.

In free or copy mode, the PreSequenceDuration and PostSequenceDuration parameters may not go below twice the value of the StimulusDuration parameters, in order to allow for presentation of FocusOn and Result announcement stimuli.

PostSequenceDuration

Duration of the pause following sequences (or sets of intensifications). Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.

When used in conjunction with the P3TemporalFilter, this value needs to be larger than the EpochLength parameter. This allows classification to complete before the next sequence of stimuli is presented.

StimulusDuration

For visual stimuli, the duration of stimulus presentation. For auditory stimuli, the maximum duration, i.e. playback of audio extending above the specified duration will be muted. Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.

EarlyOffsetExpression

Allows the specification of an Expression that is constantly monitored during stimulus presentation. When the value of the Expression transitions from zero to non-zero, the stimulus is aborted early, even if the StimulusDuration has not yet elapsed. For example, set this Expression to KeyDown==32 and start your source module with the --LogKeyboard=1 flag: then the subject will be able to advance the stimulus sequence manually by pressing the space key.

ISIMinDuration, ISIMaxDuration

Minimum and maximum duration of the inter-stimulus interval. During the inter-stimulus interval, the screen is blank, and audio is muted.

Actual inter-stimulus intervals vary randomly between minimum and maximum value, with uniform probability for all intermediate values. Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar. Note that temporal resolution is limited to a single sample block.

InterpretMode

An enumerated value selecting on-line classification of evoked responses:

  • 0: no target is announced "attended", and no classification is performed;
  • 1: online or free mode: classification is performed, but no "attended target" is defined;
  • 2: copy mode: "attended" targets are defined, classification is performed.

DisplayResults

Switches result display of copy/free spelling on or off. In the P3Speller, setting DisplayResults to 'off' will disable execution of all speller commands (such as switching matrices) as well.

MinimumEvidence

NOTE: If you are using your own classifier, this feature will not work properly unless your classifier's output matches certain criteria. Make sure to read these notes on how to use a different classifier.

By default, target selection is performed without considering the actual amount of evidence that favors the selected target over other targets. Although the selected target will always be a target with maximum classification score (i.e., evidence), other targets may have the same or a similar score. It may be useful to omit classification in such situations altogether, by specifying a minimum amount of evidence that must exist in favor of the selected target, when compared to the next-best target. When used together with the AccumulateEvidence option, this allows to dynamically control the number of stimulus presentations, by simply repeating stimulus sequences until a sufficient amount of evidence has been collected.

Setting MinimumEvidence to 0 or to a negative number will result in default behavior, i.e. there will be a target selection each time classification scores are received from the SignalProcessing module. For values greater 0, the amount of selection errors will become smaller as the value of MinimumEvidence is increased; this increases the amount of information contained in each selection. At the same time, it becomes more and more unlikely that a selection will occur at all within a certain amount of time; this decreases the amount of information transmitted per time (information flow, or bitrate). In between, a certain value will correspond to an optimal compromise between selection errors, and selection duration. At this point, the flow of information is maximized.

The meaning of the actual number entered into the MinimumEvidence parameter is relative to the amount of within-class variance present in the classification score. An evidence of 0 means a 50:50 chance for correct classification. Increasing the evidence value by two standard deviations corresponds to an improvement by a factor of roughly 88:12, four standard deviations correspond to (88:12)^2=(98:2) ... etc, approaching perfect classification as evidence increases towards infinity.

In classifier training, classifier weights may be normalized such that within-class variance is 1 (this is done by BCI2000Analysis, and recent versions of the P300Classifier tool). In this case, you may use the following equations to convert between the MinimumEvidence parameter , and the correct classification chance :

.

For large , this relationship may be approximated and expressed in terms of error probability :

.

Thus, the evidence value roughly corresponds to twice the number of leading zeros in the desired error probability, if classifier weights are normalized. Some values are provided in the following table:

Selection Error Evidence
5% 3
1% 4.6
0.5% 5.3
0.1% 6.9
0.05% 7.6
0.01% 9.2

AccumulateEvidence

By default, only those classification scores are used which have been received from the signal processing module immediately prior to classification. When AccumulateEvidence is set, classification scores are accumulated until a selection is actually performed. Typically, accumulated classification scores will have higher evidence values, such that a selection threshold set with MinimumEvidence will be eventually crossed after scores have been accumulated for some time.

This allows for dynamically choosing the number of stimulus repetitions in a P300 paradigm, by setting the number of stimulus repetitions to 1, and setting the MinimumEvidence parameter to a value greater zero.

In addition, accumulated overall evidence will not increase if there is no consistent evidence in favor of a certain target. Thus, it is possible to operate a P300 BCI in a quasi-asynchronous mode by using AccumulateEvidence, and choosing a MinimumEvidence value that is large enough to make accidental selection unlikely. In this configuration, no selection will be made unless the BCI user is actually concentrating on a target for a number of stimulus presentations, resulting in consistently accumulating evidence for that target.

NOTE: If you are using your own classifier, this feature will not work properly unless your classifier's output matches certain criteria. Make sure to read these notes on how to use a different classifier.

PhotoDiodePatch

Display a photo diode patch on the stimulus window. Recording from a photo diode located on that patch will allow triggering on actual stimulus delivery (see User_Reference:P3TemporalFilter#OnsetExpression).

PhotoDiodePatchHeight

Photo diode patch height in relative coordinates (between 0 and 1).

PhotoDiodePatchWidth

Photo diode patch width in relative coordinates (between 0 and 1).

PhotoDiodePatchLeft

Photo diode patch left position in relative coordinates (between 0 and 1).

PhotoDiodePatchTop

Photo diode patch top position in relative coordinates (between 0 and 1).

PhotoDiodePatchShape

Photo diode patch shape: 0 rectangle, 1 ellipse.

PhotoDiodePatchActiveColor

Photo diode patch color when active (RGB color in format 0xrrggbb).

PhotoDiodePatchInactiveColor

Photo diode patch color when inactive, (RGB color in format 0xrrggbb, use 0xff000000 for transparent).

PhotoDiodePatchExpression

Photo diode patch expression, determines active state by evaluating to nonnull. Defaults to StimulusBegin>0.

SequenceType

Enumerated value selecting between

  • 0 deterministic sequence mode: the sequence is explicitly defined in the Sequence parameter;
  • 1 random sequence mode: the sequence is random, with pre-defined stimulus frequencies.
  • 2 P3Speller compatible mode: the sequence is random, with each stimulus being presented exactly NumberOfSequences times, in a block-randomized fashion. In copy spelling mode, one such sequence is presented for each entry in the ToBeCopied parameter. In other InterpretModes, such sequences are repeated indefinitely; the current run must be stopped manually.

Sequence

In deterministic mode, a list of stimulus codes defining the sequence of presentations. In random mode, a list of integer stimulus frequencies. In P3Speller compatible mode, this parameter is ignored.

NumberOfSequences

The number of sequence repetitions in a run (a run corresponds to a single data file).

ToBeCopied

A list of stimulus codes defining a sequence of attended stimuli. At the beginning of each presentation sequence, another entry from this list is announced as the attended stimulus (see FocusOn).

This parameter is only used in copy mode.

UserComment

An arbitrary string intended for documentation purposes.

Stimuli

A matrix defining the properties of stimuli to be presented. Columns of the Stimuli matrix correspond to individual stimuli and their stimulus codes. For each stimulus, the following properties are defined by its row entries:

  • Caption: a text string, with its size and color depending on the CaptionHeight and CaptionColor parameters;
  • Icon: a graphic file (a number of formats are supported through libav), with its size depending on the StimulusWidth parameter;
  • AV: an AV file (video with or without audio, or an audio file), with playback starting at the onset of the visual stimuli.

For backward compatibility reasons, each of these three rows must be present. A blank entry for caption/icon/AV is accepted, and defines that no presentation of the respective element takes place. Note that for Windows .bmp files, the top left pixel is taken to be the transparent background color, so results may not be as expected. Try a different format, or add a 1-pixel frame in a different color to the image.

Additionally, a number of global stimulus parameters may be overridden with specific values for individual stimuli. To do this, for each parameter to be individualized, add an additional row to the Stimuli matrix. The row label indicates the parameter to be changed, and has to be one of:

  • StimulusDuration,
  • EarlyOffsetExpression,
  • ISIMinDuration, ISIMaxDuration,
  • StimulusWidth, CaptionHeight, CaptionColor, AudioVolume.

Whenever one of these rows is present, the corresponding global parameter will be ignored.

FocusOn

In copy mode (see InterpretMode), the attended stimulus is presented prior to the stimulus sequence (PreSequenceTime), preceded with a special announcement stimulus. This stimulus' properties are defined by the FocusOn parameter, which is a matrix in the same format as the Stimuli parameter. Usually, this matrix has a single column; when multiple columns are present, all stimuli are presented concurrently. To control the duration of the "FocusOn" announcement and the attended stimulus independently of the global StimulusDuration parameter, a StimulusDuration column may be present in the FocusOn matrix.

Result

In copy and free modes (see InterpretMode), the classification result is presented following the sequence (PostSequenceTime). Presentation of the predicted stimulus is preceded with an announcement stimulus. This stimulus' properties are defined by the Result parameter, which is a matrix in the same format as the Stimuli and FocusOn parameters. As for the "FocusOn" parameter, to control the duration of the "Result" announcement and the predicted stimulus independently of the global StimulusDuration parameter, a StimulusDuration column may be present in the Result matrix.

StimulusWidth

For icon stimuli, stimulus width in percent of screen width. Stimulus height is deduced from the stimulus' aspect ratio, which is always conserved. If this parameter is zero, all stimuli will be displayed unscaled, i.e. at their original pixel size. This parameter's value may be overridden by an additional row in the Stimuli, FocusOn, and Result matrices.

CaptionHeight

For text stimuli, the height of the stimulus' caption in percent of screen height. This parameter's value may be overridden by an additional row in the Stimuli, FocusOn, and Result matrices.

CaptionColor

For text stimuli, the color of the stimulus' caption, given as an RGB value. This parameter's value may be overridden by an additional row in the Stimuli, FocusOn, and Result matrices.

BackgroundColor

The background color of the stimulus rectangle, given as an RGB value. The height of the rectangle is defined by the CaptionHeight parameter, and its width depends on the caption's text width.

CaptionSwitch

A boolean parameter to globally switch presentation of stimulus captions on or off. To present captions for individual stimuli only, remove captions for other stimuli from the Stimuli matrix.

IconSwitch

A boolean parameter to globally switch presentation of icon stimuli on or off. To present icons for individual stimuli only, leave icon entries for other stimuli blank in the Stimuli matrix.

AudioSwitch

A boolean parameter to globally switch presentation of audio stimuli on or off. To present audio for individual stimuli only, remove audio entries for other stimuli from the Stimuli matrix.

AVSwitch

A boolean parameter to globally switch presentation of AV stimuli on or off. To present audio for individual stimuli only, remove AV entries for other stimuli from the Stimuli matrix.

AudioVolume

The volume for audio playback, in percent of maximum volume. This parameter's value may be overridden by an additional row in the Stimuli, FocusOn, and Result matrices.

AVPlayToEnd

If yes, AV stimuli will be played as long as they last; if no, AV stimuli will be aborted after the specified stimulus duration has expired.

States

PreStimulusTime

A 16-bit time stamp in the same format as the SourceTime state. This time stamp is set immediately before the application module is going to update the stimulus/feedback display. Note that a data block is saved together with the state vector that existed immediately after it was acquired. Thus, PreStimulusTime will be the prestimulus time of the block prior to the current block.

StimulusTime

A 16-bit time stamp in the same format as the SourceTime state. This time stamp is set immediately after the application module has updated the stimulus/feedback display. Note that a data block is saved together with the state vector that existed immediately after it was acquired. Thus, StimulusTime will be the stimulus time of the block prior to the current block.

PresentationRequested

This 32-bit state contains a list of ids for all presentations that have been requested during the current block. The list ends with a value of 2^31. A 32-bit presentation id is formed by combining the 16-bit SourceTime state of the presentation request's data block in the most significant bits, with the time difference between that source time stamp and the time when the presentation request happened, in the least significant bits:

PresentationID = SourceTime << 8 | (RequestTime - SourceTime) & 0xff

This definition makes presentation ids unique within the range of a minute, and allows to extract the value of the request's time stamp from its id.

PresentationDisplayed

This 32-bit state contains a list of ids for all presentations that have been displayed during the current block. The list ends with a value of 2^31. Presentation ids appear in the PresentationRequested state when presentation is requested, and subsequently in the PresentationDisplayed state when presentation has occurred.

PresentationTime

A 16-bit time stamp in the same format as the SourceTime state. For each id occurring in the PresentationDisplayed state, this state contains the time stamp of the respective update to video memory. From this point in time, up to two additional frame durations may be required before the display's surface actually shows a difference in signal. This depends on the internals of the display, and cannot be determined from software.

PresentationFrame

A 12-bit state that contains, for each id occurring in the PresentationDisplayed state, the frame number of the respective update to video memory. Frame numbers have an arbitrary offset, and wrap around at a value of 2^11. PresentationFlow.png

AudioBufferTime

A 16-bit time stamp in the same format as the SourceTime state. For each audio presentation that has occurred during the current block, this state contains the time stamp when non-zero audio data were about to enter the system's audio buffer.

AudioPresentationTime

A 16-bit time stamp in the same format as the SourceTime state. For each audio presentation that has occurred during the current block, this state contains the estimated time stamp when non-zero audio data were being played from the system's speakers or headphones. The estimate takes the length of the system's audio buffers into account but will not be able to detect external sources of delays, such as additional audio processing steps.

StimulusCode

The numerical ID of the stimulus being presented (16 bit).

StimulusType

This state is 1 during presentation of an attended stimulus, and 0 otherwise. The notion of an "attended" stimulus requires data recording in copy mode.

StimulusBegin

This state is 1 during the first block of stimulus presentation, and 0 otherwise.

PhaseInSequence

This state is 1 during pre-sequence, 2 during sequence and 3 during post-sequence (see Timeline).

PauseApplication

While this state is set to 1, no task processing occurs, i.e. the task is paused, and may be resumed by setting PauseApplication to 0.

SelectedStimulus

When classification is performed, this state contains the stimulus code of the stimulus classified as the "selected" one.

Timeline

StimulusTask Timeline.png

See also

User Reference:P3TemporalFilter, User Reference:P3SpellerTask, Programming Reference:StimulusTask Class