The main purpose of this task is to present a sequential series of auditory and/or visual stimuli to the user of the BCI system.
The StimulusPresentationTask is suited to implement evoked response (ERP) paradigms, as well as to display tasks for selective activation experiments, such as SMR screening sessions.
The sequence and nature of the stimuli can be defined by the investigator. In addition to stimulus delivery, the task can optionally be used in conjunction with BCI2000's P300 Signal Processing module (P3TemporalFilter) to provide feedback to a selected stimulus in either a copy or a free mode.
Stimuli are set up through a parameter defined by the application module. This implicitly defines the total number of stimuli as well as the details of each stimulus.
Each stimulus is defined by the following properties:
- Icon file
- Audio file
In addition to stimuli that are part of the actual stimulation sequence, the FocusOn and Result parameters contain definitions for a stimulus that indicates what to focus on, and a stimulus that presents the result. These stimuli are only used when the task is set to copy or free mode.
The following table contains an example definition of two stimuli:
A blank entry for caption/icon/audio file is accepted, and defines that no presentation of the respective element takes place (e.g., see caption in stimulus2). The stimulus definition parameter does not contain a description on how the stimuli are presented. For further details, see the Stimuli parameter description.
When defining a stimulus sequence, stimuli are referred to an integer ID called stimulus code. The stimulus code associated with a stimulus corresponds to the column in which that stimulus is defined in the Stimuli matrix parameter.
In the recorded data file, stimulus presentation is indicated by the StimulusCode state. During presentation of a stimulus, this state is set to the associated stimulus code.
Stimuli are presented in a certain sequence. This sequence can either be deterministic, i.e., defined by the investigator, or pseudo-random.
The investigator defines the order by entering a list of stimulus IDs to be presented. As an example:
1 5 3 4 2
defines a sequence in which stimulus 1 is presented first, followed by stimulus 5, etc.
The investigator defines absolute stimulus frequencies for each stimulus, with the sum of those values equaling the total number of stimulus presentations in the final sequence. The resulting random sequence is obtained by applying a random permutation to an arbitrary sequence that reproduces the given frequencies, with all index permutations being equally probable (Block Randomization).
As an example:
6 2 3
defines a sequence of 11 stimulus presentations with stimulus 1 being presented 6 times, stimulus 2, 2 times, and stimulus 3, 3 times. The resulting sequence will be a permutation of , and the probability for to equal will be .
Multiple sequences can be generated from the given frequencies. The investigator can define how many sequences are generated and presented.
Sequences are random, with each stimulus being presented NumberOfSequences times, in a block-randomized fashion. When in copy mode, each entry in the ToBeCopied parameter corresponds to a NumberOfSequences-times repetition of all stimulus codes. In other InterpretModes, stimulus presentation is repeated indefinitely. This mode is intended as a compatibility mode for transitioning experiments from or to the P3SpellerTask.
For any stimulus, delivery occurs simultaneously for caption, icon, and audio. When both caption and icon are defined, the caption appears overlaying the icon.
An investigator can specify:
- Size and position of the target window.
- Width and height of caption and icon in percent of screen width/height, or that the icon should appear in its original pixel size.
- Whether captions, icons, or audio files will be presented (i.e., a global switch). There are no individual switches for each stimulus. However, individual captions/icons/wave files are not presented if they are not defined (i.e., their respective entries are blank).
- The volume for audio playback as a percentage of maximum volume.
- Window background color in RGB.
(For convenience, RGB values may be entered in hexadecimal notation, e.g. 0xff0000 for red.)
- Caption color in RGB.
- The duration of stimulus presentation.
(Playback of audio extending above the specified duration will be muted.)
- The duration of an inter-stimulus interval following stimulus presentation.
(During the inter-stimulus interval, the screen is blank and audio is turned off.)
- Variance in inter-stimulus intervals, with probability distributed uniformly over the interval between minimum and maximum inter-stimulus interval.
- For documentation purposes, a user can enter a comment to the specific run in a string parameter.
Processing of Classification Results
The task can be configured to interpret results communicated to it by the P3 Signal Processing module. These results represent a judgment of which of the stimuli was most likely selected. Handling of these results is identical to the P3 Spelling Task.
When it transmits a classification result, Signal Processing sets the state StimulusCodeRes to the stimulus code that was originally transmitted to it by the user application. For example, when Signal Processing sets StimulusCodeRes to 3, it transmits classification results for stimulus 3. In addition, it sets StimulusTypeRes to reflect the type of stimulus (0=non-target, 1=target) when the system is in copy mode. Signal Processing transmits the classification result as one number (i.e., the first control signal).
The task can be configured to operate in free mode. In this case, the sequence of stimulus deliveries is followed by a time period, in which the Signal Processing classification result is presented. The final classification result is the stimulus with the highest classification result.
In order to present this result, the system uses the stimulus defined in the result column of the stimuli parameter. This presentation is followed by delivery of the determined stimulus. In other words, after a sequence of stimulus deliveries, the system might play a .wav file that says: "the result is," followed by a .wav file that says "yes." (assuming "yes" represents the stimulus that produced the highest classification result).
Finally, the task sends this result to the operator module as an ASCII text message so that it appears in a log window.
Free mode does not terminate until the investigator suspends operation.
Copy mode is similar to free mode. In copy mode, the investigator can define a list of stimuli to be copied (e.g., "3 5 4"). In this example, the user has to attend to stimulus 3 for the first sequence, 5 for the second sequence, etc.
In addition to presenting the result, the delivery of stimuli is preceded by a presentation that describes the stimulus to which the user must attend. This presentation uses the stimulus that is defined in the FocusOn parameter. This presentation is followed by delivery of the desired target stimulus. As an example, the system might say "Please focus on" ... "yes," before it starts with the sequence of stimulus deliveries.
Copy mode terminates (i.e., suspends the task) when the user has finished copying all stimuli specified by the investigator.
The width and height of the subject-visible application window, in pixels.
The screen position of the application window's top left corner, in pixels.
The window's background color, given as an RGB value. For convenience, RGB values may be entered in hexadecimal notation, e.g. 0xff0000 for red.
The duration of the pause preceding the first sequence. Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.
Duration of the pause following last sequence. Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.
Duration of the pause preceding sequences (or sets of intensifications). Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.
In free or copy mode, the PreSequenceDuration and PostSequenceDuration parameters may not go below twice the value of the StimulusDuration parameters, in order to allow for presentation of FocusOn and Result announcement stimuli.
Duration of the pause following sequences (or sets of intensifications). Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.
When used in conjunction with the P3TemporalFilter, this value needs to be larger than the EpochLength parameter. This allows classification to complete before the next sequence of stimuli is presented.
For visual stimuli, the duration of stimulus presentation. For auditory stimuli, the maximum duration, i.e. playback of audio extending above the specified duration will be muted. Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.
Allows the specification of an Expression that is constantly monitored during stimulus presentation. When the value of the Expression transitions from zero to non-zero, the stimulus is aborted early, even if the StimulusDuration has not yet elapsed. For example, set this Expression to KeyDown==32 and start your source module with the --LogKeyboard=1 flag: then the subject will be able to advance the stimulus sequence manually by pressing the space key.
Minimum and maximum duration of the inter-stimulus interval. During the inter-stimulus interval, the screen is blank, and audio is muted.
Actual inter-stimulus intervals vary randomly between minimum and maximum value, with uniform probability for all intermediate values. Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar. Note that temporal resolution is limited to a single sample block.
An enumerated value selecting on-line classification of evoked responses:
- 0: no target is announced "attended", and no classification is performed;
- 1: online or free mode: classification is performed, but no "attended target" is defined;
- 2: copy mode: "attended" targets are defined, classification is performed.
Switches result display of copy/free spelling on or off. In the P3Speller, setting DisplayResults to 'off' will disable execution of all speller commands (such as switching matrices) as well.
NOTE: If you are using your own classifier, this feature will not work properly unless your classifier's output matches certain criteria. Make sure to read these notes on how to use a different classifier.
By default, target selection is performed without considering the actual amount of evidence that favors the selected target over other targets. Although the selected target will always be a target with maximum classification score (i.e., evidence), other targets may have the same or a similar score. It may be useful to omit classification in such situations altogether, by specifying a minimum amount of evidence that must exist in favor of the selected target, when compared to the next-best target. When used together with the AccumulateEvidence option, this allows to dynamically control the number of stimulus presentations, by simply repeating stimulus sequences until a sufficient amount of evidence has been collected.
Setting MinimumEvidence to 0 or to a negative number will result in default behavior, i.e. there will be a target selection each time classification scores are received from the SignalProcessing module. For values greater 0, the amount of selection errors will become smaller as the value of MinimumEvidence is increased; this increases the amount of information contained in each selection. At the same time, it becomes more and more unlikely that a selection will occur at all within a certain amount of time; this decreases the amount of information transmitted per time (information flow, or bitrate). In between, a certain value will correspond to an optimal compromise between selection errors, and selection duration. At this point, the flow of information is maximized.
The meaning of the actual number entered into the MinimumEvidence parameter is relative to the amount of within-class variance present in the classification score. An evidence of 0 means a 50:50 chance for correct classification. Increasing the evidence value by two standard deviations corresponds to an improvement by a factor of roughly 88:12, four standard deviations correspond to (88:12)^2=(98:2) ... etc, approaching perfect classification as evidence increases towards infinity.
In classifier training, classifier weights may be normalized such that within-class variance is 1 (this is done by recent versions of the P300Classifier tool). In this case, you may use the following equations to convert between the MinimumEvidence parameter , and the correct classification chance :
For large , this relationship may be approximated and expressed in terms of error probability :
Thus, the evidence value roughly corresponds to twice the number of leading zeros in the desired error probability, if classifier weights are normalized. Some values are provided in the following table:
By default, only those classification scores are used which have been received from the signal processing module immediately prior to classification. When AccumulateEvidence is set, classification scores are accumulated until a selection is actually performed. Typically, accumulated classification scores will have higher evidence values, such that a selection threshold set with MinimumEvidence will be eventually crossed after scores have been accumulated for some time.
This allows for dynamically choosing the number of stimulus repetitions in a P300 paradigm, by setting the number of stimulus repetitions to 1, and setting the MinimumEvidence parameter to a value greater zero.
In addition, accumulated overall evidence will not increase if there is no consistent evidence in favor of a certain target. Thus, it is possible to operate a P300 BCI in a quasi-asynchronous mode by using AccumulateEvidence, and choosing a MinimumEvidence value that is large enough to make accidental selection unlikely. In this configuration, no selection will be made unless the BCI user is actually concentrating on a target for a number of stimulus presentations, resulting in consistently accumulating evidence for that target.
NOTE: If you are using your own classifier, this feature will not work properly unless your classifier's output matches certain criteria. Make sure to read these notes on how to use a different classifier.
Display a photo diode patch on the stimulus window. Recording from a photo diode located on that patch will allow triggering on actual stimulus delivery (see User_Reference:P3TemporalFilter#OnsetExpression).
Photo diode patch height in relative coordinates (between 0 and 1).
Photo diode patch width in relative coordinates (between 0 and 1).
Photo diode patch left position in relative coordinates (between 0 and 1).
Photo diode patch top position in relative coordinates (between 0 and 1).
Photo diode patch shape: 0 rectangle, 1 ellipse.
Photo diode patch color when active (RGB color in format 0xrrggbb).
Photo diode patch color when inactive, (RGB color in format 0xrrggbb, use 0xff000000 for transparent).
Photo diode patch expression, determines active state by evaluating to nonnull. Defaults to StimulusBegin>0.
Enumerated value selecting between
- 0 deterministic sequence mode: the sequence is explicitly defined in the Sequence parameter;
- 1 random sequence mode: the sequence is random, with pre-defined stimulus frequencies.
- 2 P3Speller compatible mode: the sequence is random, with each stimulus being presented exactly NumberOfSequences times, in a block-randomized fashion. In copy spelling mode, one such sequence is presented for each entry in the ToBeCopied parameter. In other InterpretModes, such sequences are repeated indefinitely; the current run must be stopped manually.
In deterministic mode, a list of stimulus codes defining the sequence of presentations. In random mode, a list of integer stimulus frequencies. In P3Speller compatible mode, this parameter is ignored.
The number of sequence repetitions in a run (a run corresponds to a single data file).
A list of stimulus codes defining a sequence of attended stimuli. At the beginning of each presentation sequence, another entry from this list is announced as the attended stimulus (see FocusOn).
This parameter is only used in copy mode.
An arbitrary string intended for documentation purposes.
A matrix defining the properties of stimuli to be presented. Columns of the Stimuli matrix correspond to individual stimuli and their stimulus codes. For each stimulus, the following properties are defined by its row entries:
- Caption: a text string, with its size and color depending on the CaptionHeight and CaptionColor parameters;
- Icon: a graphic file (a number of formats are supported through libav), with its size depending on the StimulusWidth parameter;
- Audio: an audio file (a number of formats are supported through libav), with playback starting at the onset of the visual stimuli.
- AV: an AV file (video with or without audio), with playback starting at the onset of the visual stimuli.
A blank entry for caption/icon/audio/AV is accepted, and defines that no presentation of the respective element takes place. Note that for Windows .bmp files, the top left pixel is taken to be the transparent background color, so results may not be as expected. Try a different format, or put add a 1-pixel frame in a different color to the image.
Additionally, a number of global stimulus parameters may be overridden with specific values for individual stimuli. To do this, for each parameter to be individualized, add an additional row to the Stimuli matrix. The row label indicates the parameter to be changed, and has to be one of:
- ISIMinDuration, ISIMaxDuration,
- StimulusWidth, CaptionHeight, CaptionColor, AudioVolume.
Whenever one of these rows is present, the corresponding global parameter will be ignored.
In copy mode (see InterpretMode), the attended stimulus is presented prior to the stimulus sequence (PreSequenceTime), preceded with a special announcement stimulus. This stimulus' properties are defined by the FocusOn parameter, which is a matrix in the same format as the Stimuli parameter. Usually, this matrix has a single column; when multiple columns are present, all stimuli are presented concurrently. To control the duration of the "FocusOn" announcement and the attended stimulus independently of the global StimulusDuration parameter, a StimulusDuration column may be present in the FocusOn matrix.
In copy and free modes (see InterpretMode), the classification result is presented following the sequence (PostSequenceTime). Presentation of the predicted stimulus is preceded with an announcement stimulus. This stimulus' properties are defined by the Result parameter, which is a matrix in the same format as the Stimuli and FocusOn parameters. As for the "FocusOn" parameter, to control the duration of the "Result" announcement and the predicted stimulus independently of the global StimulusDuration parameter, a StimulusDuration column may be present in the Result matrix.
For icon stimuli, stimulus width in percent of screen width. Stimulus height is deduced from the stimulus' aspect ratio, which is always conserved. If this parameter is zero, all stimuli will be displayed unscaled, i.e. at their original pixel size. This parameter's value may be overridden by an additional row in the Stimuli, FocusOn, and Result matrices.
For text stimuli, the height of the stimulus' caption in percent of screen height. This parameter's value may be overridden by an additional row in the Stimuli, FocusOn, and Result matrices.
For text stimuli, the color of the stimulus' caption, given as an RGB value. This parameter's value may be overridden by an additional row in the Stimuli, FocusOn, and Result matrices.
The background color of the stimulus rectangle, given as an RGB value. The height of the rectangle is defined by the CaptionHeight parameter, and its width depends on the caption's text width.
A boolean parameter to globally switch presentation of stimulus captions on or off. To present captions for individual stimuli only, remove captions for other stimuli from the Stimuli matrix.
A boolean parameter to globally switch presentation of icon stimuli on or off. To present icons for individual stimuli only, leave icon entries for other stimuli blank in the Stimuli matrix.
A boolean parameter to globally switch presentation of audio stimuli on or off. To present audio for individual stimuli only, remove audio entries for other stimuli from the Stimuli matrix.
A boolean parameter to globally switch presentation of AV stimuli on or off. To present audio for individual stimuli only, remove AV entries for other stimuli from the Stimuli matrix.
The volume for audio playback, in percent of maximum volume. This parameter's value may be overridden by an additional row in the Stimuli, FocusOn, and Result matrices.
If yes, AV stimuli will be played as long as they last; if no, AV stimuli will be aborted after the specified stimulus duration has expired.
A 16-bit time stamp in the same format as the SourceTime state. This time stamp is set immediately before the application module is going to update the stimulus/feedback display. Note that a data block is saved together with the state vector that existed immediately after it was acquired. Thus, PreStimulusTime will be the prestimulus time of the block prior to the current block.
A 16-bit time stamp in the same format as the SourceTime state. This time stamp is set immediately after the application module has updated the stimulus/feedback display. Note that a data block is saved together with the state vector that existed immediately after it was acquired. Thus, StimulusTime will be the stimulus time of the block prior to the current block.
This 32-bit state contains a list of ids for all presentations that have been requested during the current block. The list ends with a value of 2^31. A 32-bit presentation id is formed by combining the 16-bit SourceTime state of the presentation request's data block in the most significant bits, with the time difference between that source time stamp and the time when the presentation request happened, in the least significant bits:
PresentationID = SourceTime << 8 | (RequestTime - SourceTime) & 0xff
This definition makes presentation ids unique within the range of a minute, and allows to extract the value of the request's time stamp from its id.
This 32-bit state contains a list of ids for all presentations that have been displayed during the current block. The list ends with a value of 2^31. Presentation ids appear in the PresentationRequested state when presentation is requested, and subsequently in the PresentationDisplayed state when presentation has occurred.
A 16-bit time stamp in the same format as the SourceTime state. For each id occurring in the PresentationDisplayed state, this state contains the time stamp of the respective update to video memory. From this point in time, up to two additional frame durations may be required before the display's surface actually shows a difference in signal. This depends on the internals of the display, and cannot be determined from software.
A 12-bit state that contains, for each id occurring in the PresentationDisplayed state, the frame number of the respective update to video memory. Frame numbers have an arbitrary offset, and wrap around at a value of 2^11.
A 16-bit time stamp in the same format as the SourceTime state. For each audio presentation that has occurred during the current block, this state contains the time stamp when non-zero audio data were about to enter the system's audio buffer.
A 16-bit time stamp in the same format as the SourceTime state. For each audio presentation that has occurred during the current block, this state contains the estimated time stamp when non-zero audio data were being played from the system's speakers or headphones. The estimate takes the length of the system's audio buffers into account but will not be able to detect external sources of delays, such as additional audio processing steps.
The numerical ID of the stimulus being presented (16 bit).
This state is 1 during presentation of an attended stimulus, and 0 otherwise. The notion of an "attended" stimulus requires data recording in copy mode.
This state is 1 during the first block of stimulus presentation, and 0 otherwise.
This state is 1 during pre-sequence, 2 during sequence and 3 during post-sequence (see Timeline).
While this state is set to 1, no task processing occurs, i.e. the task is paused, and may be resumed by setting PauseApplication to 0.
When classification is performed, this state contains the stimulus code of the stimulus classified as the "selected" one.