User Reference:P3SpellerTask

Function

The P3SpellerTask filter implements Donchin's matrix speller paradigm (Sellers, Donchin, Schalk, 2004). Such a matrix speller presents a matrix of options; rows and columns of that matrix are flashed to elicit an evoked response (ERP) whenever the user's attended matrix element is flashed. From ERP classification results, the speller then derives which of the matrix elements was attended to, and executes the action corresponding to that matrix element. Typically, this action will consist in adding a character to a text window.

Beyond the basic features, the P3SpellerTask provides the ability to handle multiple speller menus, saving and recovering the text buffer, and an option to exchange information with an external program.

Timeline

Parameters

WindowBackgroundColor

The window's background color, given as an RGB value. For convenience, RGB values may be entered in hexadecimal notation, e.g. 0xff0000 for red.

PreRunDuration

The duration of the pause preceding the first sequence. Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.

PostRunDuration

Duration of the pause following last sequence. Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.

PreSequenceDuration

Duration of the pause preceding sequences (or sets of intensifications). Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.

In free or copy mode, the PreSequenceDuration and PostSequenceDuration parameters may not go below twice the value of the StimulusDuration parameters, in order to allow for presentation of FocusOn and Result announcement stimuli.

PostSequenceDuration

Duration of the pause following sequences (or sets of intensifications). Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.

When used in conjunction with the P3TemporalFilter, this value needs to be larger than the EpochLength parameter. This allows classification to complete before the next sequence of stimuli is presented.

StimulusDuration

For visual stimuli, the duration of stimulus presentation. For auditory stimuli, the maximum duration, i.e. playback of audio extending above the specified duration will be muted. Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar.

EarlyOffsetExpression

Allows the specification of an Expression that is constantly monitored during stimulus presentation. When the value of the Expression transitions from zero to non-zero, the stimulus is aborted early, even if the StimulusDuration has not yet elapsed. For example, set this Expression to KeyDown==32 and start your source module with the --LogKeyboard=1 flag: then the subject will be able to advance the stimulus sequence manually by pressing the space key.

ISIMinDuration, ISIMaxDuration

Minimum and maximum duration of the inter-stimulus interval. During the inter-stimulus interval, the screen is blank, and audio is muted.

Actual inter-stimulus intervals vary randomly between minimum and maximum value, with uniform probability for all intermediate values. Given in sample blocks, or in time units when immediately followed with 's', 'ms', or similar. Note that temporal resolution is limited to a single sample block.

InterpretMode

An enumerated value selecting on-line classification of evoked responses:

• 0: no target is announced "attended", and no classification is performed;
• 1: online or free mode: classification is performed, but no "attended target" is defined;
• 2: copy mode: "attended" targets are defined, classification is performed.

DisplayResults

Switches result display of copy/free spelling on or off. In the P3Speller, setting DisplayResults to 'off' will disable execution of all speller commands (such as switching matrices) as well.

MinimumEvidence

NOTE: If you are using your own classifier, this feature will not work properly unless your classifier's output matches certain criteria. Make sure to read these notes on how to use a different classifier.

By default, target selection is performed without considering the actual amount of evidence that favors the selected target over other targets. Although the selected target will always be a target with maximum classification score (i.e., evidence), other targets may have the same or a similar score. It may be useful to omit classification in such situations altogether, by specifying a minimum amount of evidence that must exist in favor of the selected target, when compared to the next-best target. When used together with the AccumulateEvidence option, this allows to dynamically control the number of stimulus presentations, by simply repeating stimulus sequences until a sufficient amount of evidence has been collected.

Setting MinimumEvidence to 0 or to a negative number will result in default behavior, i.e. there will be a target selection each time classification scores are received from the SignalProcessing module. For values greater 0, the amount of selection errors will become smaller as the value of MinimumEvidence is increased; this increases the amount of information contained in each selection. At the same time, it becomes more and more unlikely that a selection will occur at all within a certain amount of time; this decreases the amount of information transmitted per time (information flow, or bitrate). In between, a certain value will correspond to an optimal compromise between selection errors, and selection duration. At this point, the flow of information is maximized.

The meaning of the actual number entered into the MinimumEvidence parameter is relative to the amount of within-class variance present in the classification score. An evidence of 0 means a 50:50 chance for correct classification. Increasing the evidence value by two standard deviations corresponds to an improvement by a factor of roughly 88:12, four standard deviations correspond to (88:12)^2=(98:2) ... etc, approaching perfect classification as evidence increases towards infinity.

In classifier training, classifier weights may be normalized such that within-class variance is 1 (this is done by BCI2000Analysis, and recent versions of the P300Classifier tool). In this case, you may use the following equations to convert between the MinimumEvidence parameter $\eta$, and the correct classification chance $\chi$:

$\chi =\frac{50}{50}{\left(\frac{88}{12}\right)}^{\eta /2}$.

For large $\chi$, this relationship may be approximated and expressed in terms of error probability $p$:

$\eta \approx -\frac{{\log }_{10}p}{{\log }_{10}e}=-2.3{\log }_{10}p\text{<mrow data-mjx-texclass="ORD">for</mrow>}p\le 5\mathrm{\%}$.

Thus, the evidence value roughly corresponds to twice the number of leading zeros in the desired error probability, if classifier weights are normalized. Some values are provided in the following table:

Selection Error	Evidence
5%	3
1%	4.6
0.5%	5.3
0.1%	6.9
0.05%	7.6
0.01%	9.2

AccumulateEvidence

By default, only those classification scores are used which have been received from the signal processing module immediately prior to classification. When AccumulateEvidence is set, classification scores are accumulated until a selection is actually performed. Typically, accumulated classification scores will have higher evidence values, such that a selection threshold set with MinimumEvidence will be eventually crossed after scores have been accumulated for some time.

This allows for dynamically choosing the number of stimulus repetitions in a P300 paradigm, by setting the number of stimulus repetitions to 1, and setting the MinimumEvidence parameter to a value greater zero.

In addition, accumulated overall evidence will not increase if there is no consistent evidence in favor of a certain target. Thus, it is possible to operate a P300 BCI in a quasi-asynchronous mode by using AccumulateEvidence, and choosing a MinimumEvidence value that is large enough to make accidental selection unlikely. In this configuration, no selection will be made unless the BCI user is actually concentrating on a target for a number of stimulus presentations, resulting in consistently accumulating evidence for that target.

NOTE: If you are using your own classifier, this feature will not work properly unless your classifier's output matches certain criteria. Make sure to read these notes on how to use a different classifier.

PhotoDiodePatch

Display a photo diode patch on the stimulus window. Recording from a photo diode located on that patch will allow triggering on actual stimulus delivery (see User_Reference:P3TemporalFilter#OnsetExpression).

PhotoDiodePatchHeight

Photo diode patch height in relative coordinates (between 0 and 1).

PhotoDiodePatchWidth

Photo diode patch width in relative coordinates (between 0 and 1).

PhotoDiodePatchLeft

Photo diode patch left position in relative coordinates (between 0 and 1).

PhotoDiodePatchTop

Photo diode patch top position in relative coordinates (between 0 and 1).

PhotoDiodePatchShape

Photo diode patch shape: 0 rectangle, 1 ellipse.

PhotoDiodePatchActiveColor

Photo diode patch color when active (RGB color in format 0xrrggbb).

PhotoDiodePatchInactiveColor

Photo diode patch color when inactive, (RGB color in format 0xrrggbb, use 0xff000000 for transparent).

PhotoDiodePatchExpression

Photo diode patch expression, determines active state by evaluating to nonnull. Defaults to StimulusBegin>0.

NumberOfSequences

The number of row/column intensifications that make up a set of intensifications. There is a 1-to-1 relation between sets of intensifications (sequences) and selection of matrix elements.

TargetDefinitions

A 5-column matrix defining elements of the speller matrix. Each row corresponds to a single matrix element; matrix elements are enumerated row-wise, beginning with the top left matrix element.

For each matrix element, the value in the first column, labeled Display, contains a text string to be displayed in the matrix element, i.e. the matrix element's caption.

The second column, labeled Enter, specifies the speller action to be performed upon the item's selection; in most cases, this action consists in entering a text string, and is specified by that string. E.g., for the top left matrix element to display the caption "A", and also enter the letter "A" on selection, both the Display and Enter columns will contain the letter "A".

The third and fourth columns, labeled Icon File and Sound File, respectively, contain the path to an icon file to be displayed, and a sound file to be played, when the matrix element is selected. For icon files, Windows BMP format is accepted; sound files must be in Windows WAV format. Paths may be absolute, or relative to the executable's working directory at startup, which usually matches the executable's location.

Speller Actions

Speller actions are specified in the TargetDefinitions matrix second column, and consist of sequences of characters, and speller control commands enclosed in a pair of <> characters. Any combination of characters and commands is allowed, and will be executed in sequence.

Speller commands may be

• <BS> (backspace) -- delete the last character from the current text.
• <DW> (delete word) -- delete the last word from the current text.
• <UNDO> -- undo the effect of the previous speller action.
• <END> -- end spelling, put BCI2000 in suspended mode.
• <SLEEP> -- suspend spelling; resume when <SLEEP> is selected two additional times.
• <PAUSE> -- suspend spelling; resume when <PAUSE> is selected again.
• <GOTO x> -- go to speller menu #x (see Multiple Menus).
• <BACK> -- return to the previously active speller menu.
• <SAVE> -- write the text window's content into the file specified in TextWindowFilePath.
• <RETR> -- load the text window's content from the file specified in TextWindowFilePath.

Multiple Menus

The P3SpellerTask allows you to specify a number of speller menus, and to switch between them using the <GOTO> and <BACK> speller commands. For multiple speller menus, the TargetDefinitions matrix needs to be configured as a list of matrices, rather than a single matrix. Then, each submatrix should have the form described above. Each of the submatrices may have an individual number of rows and columns, and its own set of matrix elements. For information about how to create submatrices, see the information about nested matrices.

Also, switching to Multiple Menus implies additional entries in the following parameters:

• NumMatrixColumns, NumMatrixRows
• AudioStimuliRowsFiles, AudioStimuliRowsFiles (additional columns)
• TargetWidth, TargetHeight, TargetTextHeight, BackgroundColor
• TextColor, TextColorIntensified, IconHighlightMode, IconHighlightFactor

NumMatrixColumns, NumMatrixRows

The number of columns/rows in the speller matrix. For Multiple Menus, a list of numbers representing each menu's number of columns/rows.

AudioStimuliOn

Switches playback of audio stimuli on or off.

AudioStimuliRowsFiles, AudioStimuliRowsFiles

Each of these parameters is a single-column matrix specifying audio files associated with speller rows or columns, respectively. Whenever a row/column is highlighted, the associated audio file is played back. For audio files, Windows WAV format is expected. Paths may be absolute, or relative to the executable's working directory at startup, which usually matches the executable's location.

TargetWidth, TargetHeight

A single matrix element's width/height in percent of screen width/height.

TargetTextHeight

Matrix elements' text height in percent of screen height.

BackgroundColor

Matrix elements' background color, given in RGB encoding.

TextColor, TextColorIntensified

Text color in standard and highlighted (intensified) mode, given in RGB encoding.

IconHighlightMode

An enumerated value specifiying how icons are highlighted on stimulus presentation:

• 0 Show/Hide: icons are only visible during stimulus presentation,
• 1 Intensify: highlight by increasing icon brightness,
• 2 Grayscale: display a grayscale version during stimulus presentation,
• 3 Invert: invert color/brightness values during stimulus presentation,
• 4 Dim: decrease brightness during presentation.

IconHighlightFactor

If IconHighlightMode is 1 or 4, this parameter defines the brightness scaling factor. Dimming is equivalent to intensifying with a scaling factor less than 1.

FirstActiveMenu

For multiple menus, the index of the menu that should be active at the start of a run.

StatusBarSize, StatusBarTextHeight

The size and text height of the status bar in percent of screen height. The status bar is located on top of the screen, and displays a line of text currently spelled. In copy mode, it also displays the text that the user is supposed to spell.

TextToSpell

In copy mode, a string of characters defining the text to be spelled by the user. This text is displayed in the status bar, above the actually spelled text. From the difference between TextToSpell, and the actually spelled text, the speller automatically derives which matrix element the user will need to select next. This information is then used to set StimulusType state.

TextResult

At the beginning of a run, this parameter's content is copied into the lower part of the status bar. At the end of a run, the status bar's content is copied back into this parameter.

TestMode

If this is switched on, clicking on a matrix element with the mouse will select it once the current sequence of intensifications is finished. This is useful to test speller matrix configurations.

DestinationAddress

A network address to receive speller output, given in IP:port format, e.g. localhost:3582. To this address, a UDP socket is opened, and written into such that an external application may connect and receive information about selected matrix elements. As a content, the speller will write whatever is contained in the selected matrix element's Enter field.

TextWindowEnabled

If this flag is switched on, a separate window is displayed. Once the status bar is filled, text flows into the window, and back in case of text deletion.

TextWindowLeft, TextWindowTop, TextWindowWidth, TextWindowHeight

Position and dimension of the text window in pixels.

TextWindowFontName, TextWindowFontSize

Text window font name and size.

TextWindowFilePath

A (relative or absolute) path to a directory. Upon the <SAVE> and <RETR> speller commands, the text window's contents are saved to/retrieved from a file located in that directory. Repeated <SAVE> commands do not result in overwriting existing files. Rather, existing files are preserved, and the most recent file's name is written into a pointer file. Paths may be absolute, or relative to the executable's working directory at startup, which usually matches the executable's location.

States

PreStimulusTime

A 16-bit time stamp in the same format as the SourceTime state. This time stamp is set immediately before the application module is going to update the stimulus/feedback display. Note that a data block is saved together with the state vector that existed immediately after it was acquired. Thus, PreStimulusTime will be the prestimulus time of the block prior to the current block.

StimulusTime

A 16-bit time stamp in the same format as the SourceTime state. This time stamp is set immediately after the application module has updated the stimulus/feedback display. Note that a data block is saved together with the state vector that existed immediately after it was acquired. Thus, StimulusTime will be the stimulus time of the block prior to the current block.

PresentationRequested

This 32-bit state contains a list of ids for all presentations that have been requested during the current block. The list ends with a value of 2^31. A 32-bit presentation id is formed by combining the 16-bit SourceTime state of the presentation request's data block in the most significant bits, with the time difference between that source time stamp and the time when the presentation request happened, in the least significant bits:

PresentationID = SourceTime << 8 | (RequestTime - SourceTime) & 0xff

This definition makes presentation ids unique within the range of a minute, and allows to extract the value of the request's time stamp from its id.

PresentationDisplayed

This 32-bit state contains a list of ids for all presentations that have been displayed during the current block. The list ends with a value of 2^31. Presentation ids appear in the PresentationRequested state when presentation is requested, and subsequently in the PresentationDisplayed state when presentation has occurred.

PresentationTime

A 16-bit time stamp in the same format as the SourceTime state. For each id occurring in the PresentationDisplayed state, this state contains the time stamp of the respective update to video memory. From this point in time, up to two additional frame durations may be required before the display's surface actually shows a difference in signal. This depends on the internals of the display, and cannot be determined from software.

PresentationFrame

A 12-bit state that contains, for each id occurring in the PresentationDisplayed state, the frame number of the respective update to video memory. Frame numbers have an arbitrary offset, and wrap around at a value of 2^11.

AudioBufferTime

A 16-bit time stamp in the same format as the SourceTime state. For each audio presentation that has occurred during the current block, this state contains the time stamp when non-zero audio data were about to enter the system's audio buffer.

AudioPresentationTime

A 16-bit time stamp in the same format as the SourceTime state. For each audio presentation that has occurred during the current block, this state contains the estimated time stamp when non-zero audio data were being played from the system's speakers or headphones. The estimate takes the length of the system's audio buffers into account but will not be able to detect external sources of delays, such as additional audio processing steps.

StimulusCode

The numerical ID of the stimulus being presented (16 bit).

StimulusType

This state is 1 during presentation of an attended stimulus, and 0 otherwise. The notion of an "attended" stimulus requires data recording in copy mode.

StimulusBegin

This state is 1 during the first block of stimulus presentation, and 0 otherwise.

PhaseInSequence

This state is 1 during pre-sequence, 2 during sequence and 3 during post-sequence (see Timeline).

PauseApplication

While this state is set to 1, no task processing occurs, i.e. the task is paused, and may be resumed by setting PauseApplication to 0.

SelectedTarget, SelectedRow, SelectedColumn

Upon classification, these states are set to the selected target's ID, and its associated row and column, respectively. A target's ID matches its row number in the TargetDefinitions matrix.

See also

User Reference:StimulusPresentationTask