BCI2000 Wiki - User contributions [en]

Technical Reference:State Definition

2011-06-22T16:42:24Z

Cstocks: /* State Vector */ Convert to 0-based numbering, since that is what is used in the state vector definition section of the data files.

This page describes the concept of BCI2000 states, in conjunction with their textual representation as a "state line".

For information about how to access state values from code, please refer to [[Programming Reference:Environment Class]].
For information about individual states, please refer to [[User Reference:States]].

==State Concept==
States are variables that represent the internal state of a BCI2000 system as it is evolving over time in response to:
*brain signal input
*user interaction through the operator interface
*trial sequencing by the application module.

Typical quantities/qualities encoded by states are:
*whether the system is running or suspended
*the time when a block of data was recorded
*stimulus or task being presented
*the classification result
*the state of an external marker (trigger) to be saved for off-line analysis.

Typically, state values change once per block of data, or once per trial.
In a [[Technical Reference:BCI2000 File Format|BCI2000 data file]], the full set of states is stored along with the data, allowing for reconstruction of on-line system state off-line.

==State Vector==
In a BCI2000 system, a collection of states is maintained as a ''State List''.
For each state present in that state list, its value is kept as a range of bits in a bit vector called ''State Vector''.
BCI2000 modules and filters may read and write state values during processing.
The state vector's content is saved, in its binary form, into [[Technical Reference:BCI2000 File Format|BCI2000 data files]] per-block.
Using the ByteLocation, BitLocation, and Length fields from the state definitions present in a file, a state's value may be read from the data file.

A state vector is a narrowly packed bit field in '''little endian''' ordering.
This implies that, for a state containing more than a single bit, '''more''' significant bits are placed at '''higher''' bit and byte locations.

As an example, consider a state vector consisting of a 1-bit state "Running", and a 16-bit state "SourceTime". This will result in a three-byte state vector layout like this:
{|border="1" cellspacing="0" cellpadding="1"
!colspan="8"|State Vector Byte 0
!colspan="8"|State Vector Byte 1
!colspan="8"|State Vector Byte 2
|-
!0!!1!!2!!3!!4!!5!!6!!7
!0!!1!!2!!3!!4!!5!!6!!7
!0!!1!!2!!3!!4!!5!!6!!7
|-
!Running!!colspan="16"|SourceTime!!colspan="7"|unused
|-
!0
!0!!1!!2!!3!!4!!5!!6!!7!!8!!9!!10!!11!!12!!13!!14!!15
!colspan="7"| 
|}

==State Lines==
State lines are a human-readable format used to represent individual states in
*[[Technical Reference:BCI2000 File Format|BCI2000 data files]],
*[[Technical Reference:BCI2000 Messages|BCI2000 messages]] sent between modules.
Core modules and the operator module use this format to communicate in the system initialization phase, as well as during system performance and for system termination.

The format of a state line is
<pre>
Name Length Value ByteLocation BitLocation CRLF
</pre>
where <tt>Length</tt> refers to the number of bits used to represent the state's value,
and <tt>ByteLocation</tt> and <tt>BitLocation</tt> refer to its position in the State Vector, with <tt>BitLocation</tt> ranging from 0 to 7.

Depending on the context of a state line, one or more of its fields may be ignored:
*In a State message sent from the operator module to a core module, Length, ByteLocation, and BitLocation will be ignored.
*In a State line contained in a data file, the Value field matches the state's value stored with the first block of data samples.

Generally, the Value field specifies an ''initial value'' of a state. Actual values are stored in the State Vector data structure.

State values are interpreted as unsigned integers, and limited to 32 bit in the current implementation (as defined by the <tt>State::ValueType</tt> typedef in <tt>BCI2000/src/shared/types/State.h</tt>).

==See also==
[[User Reference:States]], [[Technical Reference:Parameter Definition]]

[[Category:Specification]]

User Reference:P300Classifier

2010-08-12T17:26:57Z

Cstocks: /* Location */ The previous link no longer seems to exist. While this links to a branch instead of the trunk, it was the only related link I could find.

==Synopsis==

The P300 Classifier GUI (Graphical User Interface) is a tool that allows to train and test a linear classifier for detection of evoked related potentials collected with [http://www.bci2000.org/wiki BCI2000]. This GUI is designed for the analysis of BCI2000 data collected using the P3Speller or Stimuli Presentation paradigms. The program generates feature weights by using a linear classifier algorithm called Stepwise Linear Discriminant Analysis (SWLDA). The specifics of the feature space and training routine can be manipulated using the GUI. The feature weights derived from the GUI can be saved and imported into BCI2000 as a parameter file fragment (*.prm) for online testing. Unlike the Matlab-based P300 GUI, the P300 Classifier GUI has the main advantage of being completely Matlab independent. All its core functionality is written in C++ accompanied with a friendly graphical user interface written in [http://en.wikipedia.org/wiki/Qt_(toolkit) Qt]. Additionally, this new tool is completely scriptable; that is the investigator can write all the commands needed to train and test the linear classifier in a [http://en.wikipedia.org/wiki/Batch_file batch] file without using the GUI. The P300 Classifier GUI provides the following functionality to investigators:

; Classifier Training: Generates feature weights from BCI2000 P3Speller or Stimuli Presentation data files

; Classifier Testing: Applies current feature weights to BCI2000 P3Speller or Stimuli Presentation data files and compares the results

==Location==
http://{{SERVERNAME}}/svn/branches/3.0/src/core/Tools/P300Classifier

==Versioning==
===Author===
Cristhian Mauricio Potes

===e-mail===
cmpotes@miners.utep.edu

===Source Code Revisions===
*Initial development: May 15, 2009
*Tested under: Windows XP, Windows Vista
*Known to compile under: Visual Studio 2008
*Broken since: --

==Control Panel==

The P300 Classifier GUI is composed of three panes: Data, Parameters, and Details.

===Data Pane===
Data Pane allows the user to

* Load training and testing data files and an INI file
* Generate and apply feature weights
* Write a parameter file fragment (*.prm)

[[image:dataPaneLabels.png|frame|center|Data pane]]

* [1] '''Load Training Data Files:''' Use this button to load [http://www.bci2000.org/wiki BCI2000] data files for classifier training. The information for the selected files will appear at the top of the button.

* [2] '''Load Testing Data Files:''' Use this button to load [http://www.bci2000.org/wiki BCI2000] data files for classifier testing. The information for the selected files will appear at the top of the button. Training and testing data files must be compatible.

* [3] '''Load Ini File:''' Use this button to load an [http://en.wikipedia.org/wiki/INI_file INI file] with all the parameters needed for the classifier.

<pre>
[Initialization]

maxiter = 60 // to set Max Model Features
penter = 0.1000 // to set Penter
premove = 0.1500 // to set Premove
spatial_filter = 1 // to set Spatial Filter. Set 1 for RAW and 2 for CAR
decimation_frequency_Hz = 20 // to set Decimation Frequency
channel_set = 1 2 3 4 5 6 7 8 // to set Channel Set
Resp_window_ms = 0 800 // to set Response Window. The response window must be in milliseconds (ms)
</pre>

* [4] '''Generate Feature Weights:''' Use this button to generate the feature weights after properly configuring all of the parameters in the [[User_Reference:P300_classifier#Parameters|Parameters]] pane. This button will be enable only if the parameters are properly configured and there exists training data files. Once it has been generated feature weights properly, a suggested name for the parameter file fragment (*.prm) will show up at the top of the [[User_Reference:P300_classifier#Write *.prm File|Write *.prm File]] button.

* [5] '''Apply Feature Weights:''' Use this button to test the classification accuracy of the feature weights currently stored in the GUI. The classification results will appear in the [[User_Reference:P300_classifier#Details Pane|Details]] pane.

* [6] '''Write *.prm File:''' Use this button to save the parameter file fragment with the name suggested at the top of this button. The *.prm file is a [http://www.bci2000.org/wiki BCI2000] parameter file fragment that can be imported into [http://www.bci2000.org/wiki BCI2000] for online testing of the feature weights.

===Parameters Pane===
Parameters Pane contains all the parameters needed to generate feature weights by using the SWLDA algorithm. These parameters can be loaded using the [[User_Reference:P300_classifier#Load Ini File|Data Pane]] button. If the parameters are properly configured the [[User_Reference:P300_classifier#Data Pane|Generate Feature Weights]] button is enabled.

[[image:parametersPaneLabel.png|frame|center|Parameters pane]]

* [7] '''Max Model Features:''' Used to specify the maximum number of features to be kept in the SWLDA algorithm. Only a single value can be entered for evaluation. The default value is 60.

* [8] '''Penter:''' Used to specify the maximum [http://en.wikipedia.org/wiki/P-value p-value] for a variable to be entered. The default value is 0.1000. Penter must be less than Premove and 0<Penter<1. Only a single value can be entered for evaluation.

* [9] '''Premove:''' Used to specify the maximum [http://en.wikipedia.org/wiki/P-value p-value] for a variable to be removed. The default value is 0.1500. Premove must be greater than Penter and 0<Premove<1. Only a single value can be entered for evaluation.

* [10] '''Spatial Filter:''' Selects the spatial filter applied to the training data. Select '''RAW''' or '''CAR''' from the drop-down menu. '''RAW''' is no spatial filter applied to the data, and '''CAR''' is a common average reference filter using all of the channels contained in the data file '''(not just the channels specified in the GUI channel set)'''. The default spatial filter is '''RAW'''.

* [11] '''Decimation Frequency:''' Used to specify the temporal decimation frequency of the data in Hz. Only a single value can be entered for evaluation. Set this parameter to the [[User_Reference:P300_classifier#Details Pane|Sampling Rate]] value for no decimation. The lower the Decimation Frequency, the less original data retained for processing.

* [12] '''Channel Set:''' Used to specify the channel set that will be used to create feature weights. The specified channels must be a subset of the channels contained in the training data file.

* [13] '''Response Window:''' Used to specify the ''Begin'' and ''End'' time points in milliseconds (ms) following the stimuli collected for the analysis. These two values are automatically converted into samples according to the sampling rate of the data (rounded). Only a single data window can be entered and will be evaluated. ''Begin'' must be less than ''End''.

===Details Pane===
Details Pane displays information about the [http://www.bci2000.org/wiki BCI2000] training data files and the classification results.

[[image:detailsPaneLabel.png|frame|center|Details pane]]

* [14] '''Sampling Rate:''' Displays the sampling rate in (Hz) contained in the training data files.

* [15] '''Number of Channels:''' Displays the total number of channels contained in the training data files.

* [16] '''Classifier:''' Displays the classifier applied to generate the feature weights. The P300Classifier GUI only uses the SWLDA classifier algorithm.

* [17] '''Application:''' Displays the type of application used for the investigator. The application can be either ''P3SpellerTask'' or ''StimulusPresentationTask''.

* [18] '''Interpret Mode:''' Displays the interpret mode used for the investigator. The mode can be either ''Copy Mode'' or ''Online Free Mode''.

* [19] '''Duration:''' Displays the duration in seconds (s) of the whole training data files.

* [20] The text editor displays the classification results for the training and testing data files.

==Methodology==

===Methodology Concept===
The idea behind the P300Classifier is to train and test a linear classifier (SWLDA) for detection of evoked potentials collected from brain signals. This is achieved in the following steps:
* Training Data Files
# Load BCI2000 data files
# Get P300 responses
# Generate feature weights for a linear model using SWLDA
# Apply linear classifier to get scores
# Interpret scores according to the given application

* Testing Data Files
# Load BCI2000 data files
# Get P300 responses
# Apply linear classifier to get scores
# Interpret scores according to the given application

===Methodology Step By Step===

====Step 1: [http://www.bci2000.org/wiki/index.php/UserReference:P300ClassifierMethods Load BCI2000 Data Files]====

The first step to start using the P300Classifier GUI is to load training and testing BCI2000 data files specified by the investigator. The BCI2000 data files are checked for compatibility and consistency. If all training data files are valid then the ''Generate Feature Weights'' button is enabled and each file is colored with light green. Otherwise, the ''Generate Feature Weights'' button is not enabled, and each file is colored either with yellow or pink. See the following color coded scheme and example.

* files ok : The files are valid.
* files mismatch : The files may be valid but there is a mismatch with another file.
* files error : The files are invalid.

[[image:tutorial0.png|frame|center|Training data file is not valid; the file is highlighted in pink. Testing data file is valid; the file is highlighted in light green. The ''Generate Feature Weights'' button is not enabled until the training data file is valid.]]

====Step 2: [http://www.bci2000.org/wiki/index.php/UserReference:P300ClassifierMethods Get P300 Responses] ====

Signals, states, and parameters are extracted from BCI2000 training and testing data files. However, the signals that are only extracted for analysis are those defined by the user in the [[User_Reference:P300_classifier#Parameters Pane|'''Response Window''']]. Only these signals are filtered and downsampled. The filter employed is a [http://en.wikipedia.org/wiki/Moving_average Moving Average (MA) Filter] implemented as a ''Direct form II Transposed''. The MA filter belongs to the FIR filter and works as a low pass filter removing high frequency components of the signal.

====Step 3: [http://www.bci2000.org/wiki/index.php/UserReference:P300ClassifierMethods Generate Feature Weights for a Linear Model using Stepwise Linear Discriminant Analysis (SWLDA)]====

The idea behind SWLDA is to obtain a final linear model that approximately fits a set of data (stimulus) by using multiple linear regressions and iterative statistical procedures, thus selecting only significant variables that are included in the final regression.

The SWLDA algorithm can be summarized in the following steps:

* STEP 1: Compute the autocorrelation matrix to select the variable most highly correlated to the observations. The selected variable is included in the linear model.

* STEP 2: Fit an initial model by performing a linear regression equation with the included variable using least squares.

* STEP 3: Check if the variable added is that one which makes greatest improvements in "goodness of fit." That is, remove the added variable if its variance contribution in the regression is insignificant at a specified F level (F-test). If the added variable is removed go to Step 5, else go to Step 4.

* STEP 4: Add the variable to the model if there is a significant variance reduction at a specified F-level.

* STEP 5: Remove the added variable from the correlation matrix. Go to step 1.

The method terminates when no single step improves the model.

====Step 4: Apply Linear Classifier to Get Scores ====

Scores are computed based on the variables included in the final linear model and the corresponding feature weights.

====Step 5: Interpret Scores ====

Scores are interpreted according to the given application, which can be either P300SpellerTask or StimulusPresentationTask.

==Tutorial==

===Training the SWLDA Classifier===
The ''Parameters'' pane contains all of the parameters for generating feature weights from BCI2000 data files.

The investigator must follow the next steps to generate feature weights:

* Press the ''Load Training Data Files'' [1] button in the ''Data'' pane.

* From the dialog box, select the desire BCI2000 *.dat file(s) for training. Selected files can be from different sessions of the same paradigm but must contain consistent parameters. Each training data file is colored according to the color coded scheme explained in [[User_Reference:P300_classifier#Step 1: Load BCI2000 data files|Load BCI2000 data files]]. Files can only be selected from a single directory; the desired training data files should be organized into the same directory prior to using the P300Classifier GUI.

* Once the ''Parameters'' pane is correctly set as desired, the ''Generate Feature Weights'' button is enable. Press this button to perform the analysis and generate feature weights. The ''Overall Progress'' bar displayed in ''Data'' and ''Details'' panes will indicate the progress of the SWLDA. The classification results will appear in the text editor [20] of ''Details'' pane. Every time it is pressed the ''Generate Feature Weights'' button, and the analysis is successfully completed, there is generated a new feature weights set.

* The training procedure can be repeated multiple times generating a new feature weights set that depends on the configured parameters.

* After training is completed and feature weights are generated, it is recommended to test (cross validate) the feature weights on independent data (testing data files) before saving the parameter file fragment (*.prm). Additionally, there is a suggested parameter file fragment name displayed on the top of ''Write *.prm File'' button, which can be used to save the *.prm file either with the suggested name or the name specified by the investigator.

===Testing the SWLDA Classifier===
The ''Data'' pane contains the ''Apply Feature Weights'' [5] button for testing the generated feature weights to one or more BCI2000 testing data files.

The investigator must follow the next steps for applying feature weights to testing data files:

* Once it has been successfully generated feature weights and there exists testing data files, the ''Apply Feature Weights'' button is enabled. If there are no testing data files loaded, the investigator must press this button to load testing data file(s). One or more BCI2000 *.dat files can be selected from the same directory as a single 'test file group' each time the button is pressed. Selected files can be from different sessions of the same paradigm but must contain consistent parameters. Each testing data file is colored according to the color coded scheme explained in [[User_Reference:P300_classifier#Step 1: Load BCI2000 data files|Load BCI2000 data files]]. All testing files selected should have the same sampling rate and electrode montage as the training data files selected to generate the current feature weights set.

* After all the testing data files are correctly selected, press the ''Apply Feature Weights'' [5] button to perform the analysis. Results of the classification are displayed in the text editor [20] of ''Details'' pane. The ''Overall Progress'' bar displayed in ''Data'' and ''Details'' panes will indicate the progress of the classification.

* After evaluating the classification results, the *.prm file from the current session can be saved by pressing the ''Write *.prm File'' button.

===Example===

In the next example, it is desired to compute feature weights from the following BCI2000 training data file '''eeg3_1.dat'''.

By pressing the ''Load Training Data Files'' button the file '''eeg3_1.dat''' is uploaded as shown in the next figure. Notice that the training data file is set up correctly since it is colored in light green. Remember that the classifier algorithm will be trained by using this training data file.

[[image:tutorial1.png|frame|center|By pressing the ''Load Training Data Files'' button it is uploaded the desired BCI2000 training data file. The classifier algorithm is trained by using this training data file.]]

The parameters shown in ''Parameters Pane'' are set by default. For illustration purposes, however, it is desired to load initial parameters from an INI file called parameters.ini by pressing the ''Load Ini File'' button.

<pre>
[Initialization]
maxiter = 60
penter = 0.1000
premove = 0.1500
spatial_filter = 2
decimation_frequency_Hz = 20
channel_set = 1 2 3 4 5 6 7 8
Resp_window_ms = 0 800
</pre>

[[image:tutorial2.png|frame|center|By pressing the ''Load Ini File'' button it is uploaded the desired initial parameters that the classifier algorithm will use to generate the corresponding feature weights.]]

Once the initial parameters are loaded and properly set up, the ''Generate Feature Weights'' button is enabled, and each one of the parameter fields is colored in light green as shown in the next figure.

[[image:tutorial3.png|frame|center|All of the parameter fields are properly set up. Thus, each parameter field is colored in light green.]]

The investigator can change any of the parameters fields at any time. However, if that change is not properly set up (i.e., the given parameter is invalid), the corresponding parameter field will be colored in pink.

* parameter field ok : The parameter is valid.
* parameter field error : The parameter is invalid.

In the next figure, the investigator changes the ''Channel Set'' field introducing one more channel, channel 9. Since this channel is not a subset of the channels contained in the training data file, the corresponding field is colored in pink, and a tool tip is popped up displaying the error message. Additionally, the ''Generate Feature Weights'' button is not enabled until that specific field is properly configured.

[[image:tutorial4.png|frame|center|The ''Channel Set'' field is modified by introducing one more channel, channel 9. This field is colored in pink since channel 9 is not a subset of the channels contained in the training data file.]]

Once the parameters are properly configured, the investigator is ready to generate feature weights by pressing the ''Generate Feature Weights'' button. Details of the training data file and progress of the classification are displayed in the ''Details Pane'' as shown below.

[[image:tutorial5.png|frame|center|Details of the training data file and progress of the classification. There is an overall progress bar in both the ''Data Pane'' and ''Details Pane''.]]

Once the classifier has been trained with the BCI2000 training data file, and it has generated the corresponding feature weights, it is desired to test these feature weights with the following BCI2000 testing data file '''eeg3_2.dat'''.

By pressing the ''Load Testing Data Files'' button this file is uploaded as shown in the next figure. Notice that the testing data file is set up correctly since it is colored in light green. If the testing data file is valid then the ''Apply Feature Weights'' button is enabled.

[[image:tutorial6.png|frame|center|By pressing the ''Load Testing Data Files'' button it is uploaded the desired BCI2000 testing data file. The file is colored in light green if valid.]]

The investigator is ready to apply feature weights by pressing the ''Apply Feature Weights'' button. Details of the training data file (no testing data file) and progress of the classification are displayed in the ''Details Pane'' as shown below.

[[image:tutorial7.png|frame|center|Details of the training data file and progress of the classification. Testing feature weights generated from the training data file '''eeg3_2.dat'''. There is an overall progress bar in both the ''Data Pane'' and ''Details Pane''.]]

If satisfied with the classification obtained from the current feature weights, the investigator can write a parameter file fragment *.prm with the name suggested in the ''Write *.prm File'' field by pressing the ''Write *.prm File'' button. The *.prm file is a BCI2000 parameter file fragment that can be imported into BCI2000 for online testing of the feature weights. In the following image, it is written a *.prm file called '''MUDChS1_RAW_SW.prm'''.

[[image:tutorial8.png|frame|center|Write a parameter file fragment *.prm. The *.prm file is a BCI2000 parameter file fragment that can be imported into BCI2000 for online testing of the feature weights.]]

Every time the investigator changes any of the parameters in the ''Parameters Pane'', the ''Apply Feature Weights'' button is disabled. To enable it, the investigator must generate feature weights again.

==Future work==

* Present classification results as topographies and graphs as the Matlab-based P300 GUI

* Let the P300Classifier GUI work with frequency-based features

* Speed up the P300Classifier GUI by using multithreading when solving the Least Squares (QR factorization)

==References==

* N. R. Draper, H. Smith. ''Applied Regression Analysis''. John Wiley & Sons, Inc, New York, 1966.

* P. M. Embree, B. Kimble. ''C Language Algorithms for Digital Signal Processing''. Prentice Hall, Englewood Cliffs, New Jersey, 1991.

* A. Ralston, H. S. Wilf. ''Mathematical Methods for Digital Computers''. John Wiley & Sons, Inc, New York, 1962.

* W. H. Press, S. A. Teukolsky, W. T. Vetterling, B. P. Flannery. ''Numerical Recipes in C''. Cambridge University Press, New York, 1988.

Talk:User Reference:P3SpellerTask

2010-08-05T13:57:46Z

Cstocks: /* More detail on state meaning */

== More detail on state meaning ==

It would be helpful if there were more data on the meaning of certain states, specifically with regard to how they should be interpreted. For example, how do you use the StimulusCode to figure out which row/column is flashing when in row/column mode?

[[User:Cstocks|Cstocks]] 13:55, 5 August 2010 (UTC)

User:Cstocks

2010-08-05T13:57:19Z

Cstocks:

Hi, I'm Collin. I have done some research at the Wadsworth Center, and plan to continue.

Come September, I will be attending college at The Cooper Union.

Anyway, stop wasting time reading my user page.

[[User:Cstocks|Cstocks]] 13:57, 5 August 2010 (UTC)

Talk:User Reference:P3SpellerTask

2010-08-05T13:55:33Z

Cstocks: More detail on state meaning

== More detail on state meaning ==

It would be helpful if there were more data on the meaning of certain states, specifically with regard to how they should be interpreted. For example, how do you use the StimulusCode to figure out which row/column is flashing when in row/column mode?
[[User:Cstocks|Cstocks]] 13:55, 5 August 2010 (UTC)

Programming Reference:BCI2000FileReader Class

2010-07-29T21:11:39Z

Cstocks: /* CalibratedValue(channel, sample) */ correct parameter names

The <tt>BCI2000FileReader</tt> class provides an interface to data stored in a single BCI2000 data file.

==Methods==
===BCI2000FileReader([file name])===
With a file name given as an argument, the constructor will try to open the specified file.

===Open(file name, [buffer size=50kB])===
Will open the specified data file. Use the <tt>IsOpen</tt> and <tt>ErrorState</tt> properties to check whether a file is actually open.
In a second argument, the size of <tt>BCI2000FileReader</tt>'s internal data buffer may be specified in bytes; when no buffer size argument is given, a default size of 50kB is used.

===RawValue(channel, sample)===
Sets the current sample position to the value specified in the second argument, and returns the raw sample value for the channel specified in the first argument.

===CalibratedValue(channel, sample)===
Sets the current sample position to the value specified in the second argument, and returns the calibrated sample value for the channel specified in the first argument.
The calibrated sample value is obtained by applying channel-specific sample offsets and gains as present in the file's ''SourceChOffset'' and ''SourceChGain'' parameters.

===ReadStateVector(sample)===
Sets the current sample position to the specified value, and reads state vector data.

==Properties==
===IsOpen (r)===
''true'' when a file has been opened successfully, ''false'' otherwise.
===ErrorState (r)===
One of
*NoError,
*FileOpenError,
*MalformedHeader.
===NumSamples (r)===
The number of samples in the currently opened data file.
===SamplingRate (r)===
The currently opened data file's sampling rate.
===SignalProperties (r)===
A [[Programming_Reference:Signals#SignalProperties_Class|<tt>SignalProperties</tt>]] object describing the current file's data format, number of channels, sample block size (in its Elements property), and physical units.

===FileFormatVersion (r)===
A string describing the file's format as specified by the file header. Currently, this is "1.0" or "1.1".
===HeaderLength (r)===
The data file's header length in bytes.
===StateVectorLength (r)===
The state vector's length in bytes.
===Parameters (r)===
A [[Programming Reference:Parameters#ParamList Class|<tt>ParamList</tt>]] object that contains all parameters present in the file.

===Parameter (r)===
Access to an individual parameter, following the [[Programming_Reference:Environment_Class#Access_to_Parameters_and_States|syntax provided by the <tt>Environment</tt> class]].
===States (r)===
A [[Programming Reference:States#StateList Class|<tt>StateList</tt>]] object containing all state variables present in the file.

===State (r)===
Access to an individual state variable, following the [[Programming_Reference:Environment_Class#Access_to_Parameters_and_States|syntax provided by the <tt>Environment</tt> class]]. The returned value will match the state variable's value at the current sample position, as specified to a previous call to one of the <tt>RawValue</tt>, <tt>CalibratedValue</tt>, or <tt>ReadStateVector</tt> methods.

===StateVector (r)===
Access to the full state vector, with state values matching those at the current sample position.

==Performance Considerations==
To improve performance when reading data, <tt>BCI2000FileReader</tt> uses an internal buffer which is 50kB in size by default but may be set to a different value using an optional second argument to <tt>BCI2000FileReader::Open()</tt>.

Buffering strategy is optimized for ''sequential forward access'', which is supposed to be the most relevant use case. Buffering is non-overlapping, with the requested sample always the first one in the buffer in case the buffer needs updating. In case of sequential access in ''reverse'' direction, this buffering scheme breaks down and becomes maximally inefficient because the entire buffer is updated on each access.

In a BCI2000 data file, data is organized such that, for each sample, values from all channels and states are stored, followed with all values from the next sample point, etc. When reading data from multiple channels, this implies that it is favorable to iterate over ''channels'' in an ''inner'' loop, and over ''samples'' in an ''outer'' loop; otherwise, buffering will become less efficient by a factor identical to the number of channels read.

==See also==
[[Technical Reference:BCI2000 File Format]], [[User Reference:BCI2000FileWriter]]

[[Category:Framework API]] [[Category:Development]]

Talk:Technical Reference:State Definition

2010-01-05T23:07:05Z

Cstocks: Pseudocode

== State Vector ==

The section on state vectors is totally wrong. After reverse-engineering some data files with the help of the BCI2000Viewer, it appears that the state vectors are actually encoded in reverse-big-endian. In other words, if smaller numbers represent lower order bits, then the state vector layout mentioned in this section would actually be encoded as follows:

{|border="1" cellspacing="0" cellpadding="1"
!colspan="8"|State Vector Byte 1
!colspan="8"|State Vector Byte 2
!colspan="8"|State Vector Byte 3
|-
!0!!1!!2!!3!!4!!5!!6!!7
!0!!1!!2!!3!!4!!5!!6!!7
!0!!1!!2!!3!!4!!5!!6!!7
|-
!Running!!colspan="16"|SourceTime!!colspan="7"|unused
|-
!0
!0!!1!!2!!3!!4!!5!!6!!7!!8!!9!!10!!11!!12!!13!!14!!15
!colspan="7"| 
|}

Which on the physical machine actually maps to:

{|border="1" cellspacing="0" cellpadding="1"
!colspan="8"|State Vector Byte 1
!colspan="8"|State Vector Byte 2
!colspan="8"|State Vector Byte 3
|-
!7!!6!!5!!4!!3!!2!!1!!0
!7!!6!!5!!4!!3!!2!!1!!0
!7!!6!!5!!4!!3!!2!!1!!0
|-
!colspan="7"|SourceTime!!Running!!colspan="8"|SourceTime!!colspan="7"|unused!!SourceTime
|-
!6!!5!!4!!3!!2!!1!!0!!0!!14!!13!!12!!11!!10!!9!!8!!7!!colspan="7"| !!15
|}

== Fixed section on state vector encoding ==

Thanks for pointing out the error. State vector encoding is actually little-endian (LSB first) rather than big-endian, and its layout on a little-endian machine is straightforward as a concatenation of little-endian encoded state values. I fixed the description accordingly.

== Pseudocode reference ==

Do you think it would be helpful to readers to put (in pseudocode) a way to read the State Definition?

int ExtractStateValue(unsigned char[] vector, int byteoffset, int bitoffset, int numberofbits){

int numberofbytes = numberofbits / 8 + 1;
char bytes[numbytes] = vector[ byteoffset : byteoffset + numbytes ];

int ret = bytes[0] >> bitoffset;
for ( int i=1; i<numbytes; i++ ){
ret += bytes[i] << ( 8 * i - bitoffset );
}

return ret % ( 1 << numbits );
}

This is not compilable C or C++ code, as far as I know. However, I think it might make things clearer.

[[User:Cstocks|Cstocks]] 21:15, 4 January 2010 (UTC)

== Actual C++ code ==

The actual C++ code is here:
http://www.bci2000.org/tracproj/browser/trunk/src/shared/types/StateVectorSample.cpp#L69
(log into trac if you get a message about browser view privileges)

[[User:Mellinger|Mellinger]] 15:00, 5 January 2010 (UTC)

== Pseudocode ==

Thanks for pointing that out, but to someone (like me) who is not terribly familiar with C++ code, seeing the actual code is not very helpful. For example, I would have to start hunting through the code to find out what State::ValueType, mByteLength, mpData, et cetera mean.

Also, I am proud to note that my algorithm is at least 8 times more efficient than the C++ function you pointed me to.

User:Cstocks

2010-01-04T21:23:05Z

Cstocks:

Hi, I'm Collin. I have done some research at the Wadsworth Center, and plan to continue.

I am currently developing a way to classify P300 data using known weights in Python, sort of like how the old MatLab script did it, but it will also work for classifying online free spelling data. Perhaps later I will program some techniques for generating weights in Python. If I do, it will likely be used to generate weights from online data once it has been determined what the correct classifications should have been. This method may possibly generate more useful weights, because there may be a difference in the P300 signal during copy spelling as compared to the P300 signal during free spelling.

Anyway, stop wasting time reading my user page.

[[User:Cstocks|Cstocks]] 21:23, 4 January 2010 (UTC)

Talk:Technical Reference:State Definition

2010-01-04T21:15:23Z

Cstocks: Pseudocode reference

Talk:Technical Reference:State Definition

2010-01-01T00:45:11Z

Cstocks: /* State Vector */ Finally correct, again. I made a mistake before.

Talk:Technical Reference:State Definition

2010-01-01T00:30:40Z

Cstocks: /* State Vector */ Finally got it correct

Talk:Technical Reference:State Definition

2010-01-01T00:24:31Z

Cstocks: /* State Vector */

Talk:Technical Reference:State Definition

2010-01-01T00:23:33Z

Cstocks: /* State Vector */

Talk:Technical Reference:State Definition

2010-01-01T00:22:27Z

Cstocks: State Vector

Technical Reference:BCI2000 File Format

2009-12-31T16:14:53Z

Cstocks: /* Binary Data */ Clarify what "the last CRLF" means. The binary data could possibly (and probably does for large files) contain two consecutive bytes that could be interpretted as CRLF.

A BCI2000 data file consists of a header and the actual raw brain signals.
Its associated file name extension is <code>.dat</code>.
The header consists of a definition of all system parameters and states.
Thus, parameter values cannot change within a [[BCI2000 Glossary#Run|run]], which in turn corresponds to a single data file.

==Header==
The header of a data file consists of lines of ASCII characters, separated by a carriage return/line feed (CRLF) control character sequence typically present in DOS/Windows text files.
Its total length is given by the <code>HeaderLen</code> field in the first line.
The header contains meta information, state (event) definitions, and parameters.

===Meta Information===
The fields in the first line specify meta information required to access the binary data that follows the header.

Since version 1.1, the first line begins with a <code>BCI2000V</code>
field containing a floating point version number, and ends with a
<code>DataFormat</code> field describing the format of the binary data as
<code>int16</code>, <code>int32</code>, or <code>float32</code>.
A missing <code>BCI2000V</code> field indicates a file format version of
1.0, and a <code>DataFormat</code> of <code>int16</code>.

The number of bytes in the state vector is determined by the sum of the lengths (given in bits) for all states, rounded up to the next byte (which equals the value of <code>StateVectorLength</code> in both the first line and, since <code>StateVectorLength</code> is also a system-wide parameter, in one of the lines in the <code>[ Parameter Definition ]</code> section).
Thus, the first line contains all information required to read the binary information contained in the data file.

===State Definitions===
State definitions are preceded with a line <code><nowiki>[ State Vector Definition ]</nowiki></code>, and given in the general [[Technical Reference:State Definition|State Definition]] format.
In the context of the file header, the state definitions' ''value'' fields are redundant since they match the values associated with the first sample in the data file.

===Parameter Definitions===
Parameter definitions are preceded with a line <code><nowiki>[ Parameter Definition ]</nowiki></code>, and follow the general [[Technical Reference:Parameter Definition|Parameter Definition]] format. They represent the respective parameters' values at the beginning of the recording.

===Summary===

BCI2000V= 1.1 HeaderLen= l SourceCh= m StateVectorLength= k DataFormat= f CRLF
[ State Vector Definition ] CRLF
Name1 Length1 Value1 ByteLocation1 BitLocation1 CRLF
Name2 Length2 Value2 ByteLocation2 BitLocation2 CRLF
Name3 Length3 Value3 ByteLocation3 BitLocation3 CRLF
...
[ Parameter Definition ] CRLF
Section1 DataType1 Name1= Value1 DefaultValue1 LowRange1 HighRange1 // Comment CRLF
Section2 DataType2 Name2= Value2 DefaultValue2 LowRange2 HighRange2 // Comment CRLF
Section3 DataType3 Name3= Value3 DefaultValue3 LowRange3 HighRange3 // Comment CRLF
...
CRLF

==Binary Data==
The binary data directly follows the last CRLF. (The end of the header is marked by CRLF CRLF.)
For each sample, data values for all channels are stored, followed by <code>StateVectorLength</code> bytes for the state vector.
Data samples are always stored in little endian byte ordering.
Their format depends on the <code>DataFormat</code> field:
{| border="1"
! DataFormat field !! data type
|-
| int16 || 2-byte signed integer
|-
| int32 || 4-byte signed integer
|-
| float32 || 4-byte floating point (IEEE 754)
|}

From the total file size, and the information present in the header's first line, the number of samples in a data file may be calculated as follows:
<math>
\textrm{samples}
=\frac{(\textrm{file\ size\ in\ bytes})-\texttt{HeaderLen}}
{\textrm{(data\ value\ size\ in\ bytes)}\times\texttt{SourceCh}
+\texttt{StateVectorLength}}
</math>

==Required Parameters==
These parameters will be present in any BCI2000 data file.
While some of these parameters are required to interpret a data file's content (source parameters), others reflect information about external circumstances (storage parameters), or internal housekeeping (system parameters).

===Section Source===
The parameters in this section are required to be present in any BCI2000 data file.
These parameters are necessary to interpret the raw data contained in the file, and are thus belonging into section ''Source''.

{|border="1"
|+Required Parameters
! Type !! Parameter Name !! Description
|-
| int || SampleBlockSize || number of samples acquired, transmitted, and stored at a time
|-
| float || SamplingRate || data acquisition rate in samples per second
|-
| int || SourceCh || number of digitized and stored channels
|-
| floatlist || SourceChOffset || raw data zero offset in AD units
|-
| floatlist || SourceChGain || factor to convert raw AD units into <math>\mu V</math>
|-
| int || AlignChannels || flag indicating whether data require alignment in time
|-
| floatlist || SourceChTimeOffset || individual channel offsets in time, ranging between 0 and 1; -1 indicates that offsets are distributed uniformly over 0..1
|}

===Section Storage===
These parameters are used to identify external circumstances of a recording, by listing subject, technical equipment, and technician information.

{|border="1"
|+Storage Parameters
! Type !! Parameter Name !! Description
|-
| string || SubjectName || subject alias
|-
| string || SubjectSession || subject session number
|-
| string || SubjectRun || run number within session (auto-incremented)
|-
| string || DataDirectory || directory under which the session directory is to be created
|-
| string || ID_System || rowspan="3" | equipment identification
|-
| string || ID_Amp
|-
| string || ID_Montage
|}

===Section System===
The parameters in this section are used by the BCI2000 modules to organize their connections, and they provide module versioning information.
{|border="1"
|+System Parameters
! Type !! Parameter Name !! Description
|-
| string || EEGsourceIP|| IP address the source module listens on
|-
| int || EEGsourcePort|| IP port the source module listens on
|-
| matrix || EEGsourceVersion || source module versioning information
|-
| string || SignalProcessingIP|| IP address the signal processing module listens on
|-
| int || SignalProcessingPort|| IP port the signal processing module listens on
|-
| matrix || SignalProcessingVersion || signal processing module versioning information
|-
| string || ApplicationIP|| IP address the application module listens on
|-
| int || ApplicationPort|| IP port the application module listens on
|-
| matrix || ApplicationVersion || application module versioning information
|-
| matrix || OperatorVersion || operator module versioning information
|-
| int || StateVectorLength || length of the state vector in bytes
|}

==Required States==
These states are present in all BCI2000 data files.
Individual [[User Reference:Filters|filters]] may add states to the system; see filter reference pages for a description of these states.
{|border="1"
|+Common States
! Length (bits) !! State Name !! Remarks
|-
| 1 || Running || Setting this state to 0 will put BCI2000 into suspended mode.
|-
| 16 || SourceTime || rowspan="2" | Time of data acquisition/stimulus delivery as an unsigned wrap-around value with a resolution of 1ms. This information is used to measure [[User Reference:Timing|system timing]].
|-
| 16 || StimulusTime
|}

==Software Components==
*For writing, information about the BCI2000 data file format is encapsulated into the [[User Reference:BCI2000FileWriter|<code>shared/fileio/dat/BCI2000FileWriter</code>]] component.
*For reading, information about the BCI2000 data file format is encapsulated into [[Programming Reference:BCI2000FileReader|<code>shared/fileio/dat/BCI2000FileReader</code>]].
*A Matlab mex file to read BCI2000 data files is provided at [[User Reference:Matlab MEX Files#load_bcidat|<code>Tools/mex/load_bcidat</code>]]. This mex file uses the <code>BCI2000FileReader</code> component.

==See also==
[[User Reference:Data File Formats]], [[Technical Reference:Parameter Definition]], [[Technical Reference:State Definition]]

[[Category:Data Storage]][[Category:Specification]]