bci2000.org BBS

Hi,

I am relatively new to the BCI2000 toolkit and C++. I've spent the past few days picking apart the P3Speller application to understand how the different class members operate.

My overall goal is to adopt the speller portion of the P3Speller and test different features outside of the p300 signal. For me, this is merely an exercise as a graduate student to study various signal classification methods on different features such as energy, power in various spectral bands etc.

It seems that much of the P300 ERP handling is integrated within the P3Speller code. It is my understanding that OnClassResult might be the class I'd want to modify to take in signals from my own signal processing modules. Could anyone provide more insight on what areas of the code I'd want to modify?

Thanks,
Ankit

Hi Ankit,

some of the classification code must reside inside the BCI application module because this is where stimuli or tasks are presented. Information about presented stimuli or tasks, and response scores, must be combined in order to determine the final result. However, the application only receives response scores, which are the continuous output of a linear classifier. It does not make any assumption about how these scores were computed, or what brain signal features they are based upon.

Thus, it is possible to build a universal BCI speller using the unmodified code of the P3Speller, in conjunction with a signal processing module that extracts features of interest.

More concretely, to experiment with spectral features, you might create a signal processing module that contains the following filters in order:

SpatialFilter
SpectralEstimator
P3TemporalFilter
LinearClassifier

Then, you might configure the P3Speller to very slowly display selection alternatives, e.g. with a matrix flash duration of 3s, and an epoch duration of 4s. Then, to select a row or column of the matrix, the subject might, e.g. use motor imagery to modulate the amplitude of the mu rhythm, and with an appropriate configuration of the signal processing module, the speller would receive a larger score for the row or column for which the mu rhythm was modulated than for the other rows or columns.

While this is perfectly possible, it has not been done yet, and no tool exists to train a classifier appropriately. However, BCI2000 allows to apply all its filters offline to recorded data (see the BCI2000Chain documentation), so you can record some training data, run the above filter chain up to the P3TemporalFilter over the data, and use the output of the P3TemporalFilter as features for a linear classifier training procedure (e.g., Fisher LDA, linear SVM, or SWLDA as employed by the P300Classifier training tool).

For more information about any of the keywords mentioned, google for "site:bci2000 keywords".
Let me know if you need more information.

Regards,
Juergen

Thank you, Juergen. After some days of experimenting I constructed a pipeline and better understand how the framework operates.

I have a follow up question. I am trying to acquire some training data using the built in signal processing module. Essentially when I move my mouse upwards the amplitude at 15 Hz on a single channel increases. I have fed this signal into an ARFilter in the SpectralEstimator which ends up in the P3TemporalFilter.

I am confused about how the P3TemporalFilter is handling its incoming stream. On the ARFilter I am using 400ms window to compute spectral power using a 4th order model and 11 bins from 0 to 30 Hz. My P3TemporalFilter epoch length is 4 seconds. It is my understanding that the P3TemporalFilter is simply computing the average waveform across epochs... therefore it concatenates 10 400ms-ARFilter windows = 4s. In the ERP window I am seeing an xaxis ranging 0 Hz to 330 Hz, presumably due to the concatenation in a single epoch. Can I have the P3TemporalFilter average the "sub-epoch" AR windows?

I suppose that I could make my ARFilter window match my epoch window, both either 400ms or 4s, but the former is too short to ask a subject to make a cognitive decision and the latter adds a significant delay to the processing pipeline such that the ERP mismatches with the wrong stimulus presentation (unless the ISI is made much longer, >5sec). Is there a way to mitigate these extremes? Any suggestions?

Thanks,
Ankit

In the ERP window I am seeing an xaxis ranging 0 Hz to 330 Hz, presumably due to the concatenation in a single epoch. Can I have the P3TemporalFilter average the "sub-epoch" AR windows?

You are correct, the values on the axis are wrong because that combination has never been tested before. They represent a concatenation of spectra over the entire epoch.

The P3TemporalFilter was not designed to calculate averages between distinct points of its input. This would be the role of the LinearClassifier. If you want to extract spectral features from a longer period of time, then increase the window size in the SpectralEstimator, and ignore "early" values in classification.

and the latter adds a significant delay to the processing pipeline such that the ERP mismatches with the wrong stimulus presentation

A note on terminology: When extracting spectral features, you are probably not observing an ERP (Evoked Response Potential) but an ERD or ERS (Evoked Response (De-)Synchronization. Although you might be able to classify an ERP by its appearance in the time-frequency domain, this would be a very inefficient approach to classify an ERP.

Regarding your concern of associating a response with the wrong stimulus, this is not a problem. You may just ignore an epoch's begin in classification to account for a delay.

Regarding your concern about a relationship between the ARFilter window, and the subject's time to respond, I cannot see such a connection. If it makes sense to combine earlier and later ARFilter output spectra into a response classification, then a well-trained LinearClassifier will do so. If you are afraid of too many features in classifier training, then just remove those ARFilter output spectra which you think do not carry information (e.g., because the subject has not had time to respond yet).

Regards,
Juergen