I'm trying to configure a P300 Speller in BCI2000. I'm performing calibration with three .dat files and then cross-validating to see the classifier's performance. However, I'm encountering the following issue: even when increasing the number of sequences between consecutive sessions, I sometimes get very good performance and other times very poor performance. For example this would be a "good" session:
First crossing of the three calibration files:
***** Generating Feature Weights *****
(Spatial Filter = Raw)
Loading file [1/2]...
Collecting Responses from file [1/2]...
Filtering and Decimating file [1/2]...
Loading file [2/2]...
Collecting Responses from file [2/2]...
Filtering and Decimating file [2/2]...
Targets: 80
Non Targets: 400
Applying Stepwisefit classifier...
Added features... [1/60]
Added features... [2/60]
Added features... [3/60]
Added features... [4/60]
Added features... [5/60]
Added features... [6/60]
Added features... [7/60]
Added features... [8/60]
Added feature... [9/60]
Added features... [10/60]
Added features... [11/60]
Added features... [12/60]
Added features... [13/60]
Added features... [14/60]
Added features... [15/60]
Added features... [16/60]
Added features... [17/60]
Added features... [18/60]
Added features... [19/60]
Added features... [20/60]
Added features... [21/60]
Added features... [22/60]
Added features... [23/60]
Classifying Responses...
Flashes % Correct Predicted Symbols
1 63 LOLEKOLO
2 88 HOLEEOLO
3 88 HOXAEOLO
4 100 HELLOOL
5 100 HELLOOL
Donate!
Time elapsed [s]: 0.272
*****Applying Feature Weights*****
(Spatial Filter = Raw)
Loading file [1/1]...
Collecting Responses from file [1/1]...
Filtering and Decimating file [1/1]...
Targets: 40
Non Targets: 200
Classifying Responses...
Flashes % Correct Predicted Symbols
1 25 5K1V
2 50 OKOV
3 25 CKO8
4 75 CHEV
5 100 AHOV
Donate!
Second crossing of the three calibration files:
Time elapsed [s]: 0.118
***** Generating Feature Weights *****
(Spatial Filter = Raw)
Loading file [1/2]...
Collecting Responses from file [1/2]...
Filtering and Decimating file [1/2]...
Loading file [2/2]...
Collecting Responses from file [2/2]...
Filtering and Decimating file [2/2]...
Targets: 80
Non Targets: 400
Applying Stepwisefit classifier...
Added features... [1/60]
Added features... [2/60]
Added features... [3/60]
Added features... [4/60]
Added features... [5/60]
Added features... [6/60]
Added features... [7/60]
Added features... [8/60]
Added features... [9/60]
Added features... [10/60]
Added features... [11/60]
Added features... [12/60]
Added features... [13/60]
Added features... [14/60]
Added features... [15/60]
Added features... [16/60]
Added features... [17/60]
Added features... [18/60]
Added features... [19/60]
Classifying Responses...
Flashes % Correct Predicted Symbols
1 50 CK1VWOLO
2 63 CKOVWOLO
3 100 HOVE IT
4 100 HOVE IT
5 100 HOVE IT
Donate!
Time elapsed [s]: 0.258
*****Applying Feature Weights*****
(Spatial Filter = Raw)
Loading file [1/1]...
Collecting Responses from file [1/1]...
Filtering and Decimating file [1/1]...
Targets: 40
Non Targets: 200
Classifying Responses...
Flashes % Correct Predicted Symbols
1 75 AOLA
2 100 HELLO
3 100 HELLO
4 100 HELLO
5 100 HELLO
Donate!
Time elapsed [s]: 0.114
Third cross-referencing of calibration files:
***** Generating Feature Weights *****
(Spatial Filter = Raw)
Loading file [1/2]...
Collecting Responses from file [1/2]...
Filtering and Decimating file [1/2]...
Loading file [2/2]...
Collecting Responses from file [2/2]...
Filtering and Decimating file [2/2]...
Targets: 80
Non Targets: 400
Applying Stepwisefit classifier...
Added feature... [1/60]
Added feature... [2/60]
Added feature... [3/60]
Added feature... [4/60]
Added feature... [5/60]
Added feature... [6/60]
Added feature... [7/60]
Added feature... [8/60]
Added feature... [9/60]
Added feature... [10/60]
Added feature... [11/60]
Added feature... [12/60]
Added feature... [13/60]
Added feature... [14/60]
Added feature... [15/60]
Added feature... [16/60]
Added feature... [17/60]
Added feature... [18/60]
Added feature... [19/60]
Added feature... [20/60]
Added feature... [21/60]
Added feature... [22/60]
Added feature... [23/60]
Added feature... [24/60]
Added feature... [25/60]
Classifying Responses...
Flashes % Correct Predicted Symbols
1 50 7KOVBOLD
2 75 DKOVHOLA
3 88 AKOVHOLA
4 100 AHOVHOLA
5 100 AHOVHOLA
Done!
Time elapsed [s]: 0.271
***** Applying Feature Weights *****
(Spatial Filter = Raw)
Loading file [1/1]...
Collecting Responses from file [1/1]...
Filtering and Decimating file [1/1]...
Targets: 40
Non Targets: 200
Classifying Responses...
Flashes % Correct Predicted Symbols
1 25 WML8
2 100 EOLO
3 100 EOLO
4 100 EOLO
5 100 EOLO
Done!
Time elapsed [s]: 0.111
And this would be a "bad" session:
First crossing:
***** Generating Feature Weights *****
(Spatial Filter = Raw)
Loading file [1/2]...
Collecting Responses from file [1/2]...
Filtering and Decimating file [1/2]...
Loading file [2/2]...
Collecting Responses from file [2/2]...
Filtering and Decimating file [2/2]...
Targets: 80
Non Targets: 400
Applying Stepwisefit classifier...
Added feature... [1/60]
Added feature... [2/60]
Added feature... [3/60]
Added feature... [4/60]
Added feature... [5/60]
Added feature... [6/60]
Added feature... [7/60]
Added feature... [8/60]
Added feature... [9/60]
Added feature... [10/60]
Classifying Responses...
Flashes % Correct Predicted Symbols
1 25 TDKMFO_O
2 50 FOKMEO_O
3 38 TILBEM_O
4 63 TOHAEOLR
5 63 TOHAQOLO
Done!
Time elapsed [s]: 0.277
***** Applying Feature Weights *****
(Spatial Filter = Raw)
Loading file [1/1]...
Collecting Responses from file [1/1]...
Filtering and Decimating file [1/1]...
Targets: 40
Non Targets: 200
Classifying Responses...
Flashes % Correct Predicted Symbols
1 0 LZQM
2 0 G2RP
3 0 G2RP
4 0 G4RP
5 25 A2RM
Done!
Time elapsed [s]: 0.113
Second crossing;
***** Generating Feature Weights *****
(Spatial Filter = Raw)
Loading file [1/2]...
Collecting Responses from file [1/2]...
Filtering and Decimating file [1/2]...
Loading file [2/2]...
Collecting Responses from file [2/2]...
Filtering and Decimating file [2/2]...
Targets: 80
Non Targets: 400
Applying Stepwisefit classifier...
Added feature... [1/60]
Added feature... [2/60]
Added feature... [3/60]
Added feature... [4/60]
Added feature... [5/60]
Added feature... [6/60]
Added feature... [7/60]
Added feature... [8/60]
Added feature... [9/60]
Added feature... [10/60]
Added feature... [11/60]
Added feature... [12/60]
Added feature... [13/60]
Added feature... [14/60]
Added feature... [15/60]
Added feature... [16/60]
Classifying Responses...
Flashes % Correct Predicted Symbols
1 38 GNIVFULO
2 75 AHHVKOLO
3 88 AZOVEOLO
4 88 AZOVEOLO
5 88 AZOVEOLO
Done!
Time elapsed [s]: 0.277
***** Applying Feature Weights *****
(Spatial Filter = Raw)
Loading file [1/1]...
Collecting Responses from file [1/1]...
Filtering and Decimating file [1/1]...
Targets: 40
Non Targets: 200
Classifying Responses...
Flashes % Correct Predicted Symbols
1 0 LSQX
2 0 LURF
3 25 HUFF
4 50 H1LF
5 25 H1FF
Done!
Time elapsed [s]: 0.112
Third crossing:
***** Generating Feature Weights *****
(Spatial Filter = Raw)
Loading file [1/2]...
Collecting Responses from file [1/2]...
Filtering and Decimating file [1/2]...
Loading file [2/2]...
Collecting Responses from file [2/2]...
Filtering and Decimating file [2/2]...
Targets: 80
Non Targets: 400
Applying Stepwisefit classifier...
Added feature... [1/60]
Added feature... [2/60]
Added feature... [3/60]
Added feature... [4/60]
Added feature... [5/60]
Added feature... [6/60]
Added feature... [7/60]
Added feature... [8/60]
Added feature... [9/60]
Added feature... [10/60]
Added feature... [11/60]
Added feature... [12/60]
Added feature... [13/60]
Added feature... [14/60]
Added feature... [15/60]
Added feature... [16/60]
Added feature... [17/60]
Added feature... [18/60]
Added feature... [19/60]
Added feature... [20/60]
Added feature... [21/60]
Added feature... [22/60]
Classifying Responses...
Flashes % Correct Predicted Symbols
1 25 AIIXTDHA
2 75 AHOVTELA
3 88 AHOVHCLA
4 100 AHOVHOLA
5 88 AZOVHOLA
Done!
Time elapsed [s]: 0.286
***** Applying Feature Weights *****
(Spatial Filter = Raw)
Loading file [1/1]...
Collecting Responses from file [1/1]...
Filtering and Decimating file [1/1]...
Targets: 40
Non Targets: 200
Classifying Responses...
Flashes % Correct Predicted Symbols
1 75 EO_O
2 50 EORQ
3 50 EO_R
4 50 EO_R
5 75 EOXO
Done!
Time elapsed [s]: 0.114
I don't know what's causing these zero values. I'm thinking it might be a synchronization or latency issue between Unicorn and BCI2000. Can anyone help me or tell me if they've had similar problems?
Thanks.
