Combination of BCI2000 with Fieldtrip

Forum for discussion on different signal processing algorithms
Locked
vladi
Posts: 18
Joined: 04 Jan 2012, 08:29

Combination of BCI2000 with Fieldtrip

Post by vladi » 29 Jun 2012, 10:55

Hallo,

I try to combine BCI2000 with Fieldtrip in order to design a realtime application. I managed to transfer the eeg data from BCI2000 to Matlab using the FieldTripBuffer.
What I would like to do now is to send a trigger from Matlab back to BCI2000. Therefore I looked in to the Closing the loop part from the Programming Tutorial:Working with the FieldTrip buffer but unfortunately I get the following error in matlab when I try to execute the code:

Error using buffer
field 'offset' is missing

Error in ft_write_event (line 107)
buffer('put_evt', event, host, port);

Error in write_event (line 18)
[varargout{1:nargout}] = funhandle(varargin{:});

Do I have to change something in the configuration of the FieldTripBuffer in BCI2000?

Regards,
Vladi

mellinger
Posts: 1210
Joined: 12 Feb 2003, 11:06

Re: Combination of BCI2000 with Fieldtrip

Post by mellinger » 29 Jun 2012, 11:30

Hi,

it seems you are missing an event.offset struct member in Matlab.

Apparently, the behavior of the FieldTrip write_event() function has been changed since the tutorial was written. Maybe there is a special value, such as -1, for the offset field, that indicates "now".
Please check the FieldTrip documentation of write_event() in order to obtain relevant information.

I would appreciate if you posted your solution, so I could fix the tutorial.

Regards,
Juergen

vladi
Posts: 18
Joined: 04 Jan 2012, 08:29

Re: Combination of BCI2000 with Fieldtrip

Post by vladi » 02 Jul 2012, 04:49

Hallo,

Thank you for your reply. It truly seems that the the function was changed. Furthermore the function name was changed in ft_write_event. The function write_event is just used as a backward compatibility wrapper. Additionally you have to add the following fields:

% event.type string
% event.sample expressed in samples, the first sample of a recording is 1
% event.value number or string
% event.offset expressed in samples
% event.duration expressed in samples
% event.timestamp expressed in timestamp units, which vary over systems (optional)

Regards,
Vladi

mellinger
Posts: 1210
Joined: 12 Feb 2003, 11:06

Re: Combination of BCI2000 with Fieldtrip

Post by mellinger » 02 Jul 2012, 07:30

Hi Vladi,
thanks for your answer. I would very much appreciate if you could provide me with a few more details. Did you make the tutorial code work? Is it necessary to fill in the additional fields, or are they just required to exist? If it is necessary to fill these fields in, how did you determine the right values?

Thanks in advance,
Juergen

vladi
Posts: 18
Joined: 04 Jan 2012, 08:29

Re: Combination of BCI2000 with Fieldtrip

Post by vladi » 02 Jul 2012, 11:28

Hallo Juergen,

the code from tutorial worked fine. I just used ft_write_event instead of write_event and added the additional fields to the structure. I would guess that the field offset can be used to manage a delay for activation of the trigger and that with duration one would be able to define how long the effect of the trigger should persist. But I am not sure because I tested different values for those fields an it didn't changed anything. So it is enough just to add those to the structure without any value. Please let me now when you have further information on this matter.


Thank you and Regards,
Vladi

mellinger
Posts: 1210
Joined: 12 Feb 2003, 11:06

Re: Combination of BCI2000 with Fieldtrip

Post by mellinger » 02 Jul 2012, 11:53

Hi Vladi,

thanks for the clarification. I added your changes to the tutorial.

Best regards,
Juergen

omidsani
Posts: 5
Joined: 18 Jan 2013, 12:07

Re: Combination of BCI2000 with Fieldtrip

Post by omidsani » 03 Feb 2013, 02:26

First I would like to thank anyone involved in developing BCI2000. It's awesome guys! Thank you all.

I am trying to set up an online P300 Speller. The signal acquisition is done through Emotiv and then BCI2000 transfers the data to MATAB using fieldtrip buffer. My classification method works fine with offline data. My problem is that I can not send the predicted result to P300 Speller. (I have also tried using MATLAB filter before but failed at the same step.)
As a wrap up (!), I don't know how to command P300Speller program to update its TextResult field with my predictions after each sequence. How may I do so?
I try writing events using FieldTripBuffer's ft_write_event method but I don't know what type of event P300 Speller is expecting.
Last edited by omidsani on 08 Sep 2013, 18:03, edited 1 time in total.

mellinger
Posts: 1210
Joined: 12 Feb 2003, 11:06

Re: Combination of BCI2000 with Fieldtrip

Post by mellinger » 04 Feb 2013, 09:01

Hi,

the relevant documentation is that for the P3TemporalFilter:
http://www.bci2000.org/wiki/index.php/U ... oralFilter

The P3TemporalFilter is a generic epoch averager, and does not know about P3Speller choices. Rather, it averages over responses for each stimulus code (row or column) separately, and then writes out a result, using fhe state "StimulusCodeRes" to indicate the stimulus code.

The output of the P3TemporalFilter is linked to the LinearClassifier, which applies its weights, resulting in a single number, which is then sent to the application module. The application module is in charge of defining stimulus codes, and presenting stimuli, so it can determine the user's choice from comparing results across stimulus codes, or combining them into a single choice, as appropriate.

Some background information about the interpretation of SignalProcessing module output is here:
http://bci2000.org/phpbb/viewtopic.php?f=4&t=1013

So, the answer to your question is that your signal processing should determine something that approximates a log-likelihood ratio for a stimulus to have elicited a P300, and send that to the application module, as a BCI2000 signal with just a single entry, and with the StimulusCodeRes state set to the stimulus code that was present in the StimulusCode state at the time when the signal was recorded. If the output of your classifier is binary, just send 1 for the stimulus chosen by the classifier, and 0 for all others.

You need to send a result for each stimulus code that has been presented, but the order does not matter. You can only send a single result per data block, so you will need to use multiple data blocks in sequence if results are only available after all stimuli have been seed by the classifier.

HTH,
Juergen

omidsani
Posts: 5
Joined: 18 Jan 2013, 12:07

Re: Combination of BCI2000 with Fieldtrip

Post by omidsani » 06 Feb 2013, 07:26

Thanks mellinger, I highly appreciate your informative response. I managed to make the P3Speller Application to write out something but I can't make it to write what I want. Apparently, there's a misunderstanding between my classifier and the P3Speller App!
I have done these:
- I added StimulusCodeRes state to field trip's FTStatesFromBuffer so that I can set it through field trip events. However, I got an error indicating that Field Trip's Preflight couldn't access StimulusCodeRes.
- I added the StimulusCodeRes state using operators command line scripting by adding the following line to the startup bat:

Code: Select all

INSERT STATE StimulusCodeRes 8 0
I wanted to make it an 8 bit integer and that's why the first number in the code is 8. Am I right in defining this state value?

- Next, to send the results of classification to P3Speller task, I did the following in my classifier code:

Code: Select all

% [MATLAB Code]
eventBackCount = 0;
for stimCode = 1:12                
  eventBackCount = eventBackCount  + 1;
  % Setting StimulusCodeRes to stimCode
  myEvent = struct('type', 'StimulusCodeRes', 'sample', lastEventedSample+BCI2000BlockSize*eventBackCount, 'value', stimCode, 'offset', 0, 'duration', 1);
  ft_write_event(filename, myEvent); 

  % Sending the score associated with stimCode to the next block in BCI2000 (The block after fieldtrip)
  myEvent = struct('type', 'Signal', 'sample', lastEventedSample+BCI2000BlockSize*eventBackCount, 'value', uint8(floor(StimScore(stimCode)*256)), 'offset', 0, 'duration', 1);
  ft_write_event(filename, myEvent);

  % Making StimulusCodeRes zero again [states stay where they are until we reset them and we don't want to report thousands of stimCodes again and again]
  eventBackCount = eventBackCount  + 1;
  myEvent = struct('type', 'StimulusCodeRes', 'sample', lastEventedSample+BCI2000BlockSize*eventBackCount+1, 'value', stimCode, 'offset', 0, 'duration', 1);
  ft_write_event(filename, myEvent);
end
- filename is the FieldTrip port and ft_write_event(filename, myEvent) writes the event to fieldtrip buffer so that BCI2000 would read it.
- stimCode rolls over 1 to 12 to report the log likelihood of having elicited a P300 for each of the 12 stimulus codes. StimScore(stimCode) contains the likelihood value (for stimulus of code stimCode), which is a number between 0 and 1. I scale it to 0-256 and convert it to uint8 so that it matches with what I defined as the StimulusCodeRes state.
- lastEventedSample is the last sample at which an event was received. It supposedly is the first sample after the sequence has finished (where PhaseInSequence becomes 3).
- BCI2000BlockSize is the block size defined in BCI2000's source module.
What I tried to do was to send the scores for the 12 stimulus types in 12 consecutive blocks to BCI2000. I tried different variations of the above code with different timing and I couldn't get it to work right. Sometimes [in most of the cases] P3Speller issues a warning for 11 of the stimulus types indicating that their score hasn't been received and a decision has been made without their likelihood values and then a letter (mostly irrelevant) is added to the Result Text. Sometimes P3Speller adds 12 (or even 14! or even 5!) instances of a letter to the Result at once!
Apparently, I am doing something wrong. How should I send the results? In what periods of the sequence P3Speller is looking for StimulusCodeRes values and when exactly it decides that it has enough StimulusCodeRes to write out a letter as a result?

mellinger
Posts: 1210
Joined: 12 Feb 2003, 11:06

Re: Combination of BCI2000 with Fieldtrip

Post by mellinger » 20 Feb 2013, 12:35

Hi,

your StimulusCodeRes definition is perfectly fine. For 12 stimulus codes, 4 bits would be sufficient, though.

The P3Speller expects likelihood values to be transmitted in its input signal, not as a state variable. The FieldTripBuffer filter allows you to specify the name of a certain FieldTrip event that should go into the FieldTripBuffer's output signal rather than into a state variable.
So you should set the FTOutputEventType parameter to the name of the FieldTrip event that contains the likelihood estimate:
http://www.bci2000.org/wiki/index.php/C ... TripBuffer

HTH,
Juergen

omidsani
Posts: 5
Joined: 18 Jan 2013, 12:07

Re: Combination of BCI2000 with Fieldtrip

Post by omidsani » 23 Feb 2013, 07:07

mellinger wrote:The P3Speller expects likelihood values to be transmitted in its input signal, not as a state variable. The FieldTripBuffer filter allows you to specify the name of a certain FieldTrip event that should go into the FieldTripBuffer's output signal rather than into a state variable.
So you should set the FTOutputEventType parameter to the name of the FieldTrip event that contains the likelihood estimate:
http://www.bci2000.org/wiki/index.php/C ... TripBuffer
That's exactly what I have done in the code. "Signal" is the event that contains likelihood values and it is already set in the FTOutputEventType parameter.

I think that the culprit is the timing of StimulusCodeRes events...
All suggestions are welcome...

mellinger
Posts: 1210
Joined: 12 Feb 2003, 11:06

Re: Combination of BCI2000 with Fieldtrip

Post by mellinger » 25 Feb 2013, 10:21

Your Matlab code is writing all FieldTrip events at once. You can only send a single event per BCI2000 data block. With your code, only the last value occurring in your loop will get transmitted to BCI2000.

The P3SignalProcessing module sends the likelihood value for each stimulus as soon as it is available, i.e. after the last presentation of that stimulus has occurred, and the epoch interval has passed. You can do that in a similar way, unless your classifier needs all responses to determine the likelihood for each single response (which should be conceptually independent of likelihoods for other responses, though).

HTH,
Juergen

Locked

Who is online

Users browsing this forum: No registered users and 7 guests