Data acquisition buffer overflow

Known Issues and Problems with BCI2000
Locked
lucamossa
Posts: 19
Joined: 27 Jan 2014, 08:46

Data acquisition buffer overflow

Post by lucamossa » 10 Apr 2014, 10:13

Hi, I tried to implement a simple Matlab script (highpass Filter) but when I pushed the START button appears a warning:

Warning: virtual int BufferedADC::OnExecute(): Data acquisition buffer overflow.

In the source module I set the following parameters:

SourceBuffersize 2s
NumBuffers 6
NumBuffersAmp 6

and my .bat file is:

#! ../prog/BCI2000Shell
@cls & ..\prog\BCI2000Shell %0 %* #! && exit /b 0 || exit /b 1

Change directory $BCI2000LAUNCHDIR
Show window; Set title ${Extract file base $0}
Reset system
Startup system localhost
Start executable gUsbAmpUnix --local
Start executable MatlabSignalProcessing --local --MatlabWD=matlab
Start executable StimulusPresentation --local
Wait for Connected
Load parameterfile "../parms/examples/StimulusPresentation_SignalGenerator.prm"
Load parameterfile "../parms/fragments/amplifiers/gUSBamp.prm"

Someone have any idea? I know that Matlab could have some problem in real time analysis but my Matlab Script is very simple.
Thanks.

Best regards.

Luca Mossa

boulay
Posts: 382
Joined: 25 Dec 2011, 21:14

Re: Data acquisition buffer overflow

Post by boulay » 10 Apr 2014, 21:13

Buffer overflow happens when you are not pulling data out of the buffer fast enough. This can happen if your round trip time (including the time to pull the data from the buffer, the time to process one block of data, and the time to update the feedback) is longer than your block time (i.e., the number of samples you pull out of the buffer). One way to fix this is to increase your block time. However, longer block time means the feedback is updated less frequently, so there's a tradeoff there.

I usually start with a block time as long as I can manage while keeping the feedback smooth. Then, if my round trip time is still too slow, I optimize the signal processing to make it faster.

Of course, you will also get buffer overflow when either your signal processing or application stall and therefore do not allow the next block to pull data out of the buffer. If they do not produce helpful errors to let you know they've stalled, then it can be difficult to identify the source of the error.

lucamossa
Posts: 19
Joined: 27 Jan 2014, 08:46

Re: Data acquisition buffer overflow

Post by lucamossa » 11 Apr 2014, 10:08

I Chad, thanks for your reply. I set:

SampleBlockSize : 8
SamplingRate: 256

The warning above occurs only when I use MatlabSignalProcessing and if I edit these values of sampling rate and sample block size I can't fix the issues.
Do you think that using FieldTripBuffer could be a possible solution?


Ps: when I start my .bat file (in the first post) initially appears the initial screen of matlab and then disappear (Matlab result close). At the same time appear the operator module. Do you think that this behavior is wrong? i.e. Matlab should remain open?

Best regards.

Luca

boulay
Posts: 382
Joined: 25 Dec 2011, 21:14

Re: Data acquisition buffer overflow

Post by boulay » 12 Apr 2014, 23:34

Matlab should remain open but it's the Matlab engine, not the Matlab desktop mode. The Matlab engine, at least in Windows, usually has a small application on the taskbar. I'm not sure what it would look like in linux. Can you check to see if it's still running?

lucamossa
Posts: 19
Joined: 27 Jan 2014, 08:46

Re: Data acquisition buffer overflow

Post by lucamossa » 14 Apr 2014, 05:49

Hi Chad, I really appreciate your help.
The Matlab process it's open when I open the operator module, I checked in the System Monitor.
So, I don't know why the error appear. I also tryed to change my Matlab code with the same present in Programming_Tutorial:Implementing_a_Matlab-based_Filter but the same warning appear:

Warning: virtual int BufferedADC::OnExecute(): Data acquisition buffer overflow.

Have you some hints?

Best regards.

Luca

boulay
Posts: 382
Joined: 25 Dec 2011, 21:14

Re: Data acquisition buffer overflow

Post by boulay » 15 Apr 2014, 10:17

It would help if I had the gUsbAmpUnix source and a Unix box.

First, if you switch to a different signal source (SignalGenerator), do you get the same problem? I guess that you will still have the same problem.

To check that your matlab script is functioning, do something permanent in the bci_Process matlab script. For example, (over)write a file with the current date and the block index or something similar. If that file gets written, then you know that Matlab is doing something. I'm guessing it does not.

What revision of the source code are you using? In a terminal, switch to the root of the bci2000 directory and do

Code: Select all

svn info
.

Sometime between the last "release" and the latest code, Juergen changed the way the MatlabFilter works. It now uses shared memory instead of UDP packets. Is it possible that one of your processes does not have access to the shared memory? I'm guessing there is an issue somewhere with permissions. You may have to run the MatlabSignalProcessing and the matlab engine as super user.

lucamossa
Posts: 19
Joined: 27 Jan 2014, 08:46

Re: Data acquisition buffer overflow

Post by lucamossa » 17 Apr 2014, 05:55

Hi Chad, I switched to SignalGenerator but the situation remains the same.
I don't understand what do you mean with "something permanent". Do you mean to write something wrong in matlab bci_process scritp and see if the errors are detected?
My bci revision is 4440.

I finally changed the permissions of MatlabSignalProcessing, matlab executables (and all the files in path /usr/local/MATLAB/R2011b) and I tryed to apply command SUDO with my .bat file but I visualized the following error:

MatlabFilter::Constructor: Could not open Matlab engine (maybe you need to run 'matlab /regserver'?)
2014-04-16T17:28:31 - MatlabFilter::Constructor: Could not connect to Matlab engine. Please make sure that Matlab is available on your machine.
On Windows, Matlab's bin/win32 directory must be on your system's %PATH% variable, and "Matlab /regserver" must have been executed with administrative privileges.
2014-04-16T17:28:31 - Script error: /home/user/Desktop/Trunk/batch/./StimulusPresentation_gUSBamp.bat, line 41: Wait aborted

I tryed to add to my system path the following path:

/usr/local/MATLAB/R2011b/bin/glnxa64

but the previous error appears again.

If I don't apply the command SUDO the situation remain the same, i.e I don't visualize error but if I push the Start button the following warning appears again:

Warning: virtual int BufferedADC::OnExecute(): Data acquisition buffer overflow

Do you think that installing FieldTrip Buffer can fix the problem?

Thanks for your your time.
Best regards.

Luca

boulay
Posts: 382
Joined: 25 Dec 2011, 21:14

Re: Data acquisition buffer overflow

Post by boulay » 17 Apr 2014, 12:00

What I wanted you to test previously was to put a line of code in each of your Matlab scripts that writes a datafile to disk whenever it is run. This way, you can know whether or not the scripts are being called. I am guessing that they aren't. It seems like BCI2000 is not communicating with the Matlab engine.

The revision you are on is before Juergen changed to using shared memory, so I guess it is still using UDP packets. There still might be permission issues, but that would be related to opening ports rather than accessing shared memory.

There are various ways to get the matlab engine running. One of them is to run matlab /regserver prior to launching BCI2000. You can also do this as super user. Then, you can try launching the BCI2000 batch file as super user. If that still doesn't work then I'm afraid I can't help you anymore. There's only so much I can do without a linux box.

You might want to try the newest BCI2000 source code. Do this in a different directory. I'm guessing some of the newer parts of BCI2000 won't work in linux. But, if you can get the MatlabSignalProcessing module to compile, you can try copying it back to your original directory and using it there.

Good luck. Keep us updated with your progress.

lucamossa
Posts: 19
Joined: 27 Jan 2014, 08:46

Re: Data acquisition buffer overflow

Post by lucamossa » 23 Apr 2014, 11:44

Hi Chad, I tryed to write some code that save some variables from the current workspace in a text file. When I push the START button the same warning appears but the text file is generated in the directory (the same directory of the file bci_Construct.m, bci_Initialize, etc). So, I think that BCI2000 is communicating with the Matlab engine.
Now I'm trying to use the latest version of BCI; I'll keep you updated.

Thank you very much Chad.

Ps: Have you any knowledge about FieldTrip? For the moment I haven't problem to acquisition of the signal (another proof that bci communicates with matlab engine), but I don't know if it is possibile resending the processed signal to BCI for the classification.

Best regards.

boulay
Posts: 382
Joined: 25 Dec 2011, 21:14

Re: Data acquisition buffer overflow

Post by boulay » 23 Apr 2014, 14:27

I've only used FieldTrip as a standalone analysis toolbox for Matlab.

I suppose you could build a new SignalProcessing module that chained together both the MatlabFilter and the FieldTripBuffer, but this seems terribly inefficient.

Most BCIs collect training data then train a classifier offline. This classifier is static in terms of how the signals are conditioned and combined to yield features. A signal segment is then classified based on the feature(s) values being above or below some threshold. Sometimes the non-stationarity of EEG makes it necessary to adapt the threshold and/or feature gain using BCI2000's Normalizer.

If your classification scheme is similar to above, then you can probably find most of what you need in BCI2000 already. The difficult part is only in training the classifier which is typically done offline in Matlab or Python or something similar.

However, if you have an adaptive classifier (e.g., update the inverse solution or update ICA weights based on accumulated signals) then you'll need to either write your own SignalProcessing filters or to use an external application. Matlab with/without FieldTrip could be very useful. I would suggest that you do as much of the signal conditioning and feature extraction as possible in C++ before you pass your signal off to Matlab, and that you do all the remaining processing in a single trip to Matlab/FieldTrip filter.

lucamossa
Posts: 19
Joined: 27 Jan 2014, 08:46

Re: Data acquisition buffer overflow

Post by lucamossa » 05 May 2014, 06:41

hello chad, thanks for your hints.

now i'm trying to work with fieldtrip, it is my last chance to use bci2000 with matlab in a real-time application.

for this purpose I open a new post.

thanks again

luca

Locked

Who is online

Users browsing this forum: No registered users and 15 guests