Relation between sampling rate and recording length (Memory)

Forum for software developers to discuss BCI2000 software development
Locked
slee87
Posts: 11
Joined: 20 Sep 2009, 15:49

Relation between sampling rate and recording length (Memory)

Post by slee87 » 07 Aug 2011, 16:04

Previously, I have a difficulty in increasing recording time on my system. The hardware we designed has a sampling rate of 540 k sample per second (540 kSps) consisting of 36 channels. Therefore, each channel has a sampling rate of 15 kSps. When I press 'Start' button in operate module in order to record data acquisition signals into files, I found that only 2081280 points per channel can be recorded. In this case, around 2 minutes of data (100 MB).

I found interesting phenomenon. If I increased sampling rate in BCI2000 configuration although hardware real sampling rate does not change (15 kHz per channel), then I can record more data. For example, if I type 200 kHz of sampling rate in BCI2000 configuration, then I can record up to 23 minutes (2GB).

I am wondering why this happen, and how can I fix this issue completely.

In addition, I cannot load 2GB of recording data. In MATLAB, when I use 'load_bcidat' function, I faced 'Out of Memory' error. BCI2000 viewer also does not work.

Thank you very much.

mellinger
Posts: 1065
Joined: 12 Feb 2003, 11:06

Re: Relation between sampling rate and recording length (Mem

Post by mellinger » 08 Aug 2011, 10:39

Hi,

it would help if you could post some more details about the problem. From your post, I read that BCI2000 records some data, and then stops. How does it stop? Is there any error or warning message displayed in the Operator log window? Or does the system just go into suspended state, i.e. does the "Start" button in the Operator read "Resume" when recording stops?

When the latter is the case, one possible cause could be that your data acquisition code, or the hardware driver, writes past the end of allocated memory, inadvertently overwriting the "Running" state's memory location with zero. Setting the "Running" state to zero will cause BCI2000 to stop recording, and go into suspended state.
You should check whether your code allocates buffers that are large enough to hold all data written by the hardware driver. Make sure not to confound buffer size in bytes with buffer size specified in terms of data points.

Regarding the Matlab problem: I guess you are using 32-bit Matlab. 32-bit Matlab has limited access to memory, and cannot allocate the memory required to hold all the data contained in a 2GB BCI2000 file.

It would be valuable to know more about the problem with BCI2000 viewer. What does it say when you try to load the 2GB file?

Best regards,
Juergen

slee87
Posts: 11
Joined: 20 Sep 2009, 15:49

Re: Relation between sampling rate and recording length (Mem

Post by slee87 » 08 Aug 2011, 14:06

Thank you very much.

After some period of time, recording automatically finishes without any specific error. 'Suspend' changes automatically to 'Resume' and recording stops.

For BCI2000 viewer, the error message is 'External exception EEFFACE.'.

I attached my data acquisition module important code.
I will try to understand your recommendation related to buffer.

Thank you very much.

// **************************************************************************
// Function: Initialize
// Purpose: This function parameterizes the CyAPIADC.
// It is called each time parameters have been changed.
// Parameters: References to input (ignored) and output signal properties.
// Returns: N/A
// **************************************************************************
void CyAPIADC::Initialize( const SignalProperties&, const SignalProperties& )
{
mSourceCh = Parameter( "SourceCh" );
mSampleBlockSize = Parameter( "SampleBlockSize" );
mSamplingRate = Parameter( "SamplingRate" );

int devices = USBDevice->DeviceCount();
USHORT vID, pID;
int d = 0;

do{
USBDevice->Open(d); // Open automatically calls Close( ) if necessary
vID = USBDevice->VendorID;
pID = USBDevice->ProductID;
d++;
} while ((d < devices ) && ((vID != 0x04B4) || (pID != 0x8613)));

isLoaded = TRUE;

// Recording length
alen=mSourceCh*mSampleBlockSize*2;
buf=(BYTE *) malloc (sizeof(BYTE) * alen);
postBuf=(int *) malloc (sizeof (int) * mSourceCh);
ZeroMemory(buf, alen);
ZeroMemory(postBuf, mSourceCh);
}


// **************************************************************************
// Function: Process
// Purpose: This function is called within the data acquisition loop
// it fills its output signal with values
// and does not return until all data has been acquired.
// Parameters: References to input signal (ignored) and output signal.
// Returns: N/A
// **************************************************************************
void CyAPIADC::Process( const GenericSignal&, GenericSignal& outputSignal )
{
short buffer;

if (BulkInPipe5->XferData(buf,alen))
{
// firstCh = 0 // Put first sample to last sample

// Put first sample
for (int channel = 0; channel < mSourceCh; channel++) {
outputSignal(channel,0)=postBuf[channel];
}

// Put second sample to one sample before last sample
for( int sample = 1; sample < mSampleBlockSize; ++sample ) {
for( int channel = 0; channel < mSourceCh; ++channel ) {
buffer=(int)buf[sample*mSourceCh*2+channel*2+1]*256+(int)buf[sample*mSourceCh*2+channel*2+0];
outputSignal( channel, sample ) = buffer;
}
}

}
else
{
bcierr << "Error reading data" << endl;
bcierr << "mSourceCh: " <<mSourceCh<<" mSampleBlockSize: "<<mSampleBlockSize<<" mSamplingRate: "<<mSamplingRate<< endl;
bcierr << "alen: "<<alen<<endl;
}
}

mellinger
Posts: 1065
Joined: 12 Feb 2003, 11:06

Re: Relation between sampling rate and recording length (Mem

Post by mellinger » 08 Aug 2011, 15:12

Thank you for posting your code.

From the way the buf variable is used, I guess that the following line is wrong:
buf=(BYTE *) malloc (sizeof(BYTE) * alen);
and should read
buf=(BYTE *) malloc (sizeof(int) * alen);

Besides that, I cannot understand what the postBuf buffer is good for.

HTH,
Juergen

slee87
Posts: 11
Joined: 20 Sep 2009, 15:49

Re: Relation between sampling rate and recording length (Mem

Post by slee87 » 09 Aug 2011, 02:25

Thank you very much for your suggestion.
I did not understand your assumption fully yet, but it seems very reasonable.

I deleted postBuf because it is not necessary for this version, and tried your suggested code,

buf=(BYTE *) malloc (sizeof(int) * alen);

instead of

buf=(BYTE *) malloc (sizeof(BYTE) * alen);

However, it still shows the same result.

mellinger
Posts: 1065
Joined: 12 Feb 2003, 11:06

Re: Relation between sampling rate and recording length (Mem

Post by mellinger » 10 Aug 2011, 07:04

So the result is that it runs for some time, and then goes into suspended state? Does it always record the same amount of data, or a different amount each time? Which Signal Processing and Application modules are you using?

slee87
Posts: 11
Joined: 20 Sep 2009, 15:49

Re: Relation between sampling rate and recording length (Mem

Post by slee87 » 10 Aug 2011, 11:04

So the result is that it runs for some time, and then goes into suspended state? Does it always record the same amount of data, or a different amount each time? Which Signal Processing and Application modules are you using?

Yes, it always record the same amount of data. However, interestingly the amount of recording time and data depend on sampling rate in configuration. Hardware sampling rate is always fixed to around 20 kHz. If I type 20 kHz in BCI2000 configuration, then recording time is around 2 min (100 MB). If I type 40 kHz in BCI2000 configuration, then recording time becomes 4 min (200 MB). I tested up to 200 kHz and it will record around 20 Minutes.

Signal Processing module: ARSignalProcessing.exe
Application modules: CursorTask.exe

gschalk
Posts: 615
Joined: 28 Jan 2003, 12:37

Re: Relation between sampling rate and recording length (Mem

Post by gschalk » 10 Aug 2011, 19:45

Hi,

If your hardware sampling rate is 20kHz, you also need to tell BCI2000 that it's 20 kHz. All calculations of time are based on the sampling rate, so if you specify a sampling rate that's not correct, the apparent length of the data will change.

Also, in the standard configuration of the CursorTask, the application suspends after about 2 or 3 minutes. This is specified by some parameter in the Application tab (don't remember offhand, but just check). This is most likely why it simply suspends.

Gerv

mellinger
Posts: 1065
Joined: 12 Feb 2003, 11:06

Re: Relation between sampling rate and recording length (Mem

Post by mellinger » 11 Aug 2011, 08:51

What you describe is the expected behavior of the CursorTask application module. It stops recording after a predefined amount of time, as described at
http://www.bci2000.org/wiki/index.php/U ... nRunLength

The application module measures time by counting the number of sample blocks it receives. To convert from seconds into the number of sample blocks, it uses the SamplingRate parameter. When you enter a wrong sampling rate, the application module's time measurement becomes incorrect, so the actual run duration varies with the SamplingRate parameter.

In existing BCI2000 source modules, the SamplingRate parameter is restricted to values that are actually supported by recording hardware, so it is not possible to accidentally enter a wrong value there.

Regards,
Juergen

Locked

Who is online

Users browsing this forum: Google [Bot] and 1 guest