System Timings

[ariel]

We are using DT9816ADC as signal source. Because this board acquires channels simultaneously, we have modified the source code on
"\BCI2000__source_v2\src\contrib\SignalSource\DTADC\DTADC.cpp"
by altering the line
dtfun.ConfigAD( OL_CHNT_SINGLEENDED, channels, gain, 0, /*channels **/ samplerate, (UINT) channels * blocksize );
The problem is that when we look at the system timings, these are not as they should be.



When we use the SignalGenerator the system timings are correct.



What are we doing wrong?

Ariel

[gschalk]

Ariel,

I think you need to provide a little more information. What do you mean the board acquires channels simultaneously? If it simply has multiple sample-and-holds, but otherwise collects the same amount of data, then why would you need to change anything in the code?

Gerv

[ariel]

Thank for your reply Gerv, and sorry for that short info! Here is a little more information:

We are dealing with a board of six channels, but with six independent successive-approximation A/D converters, more precisely, a Data Transaltion DT9816 USB-Econo Serie (16-bit , ±10 V or ± 5 V, 50 kS/s per channel ).

We have made a few changes on the DT2000 (DTDAC) original source (SignalSource Contributions), like:
dtfun.ConfigAD( OL_CHNT_SINGLEENDED, channels, gain, 0, /*channels **/ samplerate, (UINT) channels * blocksize );

This, is because the original code is intended for a conventional board (only one A/D converter + multiplexer), and when number of channels is set to N >1, the result sample rate in our DT9816 board become N times the correct one.

With this modification and some other like incorporating a gain setting option, and recompiling using last DT library, the acquisition is working, but the problem is we can not find an adequate compromise solution between the "SamplingRate" the "SampleBlockSize" and our application timing requirement.

We have tested this in the last BCI2000 version, both in XP and Windows Vista, with the same results.
Any idea to try to improve this timing behavior?

Thank for your help!

[gschalk]

Ariel,

From what you describe, and aside from recompiling with the newest library, I only see one necessary difference between the original contribution and what's necessary for your board.

In the original contribution, the source module assumed that the board only has one sample-and-hold. Thus, the effective sampling rate on the board was desired_sampling_rate*num_channels. In your case, the board sampling rate equals the desired sampling rate. This is consistent with your observation that the sampling rate increased for num_channels>1.

Also, can you post larger versions of the two images you had posted before?

Gerv

[ariel]

Gerv

Thank for your new reply. Here the images in original size.

The timings with our acquisition board:


The timings with SignalGenerator:


Ariel

[gschalk]

Ariel,

Something is not right. The tick mark on the Y axis of the timing display corresponds to one sample block. If you acquire the correct number of samples at the correct sampling rate, the graph should be centered around the tick mark, which it is not. Thus, the system seems to be producing more data than it should. It's difficult to trouble shoot this from remote, except to say that this is definitely a problem inside the Source module, and not a BCI2000-related problem. It would be very helpful to record a known signal (e.g., a sine wave), so we can deduce from the signal display a possible error.

I hope this helps.

Gerv

[ariel]

We agree that there is a problem inside the Source module. However, this problem does not affect the acquisition of the signal, though it affect the times in which the visual stimuli are shown.

In the pictures you can see the signal acquired on channel 1 of our acquisition board. This signal comes from a fotoresistor mounted in a voltage divider. This is placed against the application on the screen of a CRT monitor.





System:Configuration matrix OperatorVersion= { Framework Revision Build } 1 2.0 2245,%202009/02/03%2015:58:46 2009/02/03%2019:09:02 // operator module version information
System:State%20Vector int StateVectorLength= 14 16 1 30 // length of the state vector in bytes
Source:Signal%20Properties:DataIOFilter int SourceCh= 6 16 1 % // number of digitized and stored channels
Source:Signal%20Properties:DataIOFilter int SampleBlockSize= 32 32 1 % // number of samples transmitted at a time
Source:Signal%20Properties:DataIOFilter int SamplingRate= 1024 256Hz 1 % // sample rate
Source:Signal%20Properties:DataIOFilter list ChannelNames= 0 // list of channel names
Source:Signal%20Properties:DataIOFilter floatlist SourceChOffset= 6 0 0 0 0 0 0 0 % % // Offset for channels in A/D units
Source:Signal%20Properties:DataIOFilter floatlist SourceChGain= 6 0.003 0.003 0.003 0.003 0.003 0.003 0.003 % % // gain for each channel (A/D units -> muV)
Storage:Data%20Location:DataIOFilter string DataDirectory= ..\data ..\data % % // path to top level data directory (directory)
Storage:Session:DataIOFilter string SubjectName= TimingsTest Name % % // subject alias
Storage:Session:DataIOFilter string SubjectSession= 001 001 % % // three-digit session number
Storage:Session:DataIOFilter string SubjectRun= 00 00 % % // two-digit run number
Storage:Documentation:DataIOFilter string ID_System= % // BCI2000 System Code
Storage:Documentation:DataIOFilter string ID_Amp= % // BCI2000 Amp Code
Storage:Documentation:DataIOFilter string ID_Montage= % // BCI2000 Cap Montage Code
Visualize:Timing:DataIOFilter int VisualizeTiming= 1 1 0 1 // visualize system timing (0=no, 1=yes) (boolean)
Visualize:Source%20Signal:DataIOFilter int VisualizeSource= 1 1 0 1 // visualize raw brain signal (0=no, 1=yes) (boolean)
Visualize:Source%20Signal:DataIOFilter int VisualizeSourceDecimation= 1 1 1 % // decimation factor for raw brain signal
Visualize:Source%20Signal:DataIOFilter int VisualizeSourceTime= 2 2 0 % // how much time in Source visualization
Visualize:Source%20Signal:DataIOFilter int SourceMin= -100muV -100muV % % // raw signal vis Min Value
Visualize:Source%20Signal:DataIOFilter int SourceMax= 100muV 100muV % % // raw signal vis Max Value
Source:DT98ADC int SoftwareChBoard1= 6 64 1 % // # channels board1 (ignored when only one board)
Source:DT98ADC int SoftwareChBoard2= 6 64 1 % // # channels board2 (ignored when only one board)
Source:DT98ADC int Gain= 1 2 1 % // Gain 1:10V, 2:5V
Source:DT98ADC int TransmitCh= 6 5 1 % // this is the number of transmitted channels
Source:DT98ADC string BoardName= DT9816(00) // this is the name of the AD board
Source:DT98ADC string BoardName2= none // the name of the 2nd AD board or 'none' if not present
Storage:Documentation:BCI2000FileWriter int SavePrmFile= 0 1 0 1 // save additional parameter file (0=no, 1=yes) (boolean)
Storage:Documentation:BCI2000FileWriter string StorageTime= % // time of beginning of data storage
Source:Alignment:AlignmentFilter int AlignChannels= 1 0 0 1 // align channels in time (0=no, 1=yes)
Source:Alignment:AlignmentFilter floatlist SourceChTimeOffset= 0 0 % % // time offsets for all source channels (may be empty)
Source:Online%20Processing:TransmissionFilter list TransmitChList= 6 1 2 3 4 5 6 // list of transmitted channels
System:Core%20Connections string SignalSourcePort= 3737 4200 1024 65535 // the SignalSource module's listening port
System:Core%20Connections string SignalSourceIP= 192.168.0.156 127.0.0.1 % % // the SignalSource module's listening IP
System:Configuration matrix SignalSourceVersion= { Framework Revision Build } 1 2.x % Mar%20%203%202009 // SignalSource version information
System:Configuration matrix SignalSourceFilterChain= 8 { Filter%20Name Position%20String } DataIOFilter 0 DT98ADC 1 NullFileWriter 1 BCI2000FileWriter 1 GDFFileWriter 1 EDFFileWriter 1 AlignmentFilter 1.1 TransmissionFilter 1.2 // SignalSource filter chain
Visualize:Processing%20Stages int VisualizeAlignmentFilter= 0 0 0 1 // Visualize AlignmentFilter output (boolean)
Visualize:Processing%20Stages int VisualizeTransmissionFilter= 0 0 0 1 // Visualize TransmissionFilter output (boolean)
Filtering:SpatialFilter int SpatialFilterType= 1 2 0 3 // spatial filter type 0: none, 1: full matrix, 2: sparse matrix, 3: common average reference (CAR) (enumeration)
Filtering:SpatialFilter:SpatialFilter intlist SpatialFilterCAROutput= 0 // list of output channels for the CAR if used
Filtering:SpatialFilter:SpatialFilter matrix SpatialFilter= 6 6 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 % % // columns represent input channels, rows represent output channels
Filtering:P3TemporalFilter int EpochLength= 500ms 500ms 0 % // Length of data epoch from stimulus onset
Filtering:P3TemporalFilter int EpochsToAverage= 15 1 0 % // Number of epochs to average
Visualize:P3TemporalFilter int VisualizeP3TemporalFiltering= 1 0 0 1 // Visualize averaged epochs (0=no 1=yes) (boolean)
Visualize:P3TemporalFilter int TargetERPChannel= 1 1 0 128 // Input channel for which to display ERP data
Filtering:LinearClassifier matrix Classifier= 2 { input%20channel input%20element%20(bin) output%20channel weight } 1 4 1 1 1 6 2 1 // Linear classification matrix in sparse representation
System:Core%20Connections string SignalProcessingPort= 3739 4200 1024 65535 // the SignalProcessing module's listening port
System:Core%20Connections string SignalProcessingIP= 192.168.0.156 127.0.0.1 % % // the SignalProcessing module's listening IP
System:Configuration matrix SignalProcessingVersion= { Framework Revision Build } 1 2.0 2245,%202009/02/03%2015:58:46 2009/02/03%2019:09:02 // SignalProcessing version information
System:Configuration matrix SignalProcessingFilterChain= 3 { Filter%20Name Position%20String } SpatialFilter 2.A P3TemporalFilter 2.B LinearClassifier 2.C // SignalProcessing filter chain
Visualize:Processing%20Stages int VisualizeSpatialFilter= 0 0 0 1 // Visualize SpatialFilter output (boolean)
Visualize:Processing%20Stages int VisualizeLinearClassifier= 0 0 0 1 // Visualize LinearClassifier output (boolean)
Connector:ConnectorInput list ConnectorInputFilter= 0 // list of state names or signal elements to allow, "*" for any, signal elements as in "Signal(1,0)"
Connector:ConnectorInput string ConnectorInputAddress= % localhost:20320 % % // IP address/port to read from, e.g. localhost:20320
Visualize:Application%20Window:ApplicationBase:%:DisplayVisualization int VisualizeApplicationWindow= 0 0 0 1 // Display miniature copy of application window (boolean)
Visualize:Application%20Window:ApplicationBase:%:DisplayVisualization int AppWindowSpatialDecimation= 8 8 1 % // Application window decimation (shrinking) factor
Visualize:Application%20Window:ApplicationBase:%:DisplayVisualization int AppWindowTemporalDecimation= 4 16 1 % // Application window time decimation factor
Application:Window:ApplicationBase int WindowWidth= 640 640 0 % // width of application window
Application:Window:ApplicationBase int WindowHeight= 480 480 0 % // height of application window
Application:Window:ApplicationBase int WindowLeft= 0 0 % % // screen coordinate of application window's left edge
Application:Window:ApplicationBase int WindowTop= 0 0 % % // screen coordinate of application window's top edge
Application:Window:StimulusTask string WindowBackgroundColor= 0x000000 0x505050 % % // background color (color)
Application:Sequencing:StimulusTask float PreRunDuration= 0 0 0 % // pause preceding first sequence
Application:Sequencing:StimulusTask float PostRunDuration= 0 0 0 % // pause following last sequence
Application:Sequencing:StimulusTask float PreSequenceDuration= 2s 2s 0 % // pause preceding sequences/sets of intensifications
Application:Sequencing:StimulusTask float PostSequenceDuration= 2s 2s 0 % // pause following sequences/sets of intensifications
Application:Sequencing:StimulusTask float StimulusDuration= 100ms 40ms 0 % // stimulus duration
Application:Sequencing:StimulusTask float ISIMinDuration= 300ms 80ms 0 % // minimum duration of inter-stimulus interval
Application:Sequencing:StimulusTask float ISIMaxDuration= 300ms 80ms 0 % // maximum duration of inter-stimulus interval
Application:Result%20Processing:StimulusTask int InterpretMode= 0 0 0 2 // interpretation of results: 0 none, 1 online free mode, 2 copy mode (enumeration)
Application:Result%20Processing:StimulusTask int DisplayResults= 1 1 0 1 // display results of copy/free spelling (boolean)
Application:Sequencing:P3SpellerTask int NumberOfSequences= 15 15 1 % // number of sequences in a set of intensifications
Application:Speller%20Targets:P3SpellerTask matrix TargetDefinitions= 1 { Display Enter Display%20Size Icon%20File Sound } A A 1 images\transparency.bmp % // speller target properties
Application:Speller%20Targets:P3SpellerTask intlist NumMatrixColumns= 1 1 6 1 % // display matrices' column number(s)
Application:Speller%20Targets:P3SpellerTask intlist NumMatrixRows= 1 1 6 0 % // display matrices' row number(s)
Application:Audio%20Stimuli:P3SpellerTask int AudioStimuliOn= 1 0 0 1 // Audio Stimuli Mode (0=no, 1=yes) (boolean)
Application:Audio%20Stimuli:P3SpellerTask matrix AudioStimuliRowsFiles= 1 { filename } sounds/La_Fundamental_(440_Hz).wav // audio stimuli rows files
Application:Audio%20Stimuli:P3SpellerTask matrix AudioStimuliColsFiles= 1 { filename } sounds/La_Fundamental_(440_Hz).wav // audio stimuli column files
Application:Speller%20Targets:P3SpellerTask floatlist TargetWidth= 1 90 0 0 100 // target width in percent of screen width
Application:Speller%20Targets:P3SpellerTask floatlist TargetHeight= 1 90 0 0 100 // target height in percent of screen height
Application:Speller%20Targets:P3SpellerTask floatlist TargetTextHeight= 1 10 0 0 100 // height of target labels in percent of screen height
Application:Speller%20Targets:P3SpellerTask stringlist BackgroundColor= 1 0x000000 0x505050 % % // target background color (color)
Application:Speller%20Targets:P3SpellerTask stringlist TextColor= 1 0x000000 0x505050 % % // text color (color)
Application:Speller%20Targets:P3SpellerTask stringlist TextColorIntensified= 1 0x0000FF 0x505050 % % // intensified text color (color)
Application:Speller%20Targets:P3SpellerTask intlist IconHighlightMode= 1 4 1 0 4 // icon highlight method 0: Show/Hide, 1: Intensify, 2: Grayscale, 3: Invert, 4: Dim (enumeration)
Application:Speller%20Targets:P3SpellerTask floatlist IconHighlightFactor= 1 100 0.5 0 % // scale factor for highlighted icon pixel values
Application:Speller:P3SpellerTask int FirstActiveMenu= 1 1 1 % // Index of first active menu
Application:Speller:P3SpellerTask float StatusBarSize= 10 0 0 100 // size of status bar in percent of screen height
Application:Speller:P3SpellerTask float StatusBarTextHeight= 8 0 0 100 // size of status bar text in percent of screen height
Application:Speller:P3SpellerTask string TextToSpell= % // character or string to spell in offline copy mode
Application:Speller:P3SpellerTask string TextResult= % // user spelling result
Application:Speller:P3SpellerTask int TestMode= 1 0 0 1 // select targets by clicking on their associated stimuli (0=no, 1=yes) (boolean)
Application:Speller:P3SpellerTask string DestinationAddress= % // network address for speller output in IP:port format
Application:Text%20Window:P3SpellerTask int TextWindowEnabled= 0 0 0 1 // Show Text Window (0=no, 1=yes) (boolean)
Application:Text%20Window:P3SpellerTask int TextWindowLeft= 640 0 0 % // Text Window X location
Application:Text%20Window:P3SpellerTask int TextWindowTop= 0 0 0 % // Text Window Y location
Application:Text%20Window:P3SpellerTask int TextWindowWidth= 512 512 0 % // Text Window Width
Application:Text%20Window:P3SpellerTask int TextWindowHeight= 512 512 0 % // Text Window Height
Application:Text%20Window:P3SpellerTask string TextWindowFontName= Courier // Text Window Font Name
Application:Text%20Window:P3SpellerTask int TextWindowFontSize= 10 4 1 % // Text Window Font Size
Application:Text%20Window:P3SpellerTask string TextWindowFilePath= % // Path for Saved Text File (directory)
Connector:ConnectorOutput string ConnectorOutputAddress= % localhost:20320 % % // IP address/port to write to, e.g. localhost:20320
Application:Localization:Localization string Language= Default Default % % // Language for user messages
Application:Localization:Localization matrix LocalizedStrings= { German } { TIME%20OUT%20!!! Waiting%20to%20start%20... Sleeping--Select%20SLEEP%20twice%20to%20resume Select%20SLEEP%20once%20more%20to%20resume Paused--Select%20PAUSE%20again%20to%20resume } Zeit%20abgelaufen! Warte%20... Angehalten:%20Zweimal%20SLEEP%20f%fcr%20Weiter Angehalten:%20Noch%20einmal%20SLEEP%20f%fcr%20Weiter Angehalten:%20Noch%20einmal%20PAUSE%20f%fcr%20Weiter // Localized user messages
System:Core%20Connections string ApplicationPort= 3741 4200 1024 65535 // the Application module's listening port
System:Core%20Connections string ApplicationIP= 192.168.0.156 127.0.0.1 % % // the Application module's listening IP
System:Configuration matrix ApplicationVersion= { Framework Revision Build } 1 2.0 2245,%202009/02/03%2015:58:46 2009/02/03%2019:09:02 // Application version information
System:Configuration matrix ApplicationFilterChain= 3 { Filter%20Name Position%20String } ConnectorInput 2.9999 P3SpellerTask 3 ConnectorOutput 3.9999 // Application filter chain

[jethridge]

We have discovered a very similar timing issue to the one described below using our 40 Channel NuAmps amplifier from Neuroscan. We get timing plots similar to the one shown in this post when running P300 sessions (i.e. lots of spikes in the "Block" plot, and also not centered about the time for one sample block). We are acquiring data at 250Hz with a sample block size of 10, as is required by the neuroscan hardware/fragment file. Was the underlying problem ever discovered? Perhaps there is something missing in our system configuration, we have tried this on two computers and gotten the same results.

Thanks,
James

[gschalk]

James,

The previous problem had to do with an issue of this user's implementation of the support for their device. There is no general BCI2000-wide issue.

The Neuroscan support works through connection to the Scan software. These connections are done through TCP/IP, which will increase timing jitters that BCI2000 can't do anything about.

Neuroscan will soon release the so-called AccessSDK which will allow for access to the amplifiers directly, without going through the Scan program. This should improve timing behavior quite a bit. We have an early version of the AccessSDK and will soon start to integrate it into BCI2000.

At the moment, your best option is to use a very fast computer, or put Scan on a different machine as BCI2000.

The one thing that puzzles me that you report that your times are not centered around one sample block, which really should not be the case. Can you post a screen shot, or better, some statistics on the timing? (You can use load_bcidat or BCI Export to export BCI2000 data into Matlab/ASCII, and then look at the SourceTime state.)

Gerv

[jethridge]

Here are two screen-shots from a p300 session we were running.

1) This is what the timing plot typically looked like at the beginning of a trial:



2) Here is another image from the same session a little later in the trial:




It is possible that the "block" graphs are centered roughly around the 40ms mark, however I assumed that is where the tick-mark would be as well.

Haven't had time to generate the timing statistics in MATLAB today, but will look into this as well.

We attempted this on both a 3.0GHz Pentium 4 with 512MB ram as well as on a 2.0GHz Intel Core 2 laptop with 2GB ram. It may not be easy for us to try it on a faster machine at the moment since most of the lab machines are running x64 vista, however if this is the most likely source of the problem arrangements will have to be made.

Also, I should note that this concern was primarily sparked by the fact that we are continually getting unreasonable latencies during our P300 analysis (peak occurring at about 800 ms or so). This has happened across several subjects, which implied to me that there was some sort of timing issue with our system.

Thanks as always for the quick reply,
James

[gschalk]

James,

Both displays are centered around the tick mark, so in principle, there does not seem to be a fundamental problem in your setup. Considering the TCP/IP transmission through SCAN, the first image (showing jitters of <10ms) is probably the best performance you will get using that approach. The question is why you get worse performance later in the session.

A few tips:
Use a Windows XP SP2 system with discrete graphics card (no laptop with onboard graphics). Turn off firewalls, virus scanners, and any other software that may execute in parallel. Use a CRT monitor with >100 Hz refresh rate. Monitor the CPU load. It should be <75% on every processor/core.

Keep me updated.

Gerv

[jethridge]

Dr. Schalk

We have tried our system on a computer with the following specifications:

OS: Windows XP (32 bit, SP3)
3.0 GHz Intel Xenon Dual Core w/ Multi Threading
800MHz FSB
2GB DDR2 RAM
ATI FireGL V3100 Graphics, 128MB Dedicated Video Memory
Monitor: CRT, 100Hz

Unfortunately, we seem to be getting the same results as on the slower machines. Posted below are screenshots of the timing performance during a P300 calibration session. Also shown are the results of the offline analysis we performed after the calibration session. Though there is some variation in when/where the peak is occurring, these results are fairly typical of what we have been getting.

I'd really like to be sure that our setup is adequate and configured properly before diving into any real experiments, so of course your insights are much appreciated. Perhaps we could send you the param files we are using as well to be sure we haven't overlooked something in our configuration?

timing 1


timing 2


somewhat typical offline analysis results



Pleas let me know if there is any other info I can provide that would help.
-James

[gschalk]

James,

The machine you are using should be easily fast enough to accommodate most BCI2000 configurations, and it is definitely easily fast enough to run the P300 Speller at 256 Hz sampling. You can verify by using the SignalGenerator instead of the Neuroscan input. The timing should be much much better.

Thus, the problem is not with BCI2000 timing/performance, but with the timing of the Neuroscan input. As I think I mentioned earlier, if you are using Scan, data acquisition goes through a different program that sends data to BCI2000 via TCP/IP. While this approach will never give you the timing performance of a configuration that would interact with the amplifier directly, you should be able to optimize it. In particular, are you running Scan on the same or on a different machine? If it's on a different machine, are both machines on a LAN or directly connected? Your machine should be easily capable of running Scan in addition to BCI2000 (i.e., on the same machine). I would try this and report back.

Gerv

[jethridge]

We are running Scan 4.4 and BCI2000 both on the same machine. We are using the NuAmps 40ch usb amplifier for acquisition (though not all 40 ch are used in the protocol). As you can understand, I'm pretty confused as to why we are getting the results we are. What I can say at this point is that, whatever the problem, it is consistent across multiple machines.

I contacted a technician at CompuMedics/NeuroScan about a week ago and asked what settings I should be using to optimize the speed of the Scan software. He walked me through a few things which he thought might help, but it didn't seem to make much difference.

The timing is almost perfect with the signal generator.

Also, despite the timing results we are seeing with Scan, we seem to get decent control in the 2-D feedback task (however this may be due to the adaptation in the normalizer block). The P300 is consistently off though, almost like there is a ~400ms group delay on the peak for all of our subjects. I've checked the params my associate is using against the user reference (Pre/Post sequence duration etc.) and haven't seen anything that violates any of the constraints in the wiki. At the moment, I'm not really sure how to proceed to isolate the problem. I can post/e-mail the params file if that will help.

[gschalk]

James,

Timing involves at least two parameters: latency and jitter. From the one subject that you have shown, I would roughly estimate a latency of about 200ms. From your timing pictures, it seems like the jitter is sometimes +-25ms. This seems relatively high and may be more problematic for the P300 experiments than for the cursor task.

You can measure the latency of your system and subtract it later in your analysis, so this is not a big problem. The jitter is more worrisome. Since this is really dependent on what is going on in Scan, the best advice I can give you is to try to use a TCP/IP optimizer so that TCP/IP packets are delivered immediately. You may also try to put Scan on a different computer and connect it via Ethernet directly to the BCI2000 computer. Aside from these workarounds to the general problem, you will have to wait until the BCI2000 support for the Neuroscan AccessSDK is complete. I cannot give you an ETA on that one since it depends on the stability of the Neuroscan SDK.

On the flip side, many groups have done many successful experiments using Neuroscan. Are you getting people to spell? What is the speed/accuracy?

Gerv