Understanding acquisition timing

[rflint3]

I'm trying to figure out, as precisely as I can, when samples were acquired during an experiment. I've read up on the state variables SourceTime and StimulusTime, but when I look into these states in my recorded data I still have some questions.

In the past, I have simply constructed a timing vector spaced according to the ratio of SampleBlockSize to SamplingRate, i.e., assuming uniform spacing of samples. Indeed, that approach is recommended here (viewtopic.php?t=63&sid=4e0a1885e6fb172e2d0953f3e7c2e8db), as well as a related approach using regression (viewtopic.php?t=1278&sid=855f1dc1c85644 ... e5ee4c5eeb). What I am hoping to do right now is construct a more precise time-base, even if it is not uniformly spaced, but which accurately reflects when samples were acquired.

In an example file, I have SamplingRate of 2000 Hz, SampleBlockSize of 40, therefore the block duration is expected to be 20 ms. The data file is around 250s long, or about 12500 data blocks. The data blocks themselves almost universally consist of exactly 40 samples each (judging by the number of repetitions of the state variables like SourceTime in each block). Speaking of SourceTime: when I unwrap it, I see that most blocks of data have the expected duration. The second-most frequent phenomenon is that a block is 1 or 2 ms too long (or too short), but then the immediate subsequent data block "makes up" for this by compensating in the other direction. A third outcome is that the data block duration (according to SourceTime) is significantly longer than the expected 20ms, but only has the expected number of samples, and this is not compensated afterwards. In that case, I'm assuming data is lost. I'm willing to accept all of those scenarios without too much alarm.

The fourth scenario I'm seeing is that SourceTime reports a significantly *shorter* duration than expected, by as much as 10ms or more, but the *reported number of samples is still 40*. If that is what's actually happening, then the samplingRate is being violated somehow? This possibility was brought up in a prior thread (viewtopic.php?t=1349&sid=47daf3fe8cf1ea ... ed4dfdeded), but not explicitly addressed. It does say in that thread that "BCI2000 will never duplicate samples", which eliminates my first guess as to what's going on with the too-short data block times.

Can the samplingRate effectively (or literally) vary from one block to another? Or, am I expecting too much accuracy from the SoureTime state? Or is there some other explanation? I'd appreciate any insight you can share. My example data was recorded using the Blackrock signalSource module.

Thanks!

[mellinger]

The issue about signal block duration is complex because it is influenced by multiple effects that are overlaid to each other. I will try my best to explain how things play together.

1. In the acquisition device, samples are produced at a fixed rate. Even if an aquisition device were capable of switching sampling rates during a recording, BCI2000 will not configure it to do so. So you can be assured that each sample received from the acquisition device has the same duration.

2. Typically, the acquisition device will send data to BCI2000 in fixed-size blocks, and BCI2000 will typically configure the device such that the size of those blocks will match the BCI2000 SampleBlockSize. If this is not possible, various approaches may be taken depending on the source module in question, e.g. an error message may be produced, or the device’s block size is simply accepted, which will result in uneven timing.

3. BCI2000 can only set the SourceTime state once it sees the data, i.e. after IO of a block of data has completed from the device. In a preemptive multi-tasking system like Windows, this is subject to a number of influences such as the number of threads in the system, thread priorities, and scheduler time resolution. Thus, the time stamp will always include some noise, as you observed. As you observed as well, this noise rather quickly evens out as more blocks are acquired.

4. In network-based connections between acquisition device, and BCI2000, data loss may occur due to network issues (e.g., network too slow, network faulty on the physical layer…), coding issues in the transmitter or receiver (BCI2000 source module). If data loss occurs, BCI2000 block durations may become smaller (e.g., in case of a short read), or longer (e.g., in case of a timeout). Also, BCI2000 enforces that all data blocks have the same number of samples, so samples may be left over from the previous block, or set to NaN (which is the recommended way to deal with data loss).

5. Some acquisition devices have sample counters to allow detection of data loss. Whereever possible, BCI2000 uses those counters and inserts a block of NaNs when data loss is detected.

For data analysis, I suggest to determine total data duration once by using the SourceTime state of the first and last block, and then by counting the number of samples except the last block. This will give you two measures of data duration which should agree up to one or two milliseconds. If they don't, some form of data loss or data duplication will have occurred.

[rflint3]

Jurgen, thanks for the explanation; that is all valuable info I will save to reference later. I did the calculation you recommended for my test file; I came up with a difference of 53ms. That is to say, the file has 53ms worth of data points more than we'd expect to see, based on the SourceTime state. That is a larger difference than I'd like. I took it one step further, and calculated the analogous difference *for each block of data*:

(actual #samples) - (expected #samples)

I'm calling this quantity the "Acquisition Loss". The (expected #samples) is SourceTime*(SamplingRate/1000) for that block, since SourceTime is a millisecond timer. Attached is a stem plot of the Acquisition Loss for my data file. The units of the y axis are #samples, as noted. The final difference at the end of the file is 106 samples (53 ms at 2kHz sampling), which agrees with the one-time calculation you recommended. I created this plot in the hopes of pinpointing the times when issues arise with the acquisition. From the plot, you can see my description from my earlier post: most of the plot is flat, with small jumps in both the positive and negative directions. There is little net change (we are off by 6 samples at 100s into the recording), then a sizeable jump of right around 1 sampleBlockSize, or 40 samples, occurring a little after 100s in. Something similar happens at around 168s, and a final time at about 234s.

When I first saw this, I thought perhaps the jumps in the plot could be used to identify the exact moment(s) when something problematic happened. Does this seem reasonable? I can think of one alternative explanation: that BCI2000 is getting "behind" the hardware buffer gradually, by a few samples here or there, and only when these "left behind" samples accumulate to something on the order of SampleBlockSize do they get pushed to the software during an "extra" read. Of course, if that was the explanation, then it wouldn't make very much sense that the Acquisition Loss value oscillates up and down by a few samples during nominal (e.g., flat) times in the plot.

I'd be interested to hear your thoughts.

acquistionLoss.png (22.01 KiB) Viewed 30560 times

[mellinger]

(actual #samples) - (expected #samples)

I'm calling this quantity the "Acquisition Loss".

Wouldn't "excess acquisition" be better suited for this, or "acquisition loss" with sign reverted?

When I first saw this, I thought perhaps the jumps in the plot could be used to identify the exact moment(s) when something problematic happened. Does this seem reasonable?

Yes, I agree with that. I don't think samples were hiding somewhere within BCI2000

Upon your post, I took another look at the BlackrockADC source code. There are two things odd about it:
1) The aqcuisition thread is waiting in a polling loop until enough data is available, rather than waiting for a synchronizer,
2) within the polling loop, the data queue is accessed without holding a lock, and thus creating a race condition when reading/writing queue size.

1) will result in more timing noise than necessary, and 2) may result in more objects being popped off the queue than it contains (it's undefined behavior).

I'll commit a fix for 2) but cannot do any real development on the BlackrockADC because I don't have access to a Blackrock device.

[rflint3]

Thank you for your work on the Source module. Very much appreciated!

You're probably right about the name of my made-up quantity, should have called it acquisition gain or some such. More importantly, you have it right about the interpretation: positive deflections indicate that the software is somehow getting "extra" data, possible examples of the behavior you put in your point (2). Negative jumps, of course, would indicate unexpected delay.

In thinking about how to deal with this, and hopefully rehabilitate this data for further analysis: the first thing that comes to mind is to try simply eliminating the "extra" blocks of data. This has a soothing effect on the acquisition mismatch quantity (attached), ending up with a less-alarming 14 sample mismatch at the end of the data (7 ms). However, I'm not confident this is valid, and applying it to the actual signal array will be tricky due to the possibility of creating transients.

acquisitionMismatch2.png (20.62 KiB) Viewed 30553 times

[mellinger]

However, I'm not confident this is valid, and applying it to the actual signal array will be tricky due to the possibility of creating transients.

I rather think there should be transients in the original signal, and properly restoring it will remove those transients. Often, transients show up prominently in the signal's time derivative (i.e., diff) because the signal is typically low-pass filtered before being digitized.