View source for User Reference:Validating the Event Logging Mechanism

==Synopsis==
BCI2000 is able to record asynchronous data, so-called [[Programming_Reference:Events|Events]], that occur during a recording. These events are logged into [[Technical Reference:State Definition|BCI2000 states]], which provide a universal way to store information associated with brain signal sample data.

This page provides an overview over the general concept and implementation of events.

Also, it describes the ''RefLogger'', a software component which allows to quantify how reliable event recording is, and shows a simple Matlab script to analyze ''RefLogger'' data, together with results from a reference systems.

==The Event Logging Mechanism==
===Time-stamping Brain Signal Data===
As described elsewhere, BCI2000 processes brain signal data in blocks of fixed size and duration.
Whenever a block of data has been acquired from the main brain signal source, a time stamp is taken from the computer's clock.
Note that this time stamp is taken after the data block has been sampled and transmitted from the hardware.
Assuming a negligible time delay between sampling of the block's last sample, and transmission into the computer, we may say that this time stamp represents the time of sampling for the first sample of the following block.
Knowing this time stamp, and the sampling rate of the brain signal source, we can thus associate each of the block's samples with a sample time.

===Time-stamping Events===
In the BCI2000 software, a function exists that takes the name of an event, and a value. Whenever this function is called, it takes a time stamp from the computer’s clock and stores the event’s name, value, and time stamp into a queue of events. This queue is organized according to time stamps such that oldest entries will be retrieved first, even if they may not have arrived in temporal order (priority queue).

===Associating Time-stamped Events with Samples===
For the following, it is crucial to understand that events happen while a data block is being digitized by the amplifier's hardware, and may then be associated with the samples of that block once it arrives at the computer.
As soon as that block enters the computer’s memory, events are taken from the event queue in temporal order, and applied to the state variables associated with the block.

===Sources of Imprecision===
The event logging mechanism depends on precision of the BCI2000 time stamp. As this is derived from a high-precision clock available in all modern computers, this is in general not an issue.

More of interest is another source of imprecision: Acquisition irregularities. Ideally, BCI2000 block duration is constant, and data blocks arrive regularly with little to no acquisition delay -- acquisition delay is the time delay between digitizing a block’s last sample, and seeing that block in the computer’s memory. ''Regularity'' of data blocks may be assessed using the BCI2000 [[User Reference:Timing|timing window]]; there, data block duration should form a straight line. Acquisition ''delay'', in contrast, is more difficult to measure, and requires an amplifier with a digital (or analog) output that may be connected to an amplifier input, producing an input signal whenever a new block has entered the computer's memory.

Imprecise block duration will result in differences between measured block durations, and ideal block durations. If this happens, time stamps may appear older than the oldest sample in the current block. If this is the case, the affected events will be associated with the block’s first sample, as the previous block is no longer available for storing events. This will result in events that are shifted slightly into the future, so they may appear to precede the actual time of the physical event that produced them.

==The RefLogger Component==
The ''RefLogger'' is a simple BCI2000 component that logs a special event, called '''RefTime''', in regular intervals.
What is special about that event is that it does not hold simple values like "1" or "2" but time stamps derived from the computer’s clock, just as the time stamps used to align events with brain signal samples. In data analysis, this allows to compare the sample position of an event to its time stamp value, and to assess how well these two match.

===Using the RefLogger===
The ''RefLogger'' is available in all BCI2000 modules, but not active by default.
To enable it, add a command line option to the source module’s <tt>START EXECUTABLE</tt> command as in the following example:
 Start Executable SignalGenerator --RefLoggerFrequency=100Hz --local
You may specify any value for ''RefLoggerFrequency'' but notice that BCI2000 time stamps are limited to millisecond resolution, so producing ''RefLogger'' events at a frequency higher than 1000Hz is not useful. Also, if the ''SamplingRate'' parameter is set to a value lower than ''RefLoggerFrequency'', no useful results may be expected.

===Analyzing RefLogger Data===
''RefLogger'' data analysis follows two distinct purposes:

# Assessing whether the Event logging software mechanism works as designed,
# Assessing how precise and useful Event information is for a certain experiment.

In terms of data, the two questions are quite similar, but differ in the time base used to compare '''RefTime''' timestamps against.

For question (1), the time base is taken from each data block’s ''SourceTime'' timestamp, and extrapolated into the past before determining the difference to the '''RefTime''' entries.

For question (2), a sample-based time base is constructed by taking the recording’s duration as measured in terms of time stamps, and evenly distributing that across all samples in the recording, using the first time stamp as an offset. Then, the time stamp differences give an impression of how the event mechanism performs with regard to physical time.

Ideally, the results of (1) and (2) would agree. Any differences are due to the sources of imprecision discussed above.

===Example Results===
====Data recorded with SignalGenerator source module====
[[file:Event_vs_SourceTime_SignalGenerator.png]]

As one may see from this graph, event time stamp disagreement from sample position is within +/-0.5 ms, so the Event time mechanism is working close to perfectly.

[[file:Event_vs_sample_time_SignalGenerator.png]]

This graph shows that there is quite some disagreement between sample time, and event time.
This is due to the fact that SignalGenerator is not a true data source, and uses the imprecise Windows <tt>Sleep()</tt> function to simulate regular arrival of sample blocks.

====Data recorded with g.USBamp source module====
[[file:Event_vs_SourceTime_gUSBamp.png]]

Similarly to the SignalGenerator example above, the Event mechanism is working close to perfectly with the g.UBSamp amplifier, with a jitter of +/-0.4 ms (standard deviation).

[[file:Event_vs_sample_time_gUSBamp.png]]

When comparing event time stamps to sample time for data recorded with the g.USBamp amplifer, we have a nearly perfect distribution of differences, with an event timing jitter of +/-0.4 ms (standard deviation).

Note that time stamps are rounded to full milliseconds, so an event timing jitter in the order of half a millisecond is to be expected.

===Matlab Analysis Script===
The following script has been used to produce the above graphs.
Modify the filename variable to hold the name of the BCI2000 data file to analyze.

 filename = 'RefLogger_gUSBampS001R01.dat';
 
 [signal, states, parameters] = load_bcidat(filename);
 SampleBlockSize = parameters.SampleBlockSize.NumericValue;
 SamplingRateHz = parse_sampling_rate(parameters.SamplingRate.Value{1});
 SampleBlockDurationMs = SampleBlockSize / SamplingRateHz * 1e3;
 
 % prepare SourceTime state by first reducing to a single value per block,
 % then interpolating across block
 SourceTime = double(states.SourceTime);
 SourceTime2 = SourceTime(1:SampleBlockSize:end);
 SourceTime2 = extend_timestamp(SourceTime2);
 SourceTime3 = interp1([1:length(SourceTime2)], SourceTime2, [1:1/SampleBlockSize:length(SourceTime2)])';
 % SourceTime reflects the time of acquisition, so sample time is earlier
 % by a sample block duration
 SourceTime3 = SourceTime3 - SampleBlockDurationMs;
 
 % prepare a vector of equally spaced sample times (SourceTime may be
 % jittered)
 sample_time = linspace(SourceTime3(1), SourceTime3(end), length(SourceTime3))';
 
 % prepare RefTime state
 RefTime = double(states.RefTime);
 RefTime = RefTime(1:length(SourceTime3));
 % replace "unused" placeholder value with nan
 RefTime(RefTime > 65535) = nan;
 valid_idx = ~isnan(RefTime);
 RefTime2 = RefTime(valid_idx);
 RefTime2 = extend_timestamp(RefTime2);
 
 % determine difference between RefTime values, and interpolated SourceTime
 % values
 delta1 = RefTime2 - SourceTime3(valid_idx);
 
 figure;
 hist(delta1, 40);
 title('RefTime vs SourceTime disagreement');
 subtitle(sprintf('mean: %f ms, sdev: %f ms', mean(delta1), sqrt(var(delta1))));
 xlabel('offset in ms');
 
 % determine difference between RefTime values, and interpolated sample time
 % values
 delta2 = RefTime2 - sample_time(valid_idx);
 
 figure;
 hist(delta2, 40);
 title('RefTime vs sample time disagreement');
 subtitle(sprintf('mean: %f ms, sdev: %f ms', mean(delta2), sqrt(var(delta2))));
 xlabel('offset in ms');
 
 % determine difference between interpolated SourceTime, and interpolated sample time
 % values
 delta3 = SourceTime3(valid_idx) - sample_time(valid_idx);
 
 figure;
 hist(delta3, 40);
 title('SourceTime vs sample time disagreement');
 subtitle(sprintf('mean: %f ms, sdev: %f ms', mean(delta3), sqrt(var(delta3))));
 xlabel('offset in ms');
 
 
 function result = extend_timestamp(input)
 % extend timestamp from the 0..65535 to full range
    result = input;
    add = 0;
    for i = 2:length(input)
        if input(i) < input(i-1)
            add = add + 65536;
        end
        result(i) = input(i) + add;
    end
 end
 
 function result = parse_sampling_rate(input)
    [token, remain] = strtok(input, 'Hk');
    result = str2double(token);
    if strcmp(remain, 'kHz')
        result = result * 1e3;
    end
 end

==See also==
[[Programming_Reference:Events]], [[Technical Reference:State Definition]], [[Programming Tutorial:Implementing an Input Logger]]

[[Category:Data Acquisition]] [[Category:Framework API]]