Jump to content

User Reference:BCI2000Certification: Difference between revisions

From BCI2000 Wiki
← Older edit
Awilson (talk | contribs)
125 edits
No edit summary
 
(127 intermediate revisions by 8 users not shown)
Line 1: Line 1:
==Introduction==
==Introduction==
 
BCI2000 includes a certification procedure that can determine whether a computer system is capable of running all or some BCI2000 applications.  Because BCI2000 does not have a standardized hardware configuration, i.e., it can run on potentially ''any'' PC, this testing procedure is capable of running on any PC with BCI2000, with the results reporting which standard BCI2000 applications meet the minimum timing requirements. Different configurations are included in the certification procedure to test a range of likely setups for a given application, in which timing characteristics such as the sampling rate, inter-stimulus intervals, and stimulus duration, and other parameters, such as the number and size of the stimuli, are changed for each configuration.
BCI2000 v2.0 and higher includes a certification procedure that can determine whether a computer system is capable of running all or some BCI2000 applications.  Because BCI2000 does not have a standardized hardware configuration, i.e., it can run on potentially ''any'' PC, this testing procedure is capable of running on any PC with BCI2000, with the results reporting which standard BCI2000 applications meet the minimum timing requirements. Different configurations are included in the certification procedure to test a range of likely setups for a given application, in which timing characteristics such as the sampling rate, inter-stimulus intervals, and stimulus duration, and other parameters, such as the number and size of the stimuli, are changed for each configuration.


This document serves as the instruction manual for using the certification procedure, and describes how the procedure works to determine the timing characteristics of the BCI2000 system. The primary use for the certification procedure is to determine if the PC configuration used with BCI2000 is capable of running the core BCI2000 system, which is comprised of:
This document serves as the instruction manual for using the certification procedure, and describes how the procedure works to determine the timing characteristics of the BCI2000 system. The primary use for the certification procedure is to determine if the PC configuration used with BCI2000 is capable of running the core BCI2000 system, which is comprised of:
    
    
*Support for the the \gtec{} series of amplifiers, including the g.USBamp, g.MOBIlab, and the g.MOBIlab+ (bluetooth) amplifiers.
*Support for various biosignal amplifiers, including the g.USBamp, EGI GTEN 200, Nihon Kohden JE120-A, and many more on the [[Contributions:ADCs|ADCs]] page.  
*The AR, P3, FFT, and Matlab signal processing modules.   
*The AR, P3, FFT, and Matlab signal processing modules.   
*The CursorTask, P3Speller, and StimulusPresentation application modules.   
*The CursorTask, P3Speller, and StimulusPresentation application modules.   


Additionally, due to the extensibility of BCI2000, researchers who develop custom algorithms and applications, or use amplifiers not made by \gtec{}, can use the certification procedure to determine whether their custom modules are capable of running on a particular PC configuration.
Additionally, due to the extensibility of BCI2000, researchers who develop custom algorithms and applications can use the certification procedure to determine whether their custom modules are capable of running on a particular PC configuration.


The certification procedure works by comparing the differences in time of when BCI2000 ''expects'' an event to occur to when that event ''actually'' occurred. These events include a change in the display, an audio output through the system speakers, or an EEG sample being stored in the amplifier buffer. The times at which such events occur are recorded with the amplifier and stored in a data file. Changes in the display are found using an optical sensor in combination with the \gtrig{}. The \gtrig{} is a signal conditioner that generates trigger pulses for various sensors or input signals. When the sensor exceeds a threshold, the \gtrig{} outputs a digital signal which is recorded on the amplifier. A similar procedure is used for audio detection: the sound output from the PC is input to the \gtrig{}, and when the volume exceeds threshold, a high digital signal is output and recorded by the amplifier. This information can be used because BCI2000 stores a time-stamp for every event during an experiment. The difference between a time-stamp value and the time that the recorded event changes values are compared to find the latency for that particular event.  
The certification procedure works by comparing the differences in time of when BCI2000 ''expects'' an event to occur to when that event ''actually'' occurred. These events include a change in the display, an audio output through the system speakers, or an EEG sample being stored in the amplifier buffer. The times at which such events occur are recorded with the amplifier and stored in a data file. Changes in the display are found using an optical sensor and a triggering device. The documentation provided throughout this page uses the g.tec g.TRIGbox, although any device which can provide a digital or analog trigger to the amplifier can be used. The g.TRIGbox is a signal conditioner that generates trigger pulses for various sensors or input signals. When the sensor exceeds a threshold, the g.TRIGbox outputs a digital or analog signal which is recorded on the amplifier. A similar procedure is used for audio detection: the sound output from the PC is input to the g.TRIGbox, and when the volume exceeds threshold, a digital or analog signal is output and recorded by the amplifier. This information can be used because BCI2000 stores a time-stamp for every event during an experiment. The difference between a time-stamp value and the time that the recorded event changes values are compared to find the latency for that particular event.  


These latencies can be affected by many external influences, including hardware deficiencies (such as low RAM, slow CPU speed, or a video card of low quality), and other software running in the background while the experiment is being run.
These latencies can be affected by many external influences, including hardware deficiencies (such as low RAM, slow CPU speed, or a video card of low quality), and other software running in the background while the experiment is being run.


This document serves as the BCI2000Certification manual. As an example, this page will describe the certification procedure for the g.USBamp with visual and auditory stimuli triggered with the g.TRIGbox. At the end of the page, the certification results for using BCI2000 with three biosignal amplifiers (the g.USBamp, EGI GTEN 200, and Nihon Kohden JE120-A) are provided. A discussion and derivation of the specific system latencies are found [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3161621/ here].
==Video Demonstration==
<youtube alignment="center">https://youtu.be/yh7ifhCh2v0</youtube>
==Compiling the BCI2000Certification Programs==
The BCI2000Certification procedure actually consists of two separate programs, including a graphical user interface that simplifies the task of selecting and starting tests, and a program that performs the data analysis and returns results. These programs are compiled automatically with the other BCI2000 programs. Compilation is only supported in BCI2000 v3.0 with the Visual Studio or MinGW compilers. Instructions can be found at [[Programming_Howto:Building_BCI2000]]. At the point where you start CMake, be sure to check BUILD TOOLS in the BUILD menu.


==Measuring Latencies==
[[image:CMakeBuildTools.png|frame|none|Check the BUILD TOOLS box to build the certification tester and analysis tool.]]
Briefly, the different latencies inherent in the BCI2000 system include amplifier latency, processing latency, output latency, and the sum of these, the system latency. Additionally, the block duration jitter will be measured.  Figure \ref{fig:timeline} provides a timeline of events that occur within a single block of data, with significant time points labeled. This figure provides an estimate of the order that events occur within a block of data, although the time intervals are not necessarily to scale. Furthermore, for the purposes of calculating different event latencies, <math>t_{3} = t_{0}</math>, and <math>t_{5} = t_{4}</math>, because time-stamps are only stored at <math>t_{0}</math> and <math>t_{5}</math>. This does not affect the calculations because the latencies between these times are negligible.


<span id="TimeLine"></span>
==Using the BCI2000Certification Procedure==
===Starting the Tester User Interface===
# The compiled programs can be found in <code>BCI2000/tools/BCI2000Certification</code>.
# Navigate to this folder, and double-click the <code>BCI2000CertificationTester.exe</code> file.
# The certification tool opens, as below.


[[Image:TimeLine.png|frame|none|Timeline of events. Time intervals are approximate, and not to scale.]]
This program shows a list of all of the procedures to be tested for the loaded configuration. Detailed instructions for using the interface follow.


[[image:Certificationtool.png|frame|none|The BCI2000Certification user interface.]]
[[image:Tester02.PNG|frame|none|The BCI2000Certification user interface.]]


*'''Amplifier Latency''' The delay between the time that a signal is presented at the amplifier input, to when it is digitized and stored in an amplifier buffer. Amplifier latency = <math>t_{0}-t_{-1}</math>
===Loading Configuration Files===
*'''Processing Latency''' This is the total time it takes for the data to be processed by all BCI2000 modules (SignalSource, SignalProcessing, and Application). Processing latency = <math>t_{5}-t_{0}</math> in [[#TimeLine|Timeline]].
The BCI2000 certification procedure uses a configuration file and different combinations of BCI2000 parameter fragments to control the tests. The configuration file (stored as an <code>*.ini</code> file) describes all of the certification tests. The parameter fragments store information about the amplifier and application module applied in each test. More detail about the contents of the configuration file can be found below. While it is possible to edit these by hand, in most circumstances it is simpler to use the graphical user interface provided.  
*'''Output Latency''' The latency from the time that the Application issues a command to change the display or issue a sound (or another output), to the time that this output actually occurs. Output latency = <math>t_{6}-t_{5} = \mathrm{System Latency} - \mathrm{Processing Latency} - \mathrm{Amplifier Latency}</math>.
*'''System Latency'''  The minimum time interval between a change in amplifier input, and a causally related change in application output. System latency = <math>t_{6} - t_{0} + \mathrm{Amplifier Latency}</math> in [[#TimeLine|Timeline]].
*'''Jitter''' This is variance in the time interval between blocks of data, measured as the absolute value of the difference in timestamps from the beginning of data acquisition for adjacent blocks of data.  Jitter = <math>\frac{t_{8}-t_{0}}{\mathrm{Sample Block Length}}</math> in [[#TimeLine|Timeline]].


The procedures that will be used to perform these tasks follows. Figure \ref{fig:overview} gives an overview of the system latencies with the testing equipment shown.
# In the BCI2000Certification tool, select <code>File->Open *.ini</code>
# Navigate to the folder <code>BCI2000/tools/BCI2000Certification</code>.
# Load the file named <code>BCI2000Certification_16ch_100ms.ini</code> if using a 16-channel system. (This tutorial assumes a 16-channel system).
# If using more than 16 channels, then the amplifier parameter files must be modified.
## Navigate to <code>BCI2000/tools/BCI2000Certification/parms/</code>, and open the file <code>Certification_gUSBamp_24.prm</code> in Notepad.
## Find the parameter titled '''DeviceIDMaster''', and replace the existing device ID (e.g., <code>UA-2007.04.12</code> with the ID of your master device.
## Next, find the parameter titled '''DeviceIDs''', and replace the IDs with the IDs of your amplifiers. The first in the list should be identical to the '''DeviceIDMaster''' value. For example, you might enter: <code>DeviceIDs= 2 UA-2007.04.12 UA-2006.10.13</code> (the \#2 corresponds to the number of devices being used).
## Repeat these steps for the file <code>Certification_gUSBamp_32.prm</code>.
## Note that these steps are not necessary if only one g.USBamp is being used.
# All 100 tests in this configuration are loaded into the GUI.


<span id="overview"></span>
===User Interface Components===
[[Image:TimingOverview.png|frame|none|Timing overview. Times shown correspond to those in [[:Image:TimeLine.png]]]]
This section describes each user interface component, referring to the figure below.


===Amplifier Latency===
[[image:Certificationtool2.png|frame|none|The BCI2000Certification user interface components.]]
[[#overview|Overview]] shows a latency between the time that the signal is presented to the acquisition hardware, to when the signal is digitized and stored in the hardware buffer:


<center><math>
====1. Configuration: Task List====
\mathrm{AmplifierLatency} = t_{0} - t_{-1}
The task list shows all tasks that have been loaded from the configuration file or created within the GUI. To run a specific test, it should have a checkmark next to it. In order to select or deselect all tasks, press the <code>Select All</code> check box at the bottom of the list.
</math></center>


To measure this latency, the SignalSource module will set a digital output on the amplifier to a high value immediately following data acquisition, as shown in [[#overview|Overview]]. The digital output level is 3.3 V, so a voltage divider may required to attenuate the signal level to the range of amplifier input. This signal is then recorded on an input channel on the amplifier (time <math>t_{1}</math>). However, this digital signal is changed when the time-stamp is stored for data acquisition. Therefore, <math>t_{1} = t_{0}</math> for the latency calculation, and as mentioned previously, <math>t_{2} = t_{3}</math>, so:
It is possible to create new tasks, remove tasks, and copy tasks within the interface as well. To create a new task, press the <code>+</code> button. A new task is created at the bottom of the list; the task details must be filled in (described next). It is often easier to copy a similar task than to create a new one from scratch. To do so, select the desired task, and press <code>Copy</code>. The new task should be renamed appropriately. To remove a task, select a task and press the <code>-</code> button.


<center><math>
====Creating Custom Tests====
\mathrm{AmplifierLatency} = t_{3} - t_{0}
With the BCI2000Certification tool, you can test performance on a wide range of different parameters and applications. You can create a custom .ini file that will contain the name of each test, the desired signal processing module, sampling rate, block size, application, and parameter files. You will need to first create a parameter file for the test you wish to run. If you want to be able to edit the block size and sampling rate from the Certification tester GUI or its .ini file, you must omit those variables from your parameter files as they will overwrite any variables you set in the Certification Tester. For example, if you wanted to test an application that you have developed for BCI2000 using several sampling rates and block sizes you would first need to create the two parameter files for the application, and source with no sampling rate or block size specified, then export them to the <code>/BCI2000Certification/parms/</code> folder. Once you have the two parameter files, you may link to them in a new .ini file located under the <code>/BCI2000Certification/</code> directory and run them as described below.
</math></center>


Because the digital signal is  changed immediately following a data read, the digital signal should ideally be the first sample recorded in the amp, but will likely be delayed by one or more samples due to the buffering latency. The next block of data is then read into BCI2000 on the following processing cycle (<math>t_{8}</math>), containing the triggered event data. The amplifier latency can be measured as the time-difference between a digital output changing at the start of a new block of data, and the analog channel for that event changing. ([[#SignalSourceLatency|Signal Source Latency]])
====2. Configuration: Task Details====
The right side of the GUI displays the details of the currently selected task. Each option is described here.


It is important to note that there are additional latencies that are present during acquisition. The first additional latency is the transmission latency, which is dependent on the hardware interface. With the g.USBamp, USB 2.0 is used, which has a maximum transmission rate of 480 Mbit/s. If we are acquiring 16 channels of 32-bit data, and transmitting in blocks of 16 samples, this is 8192 bits of data; at USB 2.0 speed, this should take roughly 17 <math>\mu</math>s to transfer. However, a more realistic approximation for USB 2.0 speeds is 240 Mbit/s, which increases the transfer delay to 34 <math>\mu</math>s, which is only about 2.
;Task Name
:The name of the task. This must not contain any spaces, and must be unique.


A second additional latency is dependent on the sample block buffer size. When a data sample is digitized, it is not immediately transferred to the PC, but is stored in memory on the amplifier. The PC then makes the appropriate request to the amplifier, and a block of data is transferred all at once to reduce the processing load. One potential drawback to this is that a signal that is digitized at the start of the buffer must wait until the buffer is full to be read into the PC. For example, if the sample block size is 16 samples, the first sample must wait 15 more samples before it can be used in the application. Fortunately, this latency is always known, and can be reduced with a higher sampling rate and/or smaller block sizes, assuming the PC can handle it. Additionally, this latency does not affect the acquisition-to-display latency, because processing occurs immediately after a block's last sample, which is not affected by buffering latency. It is an important factor to keep in mind, because a large sample block size and/or a small sample rate will decrease the refresh rate for the application (e.g., at a sample rate of 256 Hz and a sample block size of 128 samples, the application will only be refreshed every 0.5 s.)
;Signal Source
:The amplifier that should be used, if different from the amplifier set in the global options. Generally, this should not need to be used, since all tasks should use the same amplifier.


<span id="SignalSourceLatency"></span>
;Signal Processing
[[Image:SignalSourceLatency.png|frame|none|Latency of the Amplifier, using a digital trigger software output and encoded analog input. The time between <math>t_0</math> and <math>t_{2}</math> is estimated; <math>t_{1} = t_{0}</math>]]
:The signal processing module that should be started and used for the given task. This should usually be either <code>ARSignalProcessing</code>, <code>P3SignalProcessing</code>, or <code>DummySignalProcessing</code>, depending on the specific application.


;Application
:The application that is run for the task. This will typically be <code>CursorTask</code>, <code>P3Speller</code>, or <code>StimulusPresentation</code>


===Processing Latency===
;Parameters
The processing latency is defined as the amount of time that it takes for the computer to process the data from the time that it is read into the PC to when the application output is changed. In [[#overview|Overview]], this is the time from <math>t_{0}</math> to <math>t_{5}</math>. The processing latency is calculated using two timestamps saved with every block of data, <math>SourceTime</math> and <math>StimulusTime</math>. The <math>SourceTime</math> is saved immediately following data acquisition in the SignalSource module, and the <math>StimulusTime</math> is saved immediately following application presentation in an application module. Therefore, the processing latency for each block of data is calculated as:
:A list of the parameter fragments that should be used for this task. Typically, at least two parameter files are required. The first should be the amplifier-specific fragment that configures the amplifier for the correct number of channels. The second should be the application-specific fragment that configures how the application appears on the screen. Parameter files can be added and removed with the <code>+</code> and <code>-</code> buttons, respectively. All files should be placed in the <code>BCI2000/tools/BCI2000Certification/parms</code> folder.


<center><math>
;Sample Rate
\mathrm{Processing Latency} = \mathrm{StimulusTime} - \mathrm{SourceTime} = t_{5} - t_{0}
:The sampling rate that should be used for this task. Note that it must be supported by the amplifier used.
</math></center>


Processing latency stongly depends on CPU speed, and if this latency becomes too large, high CPU load will probably be the cause, implying decreased or unsuitable BCI2000 performance.
;Sample Block Size
:The sample block size (in samples). In the provided parameter files, the sample block size is always equal to 100ms, regardless of the sampling rate.


===System Latency===
;Amp Channel
Generally, there is a particular BCI2000 state variable and state value associated with a given output. For example, if one target is flashed in a CursorTask application, then the state ''TargetCode''  should be examined and compared to the optical input channel. The difference in time between when the ''TargetCode''  state value changes (<math>t_{StateChange}</math>) and when the conditioned optical input changes values (<math>t_{InputChange}</math>) gives a measure of the output latency.
:This is the analog channel that records the amplifier digital output (i.e., the digital output recorded back in on an analog channel). This signifies the onset of acquisition of a block. This value should be between 1 and the number of channels.


However, state values are only changed at the start of a block of data, whereas the command to change the output occurs at the end of processing. Therefore, the time difference between a state change and associated change in output that is then immediately recorded at the amplifier must take the amplifier latency and processing latency into account. As a result, the calculated latency between the state change and signal change is actually the system latency, since the amp, processing, and output latencies are all included in this time difference:
;Amp Stream
:This is the digital channel that records the amplifier digital output (i.e., the digital output recorded back in on a digital input channel).


<center><math>\begin{matrix}
;Video Channel
\mathrm{System Latency} = t_{InputChange} - t_{StateChange}\\
:The recorded analog channel containing video event markers.
\mathrm{System Latency} = t_{6} - t_{0}
\end{matrix}</math></center>


===Application Display/Sound Output Latency===
;Video Stream
<span id="outOverview"></span>
:The recorded digital input channel containing video event markers.
There are many possible application output types, including visual (i.e., the computer screen), audio, or mechanical (e.g., a robotic arm). The core BCI2000 system primarily uses visual and audio output, and so the default testing system will focus on these.


Both visual and audio output can be measured using the \gtrig{} signal conditioner, which accepts inputs from optical sensors, microphones, or any other input. The user sets a threshold for the input signals, and the \gtrig{} outputs a digital pulse whenever this threshold is exceeded. This output is then recorded back into the amplifier system, so that the analysis program can measure the response times for each output.
;Video State
:The BCI2000 state that contains the event markers for the video event, e.g., TargetCode or StimulusCode.
The output latency is:


<center><math>\begin{matrix}
;Video State Values
\mathrm{Output Latency} = (\mathrm{System Latency}) - (\mathrm{Amplifier Latency}) - (\mathrm{Processing Latency})\\
:A list of state change values that should be detected by the analysis. For example, if values <code>4 11</code> are used, then the analysis tool will examine when the StimulusCode state changes to either 4 or 11, and then examine when the Video Channel changes.
\mathrm{Output Latency} = (t_{6}-t_{0}) - (t_{2}-t_{0}) - (t_{5}-t_{0})\\
\mathrm{Output Latency} = t_{6} - t_{5} - (t_{2} - t_{0})
\end{matrix}</math></center>


For example, assume that a block of data starts at 125 ms, has an amplifier latency of 1 ms (i.e., <math>t_{2} = 126</math> ms), finished processing data at 130 ms, and produces a video output at 134 ms.. The output latency is:
;Audio Channel
:The recorded analog channel containing audio event markers.


<center><math>
;Audio Stream
\mathrm{Output Latency} = 134 ms - 130 ms - (126 ms - 125 ms) = 3ms
The recorded digital input channel containing audio event markers.
</math></center>


It is important to note that this equation includes the theoretical output latency of <math>t_{6} - t_{5}</math>, and additionally accounts for the amplifier latency that is introduced when recording the optical or audio output from the \gtrig{}.
;Audio State
:The BCI2000 state that contains the event markers for the audio event, e.g., TargetCode or StimulusCode.


[[Image:OutputLatency.png|frame|none|Application output latency.]]
;Audio State Values
:A list of state change values that should be detected by the analysis. For example, if values <code>4 11</code> are used, then the analysis tool will examine when the StimulusCode state changes to either 4 or 11, and then examine when the Video Channel changes.


===Jitter===
====3. Control: Global Settings====
The jitter is calculated as the time between adjacent blocks of data, divided by the theoretical ideal value, which is the sample block size. From Figure \ref{fig:timeline}:
[[image:Tester02.PNG|frame|none|The BCI2000Certification user interface.]]
As the name suggests, the options under Global Settings are applied to all tasks, and are described here.


<center><math>
;Window Left, Top
\mathrm{Jitter} = \frac{t_{8} - t_{0}}{Sample Block Size}
:The position of the left and top sides of the application window. If using multiple monitors, this position should take this into account. For example, if the application window is to the right of the main monitor, and the main monitor has a resolution of 1280x1024 pixels, then <code>Window Left</code> should be set to 1280, and <code>Window Top</code> to 0.
</math></center>


===Latency Calculation Overview===
;Window Width, Height
Latencies are calculated by comparing changes in a BCI2000 state value to changes in a recorded signal. The analysis program follows these general steps in calculating the various latencies:
:The width and height of the application window.


;Global Signal Source
:The executable that should be used by all applications. This value is overwritten by the Signal Source value set for individual tasks.


;Data Output Directory
:The directory where all of the data should be saved during the task. This will appear relative to <code>BCI2000/tools/BCI2000Certification/</code>, or at a given absolute path. '''NOTE that if data already exists in this directory, additional data will be saved to it!''' To avoid combining new data with earlier results, the existing directory with the same name should be renamed or deleted.


#Determine which threshold values to use based on the ''thresh''  definition in the BCI2000Certification.cfg file, which defines a threshold based on the amplifier used. For example, the g.USBamp has an input range of -250 mV to + 250 mV, and the g.TRIGbox outputs from 0 to 200 mV, so the threshold used is about 12.5 mV.
;BCI2000 Directory
:This is the directory containing the BCI2000 program files, e.g., <code>C:\BCI2000\prog</code>


#Traverse the state values for a selected state and find where the state changes from 0 to a given value (or any value). The sample at which this occurs is saved for later use in <math>stateChangePos</math>.
;Additional Application Module command line arguments
#When this occurs, start looking through <math>Signal_{norm}</math> to find when a rising edge exceeds the determined threshold value. The sample at which this occurs is saved in <math>sigChangePos</math>.
:Additional arguments to add to the command line when starting the application module.
#The latency for the <math>n</math>th state change is calculated as:
<center><math>\begin{matrix}
latency_n = \frac{sigChangePos_n-stateChangePos_n}{SampleRate}
\end{matrix}</math></center>
and is stored in an array.
#Once the entire state vector is traversed, the mean, standard deviation, minimum and maximum values for this latency are calculated and reported.


;Shell execute before
:Execute command in the shell before running the experiment (e.g., ''taskkill /IM explorer'' to terminate the explorer shell and taskbar).


;Shell execute after
:Execute command in the shell after the experiment has finished.


==Getting Started With the Default Tests==
====4. Control: Controls====
This section provides step-by-step instructions on running the default certification procedure. Note that some cables must be assembled before the testing can be done; see Appendix \ref{sec:ampcables} for information on assembling the cables.
This panel handles starting and stopping the procedure, and running the analysis.


===Set Default Parameters===
;Auto Set Config
Using a text editor, open the latencyTest.prm file in the BCI2000/tools/BCI2000Certification/parms/ folder. Many of the settings should be left alone, except for some of the following cases.  
:Automatically press '''Set Config''' upon starting BCI2000. This is required for complete automation of the procedure. When unchecked, BCI2000 will pause after loading parameters, and you will be able to inspect parameters for debugging purposes. You will have to press '''Set Config''' manually to proceed.


====Sampling Rate====
;Auto Start
This is set to 512 by default. This can be changed to another value that will apply to all tests, except those that explicitly override it in their own parameter file.
:Automatically start the test after '''Set Config''' is pressed. This is required for complete automation of the procedure.


====Display parameters====
;Auto Quit
These are on the lines that begin with =Application:Window= . For many labs, the default settings should work fine; the window is setup to appear in the top-left corner of the main window (=WindowLeft= 0 \& WindowTop= 0= ), and is 800 wide by 800 pixels tall (=WindowWidth= 800 \& WindowHeight= 800= ).
:Automatically quit BCI2000 after each task. This is required for complete automation of the procedure.


Users with a dual-screen configuration may wish to change this so that the BCI2000 window is on the secondary monitor; to do so, the =WindowLeft=  parameter should be changed, which depends on the resolution and orientation of both monitors. For example, if the 2nd monitor is to the right of the primary monitor, and both monitors have a resolution of 1280x1024 pixels, setting =WindowLeft=  to =1280=  will place the window on the 2nd monitor. Similarly, if the 2nd monitor is to the left of the primary monitor, setting =WindowLeft=  to =-1280=  will accomplish the same thing. If the monitors have different resolutions (e.g., monitor 1 is 1600x1200 and monitor 2 is 1024x768, located to the right of the 1st monitor), the position should change accordingly. For example, in this scenario, to set the window to the right of the main window, set =WindowLeft=  to =1600= . Finally, set the =WindowWidth \& WindowHeight=  to appropriate values for full-screen, if desired. To summarize, for the dual-screen case in which monitor 1 is 1600x1200 pixels, and monitor 2, to the right of 1, is 1024x768 pixels, and the BCI2000 window should be full-screen on monitor 2, the parameters should look like:
;Start
:Starts the selected tasks.
\texttt{\\Application:Window int WindowWidth= 1024 ...\\
Application:Window int WindowHeight= 768 ...\\
Application:Window int WindowLeft= 1600 ...\\
Application:Window int WindowTop= 0 ...\\
}\\
Note that nothing following the ... needs to be changed above in the parameter file.


===Connect the Amplifier to the Trigger Box, and Attach Sensors===
;Analyze
\begin{figure}[h]
:Starts the analysis tool. This uses the directory entered in the '''Data Output Directory''' setting. See the documentation for the analysis tool for more information on the analysis procedure.
  \centering
  \includegraphics[width=1\textwidth]{figures/connections.pdf}
  \caption{Connections Overview}
  \label{fig:connections}
\end{figure}


This section describes the steps for connecting the necessary hardware. Refer to Fig. \ref{fig:connections} for additional information; each step is numbered in this figure.
;Progress
:While running, a progress bar appears and text provides information about the current running task, and how many tasks remain. The progress bar window includes a Cancel button, which will stop the procedure after the current task finishes.


===Setting Up and Running the Certification Procedure===
# Load the appropriate configuration file, based on the number of amplifiers that you will be using and as described above.
# Plug-in and turn on the amplifier(s).
# Configure the multi-monitor system, if applicable.
# Enter the appropriate screen dimensions under <code>Window Width</code>, etc.
# Prepare to place the optical detector on the monitor by using a ring of double-sided tape. The exact placement on the monitor will depend on the tasks used, and will be determined shortly. Plug the detector into the Optical Input A on the g.TRIGbox.
# Using a 1/8th in. cable, connect the headphone output on the PC to the Low Level input D of the g.TRIGbox.
# Connect the g.TRIGbox Trigger Out (D-Sub connector) to digital I/O 1 port of the g.USBamp using a standard cable provided by g.tec.
# Specify the ''Video Stream'' as ''DigitalInput1'' and the ''Audio Stream'' as ''DigitalInput4'' in the ''Events Configuration'' tab.
# Alternately, the audio and video event markers can be sampled by the analog channels. In this case you will need to specify the ''Video Channel'' and ''Audio Channel'' in the ''Events Configuration'' tab. In this case, leave the ''Video Stream'' and ''Audio Stream'' fields empty.
# In addition to measuring the latency of visual and auditory stimuli, you can also evaluate the delay of the analog-to-digital converter (ADC). This is done by connecting ''DigitalOutput1'' of the Digital I/O 1 port of the amplifier to the amplifier's digital input or analog input. This requires a custom cable. For details, see the [[User_Reference:BCI2000Certification#Assembling Required Cables|Assembling Required Cables]] section below. For the g.USBamp, specify the ''Amp Stream'' as ''DigitalInput0'' or the ''Amp Channel'' as the input channel accordingly.
<!-- # Next, the connectors should be configured. The procedure for creating connector cables is described in detail later in this document. To summarize the procedure for the g.USBamp, and as shown in the figure below:
## Using the digital I/O breakout connector for the g.USBamp, tie pin 3 (digital out 1) to pin 4 (digital in 0).
## Connect the digital out 1 female connector to a male connector, and plug the signal wire of the male connector into channel 1 of the amp.
## Connect output A of the g.TRIGbox to channel 2 of the amplifier.
## Connect output B of the g.TRIGbox to channel 3 of the amplifier.
## Using several EEG jumper wires, jumper the 3 ground cables together and input them to the Ground and Ref inputs of the amplifier.-->
# Next, the location to place the optical detector will be determined. The first task will be run, showing the location on the screen to place the connector.
# Press the Start button. BCI2000 will start shortly, and the application window will appear. In this test, the P300 Speller application is shown first.
# Place the detector over the "X" on the screen.
# Adjust the threshold on the g.TRIGbox for connector A so that it blinks on and off with the appearance of the "X". It may be necessary to turn off the lights in the room so that ambient light does not interfere with the optical detector.
# When finished placing the detector and setting the threshold, press Cancel in the BCI2000Certification GUI, and then Quit BCI2000. Navigate to the location of the Data folder (e.g., <code>BCI2000/tools/BCI2000Certification/</code>) and delete the data folder.
# Make sure that the computer volume is turned all the way up, and not muted.To set the audio threshold, set audio volume on the computer to maximum. On Windows, to make this process easier in the future, go to the Start menu, click the Control Panel, and go to Sounds. In the Sound window, click the box that puts the volume icon in the task bar. A small speaker should now appear in the bottom-right of the screen in the task bar. Click this icon, and slide the volume up to the max; when you click on this slider, Windows emits a short beep to give an idea of the volume level, which can be used to set the threshold. To do so, make sure that the stereo audio cable is plugged into the g.TRIGbox, and click the slider to emit a sound, checking that the green LED is flashing when you click the slider. If it is not, turn the threshold knob for channel B counter-clockwise, and click the slider again. If it is still not working after several tries, unplug the audio cable from the computer, and check that you can hear the beep through the speakers. If you cannot, make sure that the sound is not muted, and that the sound drivers are properly installed. If you can hear it, plug the audio cable back into the computer audio output (NOT the microphone input), and make sure the g.TRIGbox has power; the light next to the power switch should flash momentarily if the battery levels are OK. You can also use a wall power source (see the g.TRIGbox manual for more information). The threshold is set correctly when the channel B LED is off at rest, and is on when a sound is played.


[[File:Usbampconnections.png|800px|thumb|center|upright=2.5|Connectors schematic.]]


#Connect the amplifier to the computer; the supported g.tec amps are connected via USB, RS232, or Bluetooth.
#Connect the Digital I/O cable (see \ref{sec:digicable}) to the amplifier on the Digital I/O port.
#Connect the reference and ground wires from the assembled amplifier cables to the amplifier reference and ground connections.
#Connect the amplifier cables with the safety connectors into channels 1, 2, and 3 on the g.USBamp.
#Connect the digital output line to the channel 1 amplifier cable.
#Connect the A output from the trigger box to the channel 2 amplifier cable.
#Connect the B output from the trigger box to the channel 3 amplifier cable.
#Connect the optical sensor plug to the \gtrig{}, using the optical sensor input channel A.
#Connect the computer audio output using a stereo cable to the \gtrig{}, using the high-level input channel B.




===Setup Sensor Thresholds===
# You are now ready to run all of the tests. To do so, press the Start button on the GUI.
The audio and optical sensors each have a threshold setting on the \gtrig{}, along with an LED indicating whether the current input is above threshold for each channel. At rest, each trigger should be off; if any is on, turn the threshold knob counter-clockwise until the LED is off.
# While testing, some tasks may not complete correctly, either because they are too computationally intensive for the selected computer system, or some other mistake or artifact occurred during the test. In this case, BCI2000 can be Quit manually, and the remaining tasks will be run. At the end of all tasks, you can then de-select all tasks, and re-check just the ones that need to be completed. It is necessary to delete data files in the data folder from unusable runs prior to running them again, so that the bad data is not included in the analysis.
# When all tasks are complete, press the <code>Analyze</code> button to analyze the collected data. This procedure is detailed later in this document.


====Audio Threshold====
===Analyzing Data===
To set the audio threshold, set audio volume to maximum. On Windows, to make this process easier in the future, go to the Start menu, click the Control Panel, and go to Sounds. In the Sound window, click the box that puts the volume icon in the task bar. A small speaker should now appear in the bottom-right of the screen in the task bar. Click this icon, and slide the volume up to the max; when you click on this slider, Windows emits  a short beep to give an idea of the volume level, which can be used to set the threshold. To do so, make sure that the stereo audio cable is plugged into the \gtrig{}, and click the slider to emit a sound, checking that the green LED is flashing when you click the slider. If it is not, turn the threshold knob for channel B counter-clockwise, and click the slider again. If it is still not working after several tries, unplug the audio cable from the computer, and check that you can hear the beep through the speakers. If you cannot, make sure that the sound is not muted, and that the sound drivers are properly installed. If you can hear it, plug the audio cable back into the computer audio output (NOT the microphone input), and make sure the \gtrig{} has power; the light next to the power switch should flash momentarily if the battery levels are OK. You can also use a wall power source (see the \gtrig{} manual for more information). The threshold is set correctly when the channel B LED is off at rest, and is on when a sound is played.
[[Image:CertAnalysis.png|frame|none|The BCI2000 analysis tool.]]
Simply pressing the <code>Analyze</code> button in the Tester will start the Analysis program. It is also possible to run the program in the BCI2000Certification directory by starting <code>BCI2000CertificationAnalysis.exe</code>.


====Video Threshold====
# The BCI2000CertificationTester program stores information about how to interpret a session in the session data file itself. From there, it is picked up by the analysis program. You don't have to provide BCI2000CertificationTester .ini files to the analysis program.
To set the video threshold, first tape the optical sensor to the monitor in approximately the correct location, based on the information in the latencyTest.prm file for the window position and dimensions. To activate the optical trigger on the \gtrig{}, drag a white window, such as a Notepad window, in the background under the sensor. Move the threshold knob for channel A until the LED for that channel is just on (i.e., turning it down just a little turns the trigger off). The sensor will be placed more accurately in the following step.
# The analysis program will open in a new GUI window. When you clicked "analyze" in the tester program, the analyzer's file list is already populated. Otherwise, appropriate folders and files must be entered for this session:
## Press the "+" button to select the folder containing your data files. When you select the folder, all data files contained within its subfolders are added automatically.
# Finally, set the directory for the output file. Typically, this should be the same folder that contains all of the data files.
# When the parameters have been correctly configured, press "Run" to start the analysis, which can take several minutes. Note that debug builds run very slowly, so you should always use a release build for analysis.
# When finished, the analysis tool will display results at several levels of detail. On top, a summary is displayed, then a list of requirements is shown, with a global result for each requirement. Further down, results are shown for individual files.
# To store results for documentation, you may press ''Save and Open...'' to save results to a text file, which can be found in the folder choosen at the bottom left, defaulting to the data folder. The file name will have the format <code>results_(date)_(time).txt</code>. Additionally, this contents of this file will be opened automatically at the end of the process in a new window.


===Starting the Software===
===Interpreting Analysis Results===
\label{sec:startingsoftware}
Navigate to BCI2000/tools/BCI2000Certification, and double-click the \bcicert{} program. The BCI2000 program should start, with the output window in the location specified in the latencyTest.prm file. If there are any errors at startup, the BCI2000 error log will report them. If there are errors, correct them, exit out of the \bcicert{} program, and restart the program.


The certification analysis results are shown as the overall result at the top of the analysis tool. In the example below, 33/60 tasks passed, while 27 tasks did not meet the success criteria. Additionally, 48 tasks are missing data, as they do not include auditory stimuli (this is by design). In more detail, the results are segregated into three sections, with overlapping information contained in each section. These sections are Requirements, Performance Statistics, and Individual Tasks. The Requirements section provides details about the global performance of the various certification tests. The success criteria are somewhat arbitrary and are based on our experience measuring response latency in various setups. The Performance Statistics section contains very similar information, with the statistics reported at the top level for easy viewing. Finally, the Individual Tasks section organizes the certification results by task.


[[Image:Analysis03.PNG|frame|none|The BCI2000 analysis results. Results are described in three sections: Requirements, Performance Statistics, and Individual Tasks.]]


Navigating into the sub-menu structure provides details about the various tests run, including the stimulus source, the method used to determine the event latency, and the event latency and variance across trials.
[[Image:6.PNG|frame|none|The BCI2000 analysis results. Individual Tasks.]]


The results text file contains detailed results for all tasks. A sample output for a single task is shown here:


[[Image:Analysis_03_3.PNG|frame|none|The BCI2000 analysis results. The top of the file includes details about the global results.]]
[[Image:Analysis03_4.PNG|frame|none|The BCI2000 analysis results. Throughout the file there are details about each task.]]




====A Note on "Pass/Fail"====
The values that determine whether or not a task passes or fails are determined somewhat arbitrarily. For example, we determined that an audio output latency of more than 65 ms is unacceptable for psychophysical research, e.g., BCIs. In the example shown, the audio output latency was ~62 ms for all tasks with audio, and thus these tasks registered as a "pass." However, it is ultimately up to the researcher to determine what is acceptable system timing; this method provides a method of quantifying the system latency values to simplify the process of making these decisions.


You may move the slider near the top of the window in order to display the original GUI elements you saw before clicking "Run". When clicking the checkbox close to "Override requirements", a small editor window appears, containing code for all requirements applied to the data. You may modify requirements there, as you see fit.


====Visualizing Certification Test Results using GNUPlot====


===Setting up the Optical Sensor===
If you have GNUPlot installed, you can also visualize the distribution of latencies. Simply right-click on the field and select ''plot.''
Each application has a unique display configuration, where the visual target of interest is located in a specific position on the monitor. Generally, the optical sensor will not likely have to be moved far in between tasks due to the design of the parameter files. However, it is still necessary to ensure that the sensor is in the correct position for each task, and to be able to change the position if necessary. This section describes the process to find the correct location on the monitor for the sensor. To start, the certification procedure should have been started, as described in Sec. \ref{sec:startingsoftware}.


====CursorTask====
In the default certification configuration, the basic cursor task will be the first to run. The BCI2000 window will contain a black background with a small white cross near the middle. The optical sensor should be placed over this X.
====P3Speller====
The P3Speller task has an group of letters arranged into rows and columns. The analysis uses the letter in the 3rd column and 3rd row, which will contain the letter X when the program is run. This location should be very close to that specified for the CursorTask, described above. Fortunately, the arrangement of the letters is shown before the test starts, which allows the sensor to be adjusted if necessary.
====StimulusPresentation====
For the StimulusPresentation tasks, a single large icon appears in the center of the screen, which will be detected by the sensor. The default location of the sensor as defined for the CursorTask should be able to capture the icon on the screen. However, if the task begins and it appears that it is outside the icon location, move the sensor to the middle of the icon. Keep in mind that it may have to be relocated for following tasks.


===Running the Tests===
[[Image:CertificationTool_GNUPlot.PNG|frame|none|GNU Plot.]]
Once the optical sensor is setup and calibrated for a task, begin the testing process. To start the process, press the Start button in the BCI2000 operator window. At the end of each task, the program will quit automatically, and the next one will start automatically. For each task, press the Start button to begin.


Occasionally, the optical sensor may not be in the exact correct place for a given task.You should be able to see changes in on the video channel (channel 2 by default) in the window showing the recorded channels. If the signal is not changing when it should be (e.g., during the correct visual stimulus), you may try increasing the threshold to catch the stimulus, or you may need to place the sensor in the correct location. To do this, place the sensor in the better location, then quit the current BCI2000 test, and quit the BCI2000Certification program. The procedure will need to be restarted.
==Assembling Required Cables==
Depending on what hardware and the latencies of interest are, you may or may not need to assemble custom cables. If you are interested in measuring the audio and visual latency with the gTRIGbox and an amplifier manufactured by a company other than g.tec, you will need to assemble a custom cable going from the output of the gTRIGbox to the digital inputs or analog inputs of your amplifier. It is also possible measure the audio and visual latency without the gTRIGbox, but this will require another trigger onset detection method. If you have a g.tec g.USBamp and want to measure the system latency and the ADC latency, you can follow the following guide.  


When the testing is complete, the BCI2000CertAnalysis program will automatically start. It will analyze the data that have been collected, and return the results in the results.txt file. In general, a Pass or Fail will be written next to a task, and each task component, if it passed or failed the test. For more information on interpreting results, see Sec. \ref{sec:results}.
In the following sections, we provide examples on how to build this cable if you’re using the gTRIGbox and a gUSBamp with a UB serial number (see the [[User_Reference:gUSBampADC#Digital_Inputs|gUSBamp digital input page]] for more information regarding the difference pin assignments between serial numbers). There are two versions of the gTRIGbox, an older legacy version with four 3.5 mm audio connector outputs, and a newer one that has a 15 pin D-sub type connector. We describe the process of building cables for both of these devices.  


==BCI2000Certification: Additional Information==
===Legacy gTRIGbox===
The previous section on Getting Started provided information to quickly setup and run the default tests. This section provides detailed information about each file used, and how to modify the files for your specific system and applications.
[[Image:Usbampconnections_legacy.png|frame|none|Connectors schematic.]]
====Amplifier Input====
Parts List:


===Overview of Important Files===
*3 Male Mono Audio Connectors 
There are several important files that are used in the BCI2000 Certification procedure. These files and their contents are described in this section.
*6 12in+ Safety Connector Wires (to USB amplifier input), different colors 
*Soldering Iron and Solder


====BCI2000Certification.ini====
This section describes how to assemble cables that connect the output of the g.TRIGbox to the amplifier input channels.
The BCI2000Certification.ini file contains the information that defines each task configuration. It is located in the BCI2000/tools/BCI2000Certification/ folder. It specifies the parameter file, the SignalSource, SignalProcessing, and Application modules, and the analysis definitions for each task. It contains all of the default tests, although it can easily be expanded to include additional or customized task configurations.


Each task is defined by a set of '''case-insensitive''' , space-separated fields, as described below. A * indicates an optional field. The \
# Strip the free ends of the 6 safety connector wires (~1/4 in)
# Take one wire, and solder the wire to the "signal" pad of one of the mono audio connectors.
# Solder another wire to the "ground" pad of the same connector.
# If necessary, pass the wires through end of the connector that screws in to cover the connection (see below, in the ''g.USBamp Amplifier Plug'').


\begin{description}
[[Image:ampPlug_1.png|frame|none|g.USBamp Amplifier Plug]]
\item[name]The name of the task, which should be unique. This generally describes the task. It must not contain any spaces. Example:


'''name StimulusPresentation'''
Each connector now has a signal and ground wire that can be connector to the g.USBamp inputs, or any amplifier that accepts the standard EEG connector.


\item[folder]The name of the folder in the BCI2000/tools/BCI2000Certification/latencyData/ folder that contains the data for this task. Example:
====Digital I/O Cable====


'''folder SimpleAV'''
Parts List:
 
\item[parm]The BCI2000 *.prm file used for this task. It should be placed in the BCI2000/tools/BCI2000Certification/parms/ folder. Note that the latencyTest.prm is included in ''all''  configurations by default, and does not need to be listed here. This file contains the information for this specific task configuration. See section \ref{sec:parmFiles} for further information.
 
'''parm SimpleAV.prm'''
 
\item[source] The SignalSource module to be used for this test. The \gtec{} g.USBamp system is used in all configurations by default. If the g.MOBIlab or g.MOBIlab+ are used, the SignalSource module should be changed to the appropriate executable. All BCI2000 executables are located in BCI2000/prog/. Examples:
 
'''source gUSBampSource.exe'''  \\
or \\
'''source gMOBIlabSource.exe'''
 
\item[sigproc] The SignalProcessing module that is used for this configuration. This is likely dependent on the application used as well. This program should be located in BCI2000/prog/. Example:
 
'''sigproc P3SignalProcessing.exe'''
 
\item[app] The application for this configuration. This program should be located in BCI2000/prog/. Example:
 
'''app StimulusPresentation.exe'''
 
\item[*amp] This is a single value that gives the channel that is used to compute the amplifier latency. Example:
 
'''amp 0'''
 
\item[*vid] This field is followed by 3 values, that give the channel number that is recording the video output, and the state name and state value to use as a trigger for the comparison. See the example entry for more information. Example:
 
'''vid 1 StimulusCode 1'''
 
\item[*aud]  This field is followed by 3 values, that give the channel number that is recording the audio output, and the state name and state value to use as a trigger for the comparison. See the example entry for more information. Example:
 
'''aud 1 StimulusCode 1'''
 
\item[skip] Entering '''skip'''  will cause this task to not be run or analyzed. This is used to reduce the number of tasks that are run, which helps to repeat individual tasks that may have had problems in previous sessions.
 
\item[end] This must come at the end of each task definition.
\end{description}
 
In addition to task-specific entries, there are several global entries that will apply to every task, unless explicitly overwritten in that task. This allows the user to quickly change settings for all tasks simultaneously, e.g., if tests will be run using a \gtec{} g.USBamp for one set of tests, and a \gtec{} g.MOBIlab for another. Global entries must be defined at the beginning of the BCI2000Certification.ini file outside of the task definitions. These entries are defined as:
 
\begin{description}
\item[source] The SourceModule to be used for all tests, unless explicitly overwritten by an individual task. Otherwise identical to the '''source'''  definition in a task entry.
\item[export] This entry does not use any other parameters on the line on which it is defined. If '''export'''  is present in the BCI2000Certification.ini file, test results will be written to a comma-separated-value (CSV) text file that is located in the folder where data is saved for each task. For example, if the data folder is BCI2000/tools/BCI2000Certification/latencyData/, then the export file for the SimpleTarget task will be placed in the folder\\
BCI2000/tools/BCI2000Certification/latencyData/SimpleTarget.
\end{description}
 
Here is a full example entry that includes two tasks.
 
\texttt{\\
source gUSBampSource\\
export\\
\\
name StimulusPresentation\\
folder SimpleAV\\
parm SimpleAV.prm\\
sigproc P3SignalProcessing\\
app StimulusPresentation\\
amp 0\\
vid 1 StimulusCode 3\\
aud 2 StimulusCode 3\\
end\\
\\
name SimpleTarget\\
folder SimpleTarget\\
parm SimpleTarget.prm\\
source gMOBIlab\\
sigproc ARSignalProcessing\\
app CursorTask\\
amp 0\\
vid 1 TargetCode 1\\
end\\
}\\
 
The =StimulusPresentation=  task uses the =gUSBampSource=  source module, and records block-triggered digital outputs on channel 0, video output triggered off of stimulus code 3 on channel 1, and audio output triggered off of stimulus code 3 on channel 2. The =SimpleTarget=  tasks overrides the global =source=  entry, and uses its own =source=  definition, specifying that a gMOBIlab should be used for acquisition instead of a gUSBamp. Finally, the =export=  entry is set at the beginning, so test results for all files are output to a CSV file.
 
The BCI2000Certification.ini file contains many task definitions. Each entry is read by the \bcicert{} program, and is entered as a separate task to be run and then analyzed.
 
====BCI2000Certification.cfg====
The latencyTest.prm file is located in the BCI2000/tools/BCI2000Certification/ folder. It contains many default settings and definitions that determine how the results are interpreted, and should generally not be changed by the user. The timing definitions provide the absolute maximum latencies in ms for each step in the system chain, and include the Amp, ProcLat, VidOut, AudOut, VidSys, AudSys, and Jitter (see below for explanations). Timing definitions can include either one or two fields; the first field is mandatory, and is the mean value in ms, while the second field is optional, and is the standard deviation in ms.
 
\begin{description}
\item[Amp] The amplifier latency.
\item[ProcLat] The processing latency.
\item[VidOut] The video output latency.
\item[AudOut] The audio output latency.
\item[VidSys] The video system latency.
\item[AudSys] The audio system latency.
\item[Jitter] The system jitter. This is the difference, in ms, between the actual block duration and theoretical block duration, which is determined by the block size.
\item[Threshold Source Value] This is the threshold defined for a specific SignalSource module. Individual amplifiers may have different input ranges, resulting in recording signals with different ranges of magnitude. The '''Source'''  entry is a value in mV. For example, the threshold setting for the g.USBamp is '''Threshold gUSBampSource 12.5''' .
\item[ResultOut] This is the file that contains the testing results.
\item[DataDir] The default data directory. It defaults to latencyData in the BCI2000/tools/BCI2000Certification/ folder.
\end{description}
 
These values should not generally be changed by the user, although they may be updated in future BCI2000 versions.
 
====latencyTest.prm====
This parameter file contains some of the default task parameters common to every test, including the save location, monitor configuration, sampling rate, and visualization settings. Note that these parameters can be overwritten in the *.prm files for each individual test configuration.
 
====\bcicert{====}
This file controls the execution of each task. It parses the BCI2000Certification.ini and BCI2000Certification.cfg files to determine which files and tasks to run, and controls the CPU load monitor. When all tasks are complete, this program calls the BCI2000Analysis.exe program to handle the analysis of the collected data.
 
====BCI2000CertAnalysis.exe====
This program analyzes the data collected, and reports results to a data file. It uses the DataDir parameter in the BCI2000Certification.cfg file to determine where the data is located. Results are output to results.txt in the BCI2000/tools/BCI2000Certification/ folder.
 
====Parameter Files====
\label{sec:parmFiles}
Each task should have it's own parameter file that contains the unique configuration and settings for that task. For example, there may be several configurations that use the g.USBamp, ARSignalProcessing, and CursorTask, each with its own parameter file. These files end in .prm, and are placed in the BCI2000/tools/BCI2000Certification/parms/ folder.
 
 
 
 
 
 
 
 
%
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
===Interpreting Results===
\label{sec:results}
Results are recorded to the BCI2000/tools/BCI2000Certification/results.txt file. The results for each task, and each applicable latency tested, are reported here. If the entire task  passed, then each test performed met the minimum timing requirements specified in the BCI2000Certification.cfg file. If one or more did not pass, then the specific test that did not pass the test has Fail written next to it.
 
====What if a Task Did Not Pass?====
TO BE ADDED.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
%
 
 
 
 
 
 
 
 
 
%
 
 
 
 
 
 
 
 
 
 
==Certification Protocol==
\label{sec:TestRun}
This section describes how the testing protocol is run, the various components involved in the certification analysis, and how to interpret results.
 
===Running the Tests===
The certification protocol consists of running several BCI2000 applications in multiple configurations to determine if each application meets the minimum requirements for BCI2000, and to determine if individual configurations of an application meet the requirements. For example, the CursorFeedback task may be tested with several combinations of sampling rates, sample block sizes, and processed channels, only some of which may work with a given computer setup.
 
The testing set is run from a single batch file called LatencyAnalysis.bat. This file is located in the folder BCI2000/tools/BCI2000Certification/batch. This batch file executes additional batch files in this folder, each corresponding to a different application configuration. Each of these batch files starts the Operator module, automatically loading multiple parameter files used for a given configuration. There is a general-purpose parameter file called latencyTest.prm that all configurations use. This file configures the save location for all latency tests, as well as default window positions and sampling rates. A second parameter file is passed to the operator that contains the parameters for a unique application configuration. After starting the operator module, the batch file starts the SignalSource module, SignalProcessing module, and Application module. The loaded configuration is set automatically, and the user needs only to press the start button to start the analysis.
 
 
\begin{appendices}
==Software Templates==
This describes how to implement the testing protocols for your own amplifier system and custom applications. This is useful for determining if your hardware setup and software is capable of maintaining an acceptable level of performance for real-time testing.
 
===Amplifier Latency===
The amplifier latency requires an output that can be recorded by the amplifier. In the case of the g.USBamp, a digital output line is used, although either analog or digital outputs could be used in a system.
 
In the SignalSource module code for your custom amp, find the Process function. Near the beginning of this function, before any data is acquired, set the output signal to 0, either a digital low or analog value of 0 V. This will be dependent on the particular amplifier and API used; for the g.USBamp, this function call looks like:
 
<br><code>
GT_SetDigitalOut ( m_hdev.at(0), (UCHAR)1, (UCHAR)0);
</code><br>
 
This sets the digital output 1 on device 0 to a value of 0.
 
At the end of the Process function, the output should be set to a high level. A similar function call will be used to above, only the value is 1 instead of 0; again, the function will be dependent on the system used, and whether analog or digital output is used. Here is the code for the g.USBamp:
 
<br><code>
GT_SetDigitalOut ( m_hdev.at(0), (UCHAR)1, (UCHAR)1);
</code><br>
 
This sets the digital output 1 on device 0 to a value of 1.
 
===Visual and Audio Latency===
Developing new video and optical latency tests does not necessarily require adding additional code to existing modules (beyond any customization already done).
 
==Supported Hardware==
A list of officially supported hardware configurations, and suggested minimum requirements is provided.
 
===Supported and Tested Hardware===
====Amplifiers====
    
    
*g.USBamp 
*3 Colored Electrical Wires
*gMOBIlab (Serial RS232)  
*7-Pin Digital Connector from '''g.tec'''
*gMOBIlab (Bluetooth)  
*Soldering Iron and Solder  
*TDT Pentusa (RX5)  
*Silicone or electrical tape for insulation  
*1 Female Mono Audio Plug  


===Minimum Requirements===
In order to access the digital I/O pins on the g.USBamp, a connector needs to be assembled. The 7-pin digital I/O connector should be ordered through g.TEC, the manufacturer of the g.USBamp.
Table \ref{tab:minreqs} shows the minimum requirements for running BCI2000. THESE WILL BE UPDATED LATER BASED ON OUR RESULTS!
 
\begin{table}[h]
\caption{Minimum System Requirements for BCI2000}
\begin{center}
\begin{tabular}{|c|c|}
\hline
CPU & 1 GHz\\
\hline
RAM & 512 MB (dependent on OS; 1 GB for Vista) \\
\hline
Video Card & 32 MB Ram; OpenGL support\\
\hline
Sound Card & \\
\hline
Monitor & \\
\hline
OS & Windows 2000; Windows XP SP2 or greater; Windows Vista\\
\hline
 
\end{tabular}
\end{center}
\label{tab:minreqs}
\end{table}
 
 
 
\pagebreak
===Voltage Divider \& Amplifier Input===
\label{sec:ampcables}
Parts List:
 
 
*4 Male Mono Audio Connectors 
*6 12in Safety Connector Wires (to USB amplifier input), different colors 
*4 3in Wires 
*Soldering Iron and Solder 
*resistors for voltage dividers 
This section describes how to assemble cables that connect the output of the \gtrig{} to the amplifier input channels.


Strip the free ends of all 6 safety connector wires, about <math>\frac{1}{4}</math> in. If you are not using voltage dividers, solder 4 of the wires directly  to the center signal connection on the 4 mono audio connectors (Fig. \ref{fig:ampPlug1}). Cover the connection.
At least three wires are needed for the circuit, which connect to the digital output 1 pin, the digital input 0 pin, and the ground pin.
[[Image:DigIO.png|frame|none|g.USBamp Digital I/O Pins]]
[[Image:digConnector.png|frame|none|g.USBamp Digital I/O Pins]]


\begin{figure}[h]
# Strip the ends of the 3 wires (~1/4 in)
  \centering
# Take one wire, and solder the wire to the "signal" pad of the female mono audio connectors.
  \includegraphics[width=.75\textwidth]{figures/ampPlug_1}
# Solder another wire to the "ground" pad of the same connector.
  \caption{g.USBamp amplifier Plug}
# If necessary, pass the wires through end of the connector that screws in to cover the connection (see below, in the ''g.USBamp Amplifier Plug'').
  \label{fig:ampPlug1}
# Solder the ground wire to the center ground pin on the digital I/O connector.
\end{figure}
# Locate the top of the connector by locating the small plastic notch, and locate pins 3 and 4 relative to the notch.
# Solder the signal wire from the mono connector to pin 3 (digital out 1)
# Finally, solder the remaining wire (it has not been connected to anything yet) between pin 3 and pin 4. This connects the digital output directly to the digital input.
# It may be helpful to insulate the wires from the casing using either a piece of black electrical tape, or an insulating gel, like silicone. If a gel is used, be sure to let it set for several hours or overnight. It is also probably a good idea to check the electrical connection with a multimeter prior to insulating.
# Next, attach the rubber sleeve over the end of the part with the female threads, as shown in the panel.
# Slide the smaller metal sleeve over the wires, then slide the larger rubber sleeve over the wires.


Next, twist the two remaining safety connector wire ends together, and solder them together. Solder the 4 short wires to the ground connections on the mono plugs. Finally, solder the 4 ground wires to the two safety connector wires that were twisted together and soldered previously, as in Fig. \ref{fig:ampPlugs2}.
[[Image:digPlug.png|frame|none|g.USBamp Digital Wire Plug]]


\begin{figure}[h]
# For the next step, attach the two metallic pieces that surround the cables, shown laying to the sides in the figure below.
  \centering
# One of the metal pieces has a small groove in it, which should be placed over the notch on the connector; the other metal piece fits on the opposite side of the connector.
  \includegraphics[width=.75\textwidth]{figures/ampPlugs_2}
# Push the small metal sleeve up to the side piece with the groove, and rotate the sleeve until the notch fits in the groove.
  \caption{g.USBamp Amplifier Plugs}
# Place the end piece over the connector, with the red dot on top and aligned with the connector notch.
  \label{fig:ampPlugs2}
# Finally, slide the rubber sleeve over the connector, and screw the sleeve into the end piece.
\end{figure}


These plugs can plug directly into the \gtrig{} outputs, and the safety connectors plug into the channels 1-4 inputs on the g.USBamp; the ground wires plug into the ground and reference on the g.USBamp.
[[Image:finalDigCable.png|frame|none|g.USBamp Digital I/O Cable]]


The plug is assembled next, similar to the. Solder the signal wire from Pin 3 (Digital Out 1) on the digital I/O connector to the signal connector on the female mono plug, and solder the ground wire to the outer ground connector on the plug. The below figure shows this; in this figure, the red wire is the signal wire, and the brown wire is the ground.
===Latest gTRIGbox===
There are two ways to connect the output of the gTRIGbox to the g.USBamp, “Analog” or “Digital”, as indicated in the figure below. Pins connected by a  <span style="color:#f97e00">orange</span> dotted line denote connections that are necessary to measure the amplifier latency.
[[File:Usbampconnections.png|800px|thumb|center|upright=2.5|Connectors schematic.]]


\pagebreak
[[File:GTrigBoxOutputSchematic.png|800px|thumb|center|upright=2.5|gTrigBox output schematic.]]
===Digital I/O Cable===
\label{sec:digicable}


Parts List:
====Digital Configuration====
The digital configuration uses digital I/O 1 on the back of the g.tec amplifier. It requires a specific 7 pin g.tec connector that can be ordered from g.tec, the manufacturer of the g.USBamp.
Parts list for digital cable:
# One 15 pin D-Sub Type Female Connector
# One 7 pin Digital Pin Connector from g.tec
# Electrical Wire
# Soldering Iron and Solder


 
The D-Sub Type connector can be purchased such that you only need to strip the electrical wires and then insert them to receptacles that tighten with a screwdriver, thus reducing the need for any soldering. This makes the process of assembling the cable easier. In example of this type of connector is shown below:
*3 Colored Wires 
*7-Pin Digital Connector from g.tec 
*Soldering Iron and Solder 
*Silicone or electrical tape for insulation 
*1 Female Mono Audio Plug 


In order to access the digital I/O pins on the g.USBamp, a connector needs to be assembled. The 7-pin digital I/O connector should be ordered through g.TEC, the manufacturer of the g.USBamp.
[[File:F15pin_d-sub_connector.jpg|600px|thumb|center|upright=2.5|15 Pin D-Sub Connector with Breakout Board.]]


At least three wires are needed for the circuit, which connect to the digital output 1 pin, the digital input 0 pin,  and the ground pin (Fig. \ref{fig:digIO}).
In the gUSB Digital Out cables we have seen, here is the coloring of the wires:
\begin{figure}[h]
{| class="wikitable"
  \centering
|-
  \includegraphics[width=.65\textwidth]{figures/digIO}
! colspan="2" style="text-align:center; font-weight:bold;" | DIGITAL I/O CABLE
  \caption{g.USBamp Digital I/O Pins}
|-
  \label{fig:digIO}
| gTRIGbox Wire Color
\end{figure}
| g.USBamp Digital I/O 1
|-
| Blue
| 7
|-
| White
| 1
|-
| Brown
| 2
|-
| Green
| 3
|-
|Yellow
|4
|-
|Pink
|5 or 6
|-
|Grey
|5 or 6
|}


To start, cut three 10 in wires, preferably with  different colors to differentiate between the signal and ground wires. Cut back a small portion of the ground wire, and solder it to the center ground pin on the connector. Now, find the top of the connector by locating the small plastic notch, and locate pins 3 and 4 relative to the notch. Solder one wire to pin 3, and the remaining wire to pin 4. It may be helpful to insulate the wires from the casing using either a piece of black electrical tape, or an insulating gel, like silicone. If a gel is used, be sure to let it set for several hours or overnight. It is also probably a good idea to check the electrical connection with a multimeter prior to insulating.


\begin{figure}[h]
<!--
  \centering
{| class="wikitable"
  \includegraphics[width=.5\textwidth]{figures/digConnector}
|-
  \caption{g.USBamp Digital I/O Pins}
! colspan="2" style="text-align:center; font-weight:bold;" | DIGITAL
  \label{fig:digConnect}
|-
\end{figure}
| gTRIGbox Output
| g.USBamp Digital I/O 1
|-
| 1 (GND)
| 7
|-
| 2 (A)
| 2
|-
| 3 (B)
| 3
|-
| 4 (C)
| 4
|-
| -
| style="color:#f97e00;" | 1 & 5
|}
-->
=====Procedure for Assembling Cable=====
# Cut 3 pieces of wire, each the same length (about 3 ft should be sufficient).
# Strip the ends of these pieces of wire, leaving about a quarter-inch of wire exposed.
# Start with the D-Sub connector. Connect each of the pins in ''gTRIGbox Output'' column of the table above to its own piece of wire.
# Solder the other ends of each of these wires to its appropriate location, as defined in the table above, on the g.USBamp digital I/O connector.
# If you also want to measure amplifier latency, strip and solder a piece of wire between pins 1 and 5 on the g.USBamp digital I/O connector.
# When assembling the digital I/O connector, it may be helpful to insulate the wires from the casing using either a piece of black electrical tape or an insulating gel, like silicone. If a gel is used, be sure to let it set for several hours or overnight. It is also probably a good idea to check the electrical connection with a multimeter prior to insulating.
# Next, attach the rubber sleeve over the end of the part with the female threads, as shown in the panel.
# Slide the smaller metal sleeve over the wires, then slide the larger rubber sleeve over the wires.


Next, attach the rubber sleeve over the end of the part with the female threads, as shown in Fig. \ref{fig:digFinal} in the left side of the figure. Slide the smaller metal sleeve over the wires, then slide the larger rubber sleeve over the wires, as shown in Fig. \ref{fig:digFinal}.
====Analog Configuration====
The analog configuration uses the analog inputs on the front on the amplifier to measure latency. To also measure the amplifier latency you will need a specific g.tec connector.  
Parts list to make analog cable to measure system latency:  
# One 15 pin D-Sub Type Female Connector
# Five Safety Connector Wires (to USB amplifier input)


For the next step, attach the two metallic pieces that surround the cables, shown laying to the sides in Fig. \ref{fig:digFinal}. One of the metal pieces has a small groove in it, which should be placed over the notch on the connector; the other metal piece fits on the opposite side of the connector. Push the small metal sleeve up to the side piece with the groove, and rotate the sleeve until the notch fits in the groove.
If you also want to measure amplifier latency, you will also need:
# One 7 pin Digital Pin Connector from g.tec
# One Additional Safety Connector Wire (to USB amplifier input)
# Two resistors for Voltage Divider
# Soldering Iron and Solder


Place the  end piece over the connector, with the red dot on top and aligned with the connector notch. Finally, slide the rubber sleeve over the connector, and screw the sleeve into the end piece.
If you use the D-Sub connector with a breakout board and do not want to measure amplifier latency, you will not need to do any soldering, making assembly quite easy. Below is a table of what pin connections must be made. The wiring diagram above provides a guide for what pins need to be connected.
<!--
The connections shown in <span style="color:#f97e00">orange</span> are necessary to measure amplifier latency.
{| class="wikitable"
|-
! colspan="3" style="text-align:center; font-weight:bold;" | ANALOG
|-
| gTRIGbox Output
| g.USBamp Digital I/O 1
| g.USBamp Analog Input
|-
| 1 (GND)
| -
| REF & GND
|-
| 9 (A)
| -
| 2
|-
| 10 (B)
| -
| 3
|-
| 11 (C)
| -
| 4
|-
| -
| style="color:#f97e00;" | 5
| style="color:#f97e00;" | 1
|}
-->
=====Procedure for Assembling Cable=====
# Strip the ends of five wires with safety connectors at one end, leaving a quarter-inch exposed (if you want to measure amplifier latency, strip one additional wire).
# Start with the D-Sub connector. Connect each of the pins as shown in the connector schematic to its own piece of safety connector wire.
# If you also want to measure amplifier latency, a voltage divider is necessary to bring the 5V output at the gUSBamp's digital output at pin 5 down to a voltage less than 250mV (the max input range on the gUSBamp's analog inputs). Selecting a <math>R_1</math> and <math>R_2</math> such that the following equation is satisfied would result in a 100mV signal at the analog inputs:
<math>0.02 = \frac{R_2}{R_1+R_2}</math>
# Assemble this voltage divider by soldering the resistors, pin 5 on the digital I/O, and additional safety connector together as detailed in the connectors schematic. You can also use a bread board to avoid soldering.
# When assembling the digital I/O connector, it may be helpful to insulate the wires from the casing using either a piece of black electrical tape or an insulating gel, like silicone. If a gel is used, be sure to let it set for several hours or overnight. It is also probably a good idea to check the electrical connection with a multimeter prior to insulating.
# Next, attach the rubber sleeve over the end of the part with the female threads, as shown in the panel.
# Slide the smaller metal sleeve over the wires, then slide the larger rubber sleeve over the wires.


\begin{figure}[h]
== Troubleshooting ==
  \centering
Things to watch out for:
  \includegraphics[width=1.\textwidth]{figures/finalDigCable}
*Parameter files for gUSBamp with channels greater than 16 have pre-defined amplifier IDs.  Please set these to your amplifier IDs.
  \caption{g.USBamp Digital I/O Pins}
*BCI2000Certification runs BCI2000 applications from the working directory: <tt>C:\BCI2000\tools\BCI2000Certification</tt>. The images folder there doesn't contain all required images for <tt>CursorTask</tt> or a sounds folder for "StimulusPresentation". If necessary, paste the images and sounds folder from the <tt>BCI2000/prog</tt> directory into the <tt>BCI2000/tools/BCI2000Certification</tt> directory - overwriting what is currently in that directory.
  \label{fig:digFinal}
\end{figure}


The plug is assembled next. Solder the signal wire from Pin 3 (Digital Out 1) on the digital I/O connector to the signal connector on the female mono plug, and solder the ground wire to the outer ground connector on the plug. Figure \ref{fig:digplug} shows this; in this figure, the red wire is the signal wire, and the brown wire is the ground.
==See also==
[[User Reference:Module Command Line Options]], [[User Reference:Operator Module]]


\begin{figure}[h]  
[[Category:Performance Measurement]][[Category:Data Analysis Tools]][[Category:Video]]
  \centering
  \includegraphics[width=.75\textwidth]{figures/digPlug}
  \caption{g.USBamp Digital Wire Plug}
  \label{fig:digplug}
\end{figure}

Latest revision as of 17:59, 3 September 2024

Introduction

BCI2000 includes a certification procedure that can determine whether a computer system is capable of running all or some BCI2000 applications. Because BCI2000 does not have a standardized hardware configuration, i.e., it can run on potentially any PC, this testing procedure is capable of running on any PC with BCI2000, with the results reporting which standard BCI2000 applications meet the minimum timing requirements. Different configurations are included in the certification procedure to test a range of likely setups for a given application, in which timing characteristics such as the sampling rate, inter-stimulus intervals, and stimulus duration, and other parameters, such as the number and size of the stimuli, are changed for each configuration.

This document serves as the instruction manual for using the certification procedure, and describes how the procedure works to determine the timing characteristics of the BCI2000 system. The primary use for the certification procedure is to determine if the PC configuration used with BCI2000 is capable of running the core BCI2000 system, which is comprised of:

  • Support for various biosignal amplifiers, including the g.USBamp, EGI GTEN 200, Nihon Kohden JE120-A, and many more on the ADCs page.
  • The AR, P3, FFT, and Matlab signal processing modules.
  • The CursorTask, P3Speller, and StimulusPresentation application modules.

Additionally, due to the extensibility of BCI2000, researchers who develop custom algorithms and applications can use the certification procedure to determine whether their custom modules are capable of running on a particular PC configuration.

The certification procedure works by comparing the differences in time of when BCI2000 expects an event to occur to when that event actually occurred. These events include a change in the display, an audio output through the system speakers, or an EEG sample being stored in the amplifier buffer. The times at which such events occur are recorded with the amplifier and stored in a data file. Changes in the display are found using an optical sensor and a triggering device. The documentation provided throughout this page uses the g.tec g.TRIGbox, although any device which can provide a digital or analog trigger to the amplifier can be used. The g.TRIGbox is a signal conditioner that generates trigger pulses for various sensors or input signals. When the sensor exceeds a threshold, the g.TRIGbox outputs a digital or analog signal which is recorded on the amplifier. A similar procedure is used for audio detection: the sound output from the PC is input to the g.TRIGbox, and when the volume exceeds threshold, a digital or analog signal is output and recorded by the amplifier. This information can be used because BCI2000 stores a time-stamp for every event during an experiment. The difference between a time-stamp value and the time that the recorded event changes values are compared to find the latency for that particular event.

These latencies can be affected by many external influences, including hardware deficiencies (such as low RAM, slow CPU speed, or a video card of low quality), and other software running in the background while the experiment is being run.

This document serves as the BCI2000Certification manual. As an example, this page will describe the certification procedure for the g.USBamp with visual and auditory stimuli triggered with the g.TRIGbox. At the end of the page, the certification results for using BCI2000 with three biosignal amplifiers (the g.USBamp, EGI GTEN 200, and Nihon Kohden JE120-A) are provided. A discussion and derivation of the specific system latencies are found here.

Video Demonstration

Load video
YouTube

Compiling the BCI2000Certification Programs

The BCI2000Certification procedure actually consists of two separate programs, including a graphical user interface that simplifies the task of selecting and starting tests, and a program that performs the data analysis and returns results. These programs are compiled automatically with the other BCI2000 programs. Compilation is only supported in BCI2000 v3.0 with the Visual Studio or MinGW compilers. Instructions can be found at Programming_Howto:Building_BCI2000. At the point where you start CMake, be sure to check BUILD TOOLS in the BUILD menu.

Check the BUILD TOOLS box to build the certification tester and analysis tool.

Using the BCI2000Certification Procedure

Starting the Tester User Interface

  1. The compiled programs can be found in BCI2000/tools/BCI2000Certification.
  2. Navigate to this folder, and double-click the BCI2000CertificationTester.exe file.
  3. The certification tool opens, as below.

This program shows a list of all of the procedures to be tested for the loaded configuration. Detailed instructions for using the interface follow.

The BCI2000Certification user interface.
The BCI2000Certification user interface.

Loading Configuration Files

The BCI2000 certification procedure uses a configuration file and different combinations of BCI2000 parameter fragments to control the tests. The configuration file (stored as an *.ini file) describes all of the certification tests. The parameter fragments store information about the amplifier and application module applied in each test. More detail about the contents of the configuration file can be found below. While it is possible to edit these by hand, in most circumstances it is simpler to use the graphical user interface provided.

  1. In the BCI2000Certification tool, select File->Open *.ini
  2. Navigate to the folder BCI2000/tools/BCI2000Certification.
  3. Load the file named BCI2000Certification_16ch_100ms.ini if using a 16-channel system. (This tutorial assumes a 16-channel system).
  4. If using more than 16 channels, then the amplifier parameter files must be modified.
    1. Navigate to BCI2000/tools/BCI2000Certification/parms/, and open the file Certification_gUSBamp_24.prm in Notepad.
    2. Find the parameter titled DeviceIDMaster, and replace the existing device ID (e.g., UA-2007.04.12 with the ID of your master device.
    3. Next, find the parameter titled DeviceIDs, and replace the IDs with the IDs of your amplifiers. The first in the list should be identical to the DeviceIDMaster value. For example, you might enter: DeviceIDs= 2 UA-2007.04.12 UA-2006.10.13 (the \#2 corresponds to the number of devices being used).
    4. Repeat these steps for the file Certification_gUSBamp_32.prm.
    5. Note that these steps are not necessary if only one g.USBamp is being used.
  5. All 100 tests in this configuration are loaded into the GUI.

User Interface Components

This section describes each user interface component, referring to the figure below.

The BCI2000Certification user interface components.

1. Configuration: Task List

The task list shows all tasks that have been loaded from the configuration file or created within the GUI. To run a specific test, it should have a checkmark next to it. In order to select or deselect all tasks, press the Select All check box at the bottom of the list.

It is possible to create new tasks, remove tasks, and copy tasks within the interface as well. To create a new task, press the + button. A new task is created at the bottom of the list; the task details must be filled in (described next). It is often easier to copy a similar task than to create a new one from scratch. To do so, select the desired task, and press Copy. The new task should be renamed appropriately. To remove a task, select a task and press the - button.

Creating Custom Tests

With the BCI2000Certification tool, you can test performance on a wide range of different parameters and applications. You can create a custom .ini file that will contain the name of each test, the desired signal processing module, sampling rate, block size, application, and parameter files. You will need to first create a parameter file for the test you wish to run. If you want to be able to edit the block size and sampling rate from the Certification tester GUI or its .ini file, you must omit those variables from your parameter files as they will overwrite any variables you set in the Certification Tester. For example, if you wanted to test an application that you have developed for BCI2000 using several sampling rates and block sizes you would first need to create the two parameter files for the application, and source with no sampling rate or block size specified, then export them to the /BCI2000Certification/parms/ folder. Once you have the two parameter files, you may link to them in a new .ini file located under the /BCI2000Certification/ directory and run them as described below.

2. Configuration: Task Details

The right side of the GUI displays the details of the currently selected task. Each option is described here.

Task Name
The name of the task. This must not contain any spaces, and must be unique.
Signal Source
The amplifier that should be used, if different from the amplifier set in the global options. Generally, this should not need to be used, since all tasks should use the same amplifier.
Signal Processing
The signal processing module that should be started and used for the given task. This should usually be either ARSignalProcessing, P3SignalProcessing, or DummySignalProcessing, depending on the specific application.
Application
The application that is run for the task. This will typically be CursorTask, P3Speller, or StimulusPresentation
Parameters
A list of the parameter fragments that should be used for this task. Typically, at least two parameter files are required. The first should be the amplifier-specific fragment that configures the amplifier for the correct number of channels. The second should be the application-specific fragment that configures how the application appears on the screen. Parameter files can be added and removed with the + and - buttons, respectively. All files should be placed in the BCI2000/tools/BCI2000Certification/parms folder.
Sample Rate
The sampling rate that should be used for this task. Note that it must be supported by the amplifier used.
Sample Block Size
The sample block size (in samples). In the provided parameter files, the sample block size is always equal to 100ms, regardless of the sampling rate.
Amp Channel
This is the analog channel that records the amplifier digital output (i.e., the digital output recorded back in on an analog channel). This signifies the onset of acquisition of a block. This value should be between 1 and the number of channels.
Amp Stream
This is the digital channel that records the amplifier digital output (i.e., the digital output recorded back in on a digital input channel).
Video Channel
The recorded analog channel containing video event markers.
Video Stream
The recorded digital input channel containing video event markers.
Video State
The BCI2000 state that contains the event markers for the video event, e.g., TargetCode or StimulusCode.
Video State Values
A list of state change values that should be detected by the analysis. For example, if values 4 11 are used, then the analysis tool will examine when the StimulusCode state changes to either 4 or 11, and then examine when the Video Channel changes.
Audio Channel
The recorded analog channel containing audio event markers.
Audio Stream

The recorded digital input channel containing audio event markers.

Audio State
The BCI2000 state that contains the event markers for the audio event, e.g., TargetCode or StimulusCode.
Audio State Values
A list of state change values that should be detected by the analysis. For example, if values 4 11 are used, then the analysis tool will examine when the StimulusCode state changes to either 4 or 11, and then examine when the Video Channel changes.

3. Control: Global Settings

The BCI2000Certification user interface.

As the name suggests, the options under Global Settings are applied to all tasks, and are described here.

Window Left, Top
The position of the left and top sides of the application window. If using multiple monitors, this position should take this into account. For example, if the application window is to the right of the main monitor, and the main monitor has a resolution of 1280x1024 pixels, then Window Left should be set to 1280, and Window Top to 0.
Window Width, Height
The width and height of the application window.
Global Signal Source
The executable that should be used by all applications. This value is overwritten by the Signal Source value set for individual tasks.
Data Output Directory
The directory where all of the data should be saved during the task. This will appear relative to BCI2000/tools/BCI2000Certification/, or at a given absolute path. NOTE that if data already exists in this directory, additional data will be saved to it! To avoid combining new data with earlier results, the existing directory with the same name should be renamed or deleted.
BCI2000 Directory
This is the directory containing the BCI2000 program files, e.g., C:\BCI2000\prog
Additional Application Module command line arguments
Additional arguments to add to the command line when starting the application module.
Shell execute before
Execute command in the shell before running the experiment (e.g., taskkill /IM explorer to terminate the explorer shell and taskbar).
Shell execute after
Execute command in the shell after the experiment has finished.

4. Control: Controls

This panel handles starting and stopping the procedure, and running the analysis.

Auto Set Config
Automatically press Set Config upon starting BCI2000. This is required for complete automation of the procedure. When unchecked, BCI2000 will pause after loading parameters, and you will be able to inspect parameters for debugging purposes. You will have to press Set Config manually to proceed.
Auto Start
Automatically start the test after Set Config is pressed. This is required for complete automation of the procedure.
Auto Quit
Automatically quit BCI2000 after each task. This is required for complete automation of the procedure.
Start
Starts the selected tasks.
Analyze
Starts the analysis tool. This uses the directory entered in the Data Output Directory setting. See the documentation for the analysis tool for more information on the analysis procedure.
Progress
While running, a progress bar appears and text provides information about the current running task, and how many tasks remain. The progress bar window includes a Cancel button, which will stop the procedure after the current task finishes.

Setting Up and Running the Certification Procedure

  1. Load the appropriate configuration file, based on the number of amplifiers that you will be using and as described above.
  2. Plug-in and turn on the amplifier(s).
  3. Configure the multi-monitor system, if applicable.
  4. Enter the appropriate screen dimensions under Window Width, etc.
  5. Prepare to place the optical detector on the monitor by using a ring of double-sided tape. The exact placement on the monitor will depend on the tasks used, and will be determined shortly. Plug the detector into the Optical Input A on the g.TRIGbox.
  6. Using a 1/8th in. cable, connect the headphone output on the PC to the Low Level input D of the g.TRIGbox.
  7. Connect the g.TRIGbox Trigger Out (D-Sub connector) to digital I/O 1 port of the g.USBamp using a standard cable provided by g.tec.
  8. Specify the Video Stream as DigitalInput1 and the Audio Stream as DigitalInput4 in the Events Configuration tab.
  9. Alternately, the audio and video event markers can be sampled by the analog channels. In this case you will need to specify the Video Channel and Audio Channel in the Events Configuration tab. In this case, leave the Video Stream and Audio Stream fields empty.
  10. In addition to measuring the latency of visual and auditory stimuli, you can also evaluate the delay of the analog-to-digital converter (ADC). This is done by connecting DigitalOutput1 of the Digital I/O 1 port of the amplifier to the amplifier's digital input or analog input. This requires a custom cable. For details, see the Assembling Required Cables section below. For the g.USBamp, specify the Amp Stream as DigitalInput0 or the Amp Channel as the input channel accordingly.
  11. Next, the location to place the optical detector will be determined. The first task will be run, showing the location on the screen to place the connector.
  12. Press the Start button. BCI2000 will start shortly, and the application window will appear. In this test, the P300 Speller application is shown first.
  13. Place the detector over the "X" on the screen.
  14. Adjust the threshold on the g.TRIGbox for connector A so that it blinks on and off with the appearance of the "X". It may be necessary to turn off the lights in the room so that ambient light does not interfere with the optical detector.
  15. When finished placing the detector and setting the threshold, press Cancel in the BCI2000Certification GUI, and then Quit BCI2000. Navigate to the location of the Data folder (e.g., BCI2000/tools/BCI2000Certification/) and delete the data folder.
  16. Make sure that the computer volume is turned all the way up, and not muted.To set the audio threshold, set audio volume on the computer to maximum. On Windows, to make this process easier in the future, go to the Start menu, click the Control Panel, and go to Sounds. In the Sound window, click the box that puts the volume icon in the task bar. A small speaker should now appear in the bottom-right of the screen in the task bar. Click this icon, and slide the volume up to the max; when you click on this slider, Windows emits a short beep to give an idea of the volume level, which can be used to set the threshold. To do so, make sure that the stereo audio cable is plugged into the g.TRIGbox, and click the slider to emit a sound, checking that the green LED is flashing when you click the slider. If it is not, turn the threshold knob for channel B counter-clockwise, and click the slider again. If it is still not working after several tries, unplug the audio cable from the computer, and check that you can hear the beep through the speakers. If you cannot, make sure that the sound is not muted, and that the sound drivers are properly installed. If you can hear it, plug the audio cable back into the computer audio output (NOT the microphone input), and make sure the g.TRIGbox has power; the light next to the power switch should flash momentarily if the battery levels are OK. You can also use a wall power source (see the g.TRIGbox manual for more information). The threshold is set correctly when the channel B LED is off at rest, and is on when a sound is played.
Connectors schematic.


  1. You are now ready to run all of the tests. To do so, press the Start button on the GUI.
  2. While testing, some tasks may not complete correctly, either because they are too computationally intensive for the selected computer system, or some other mistake or artifact occurred during the test. In this case, BCI2000 can be Quit manually, and the remaining tasks will be run. At the end of all tasks, you can then de-select all tasks, and re-check just the ones that need to be completed. It is necessary to delete data files in the data folder from unusable runs prior to running them again, so that the bad data is not included in the analysis.
  3. When all tasks are complete, press the Analyze button to analyze the collected data. This procedure is detailed later in this document.

Analyzing Data

The BCI2000 analysis tool.

Simply pressing the Analyze button in the Tester will start the Analysis program. It is also possible to run the program in the BCI2000Certification directory by starting BCI2000CertificationAnalysis.exe.

  1. The BCI2000CertificationTester program stores information about how to interpret a session in the session data file itself. From there, it is picked up by the analysis program. You don't have to provide BCI2000CertificationTester .ini files to the analysis program.
  2. The analysis program will open in a new GUI window. When you clicked "analyze" in the tester program, the analyzer's file list is already populated. Otherwise, appropriate folders and files must be entered for this session:
    1. Press the "+" button to select the folder containing your data files. When you select the folder, all data files contained within its subfolders are added automatically.
  3. Finally, set the directory for the output file. Typically, this should be the same folder that contains all of the data files.
  4. When the parameters have been correctly configured, press "Run" to start the analysis, which can take several minutes. Note that debug builds run very slowly, so you should always use a release build for analysis.
  5. When finished, the analysis tool will display results at several levels of detail. On top, a summary is displayed, then a list of requirements is shown, with a global result for each requirement. Further down, results are shown for individual files.
  6. To store results for documentation, you may press Save and Open... to save results to a text file, which can be found in the folder choosen at the bottom left, defaulting to the data folder. The file name will have the format results_(date)_(time).txt. Additionally, this contents of this file will be opened automatically at the end of the process in a new window.

Interpreting Analysis Results

The certification analysis results are shown as the overall result at the top of the analysis tool. In the example below, 33/60 tasks passed, while 27 tasks did not meet the success criteria. Additionally, 48 tasks are missing data, as they do not include auditory stimuli (this is by design). In more detail, the results are segregated into three sections, with overlapping information contained in each section. These sections are Requirements, Performance Statistics, and Individual Tasks. The Requirements section provides details about the global performance of the various certification tests. The success criteria are somewhat arbitrary and are based on our experience measuring response latency in various setups. The Performance Statistics section contains very similar information, with the statistics reported at the top level for easy viewing. Finally, the Individual Tasks section organizes the certification results by task.

The BCI2000 analysis results. Results are described in three sections: Requirements, Performance Statistics, and Individual Tasks.

Navigating into the sub-menu structure provides details about the various tests run, including the stimulus source, the method used to determine the event latency, and the event latency and variance across trials.

The BCI2000 analysis results. Individual Tasks.

The results text file contains detailed results for all tasks. A sample output for a single task is shown here:

The BCI2000 analysis results. The top of the file includes details about the global results.
The BCI2000 analysis results. Throughout the file there are details about each task.


A Note on "Pass/Fail"

The values that determine whether or not a task passes or fails are determined somewhat arbitrarily. For example, we determined that an audio output latency of more than 65 ms is unacceptable for psychophysical research, e.g., BCIs. In the example shown, the audio output latency was ~62 ms for all tasks with audio, and thus these tasks registered as a "pass." However, it is ultimately up to the researcher to determine what is acceptable system timing; this method provides a method of quantifying the system latency values to simplify the process of making these decisions.

You may move the slider near the top of the window in order to display the original GUI elements you saw before clicking "Run". When clicking the checkbox close to "Override requirements", a small editor window appears, containing code for all requirements applied to the data. You may modify requirements there, as you see fit.

Visualizing Certification Test Results using GNUPlot

If you have GNUPlot installed, you can also visualize the distribution of latencies. Simply right-click on the field and select plot.


GNU Plot.

Assembling Required Cables

Depending on what hardware and the latencies of interest are, you may or may not need to assemble custom cables. If you are interested in measuring the audio and visual latency with the gTRIGbox and an amplifier manufactured by a company other than g.tec, you will need to assemble a custom cable going from the output of the gTRIGbox to the digital inputs or analog inputs of your amplifier. It is also possible measure the audio and visual latency without the gTRIGbox, but this will require another trigger onset detection method. If you have a g.tec g.USBamp and want to measure the system latency and the ADC latency, you can follow the following guide.

In the following sections, we provide examples on how to build this cable if you’re using the gTRIGbox and a gUSBamp with a UB serial number (see the gUSBamp digital input page for more information regarding the difference pin assignments between serial numbers). There are two versions of the gTRIGbox, an older legacy version with four 3.5 mm audio connector outputs, and a newer one that has a 15 pin D-sub type connector. We describe the process of building cables for both of these devices.

Legacy gTRIGbox

Connectors schematic.

Amplifier Input

Parts List:

  • 3 Male Mono Audio Connectors
  • 6 12in+ Safety Connector Wires (to USB amplifier input), different colors
  • Soldering Iron and Solder

This section describes how to assemble cables that connect the output of the g.TRIGbox to the amplifier input channels.

  1. Strip the free ends of the 6 safety connector wires (~1/4 in)
  2. Take one wire, and solder the wire to the "signal" pad of one of the mono audio connectors.
  3. Solder another wire to the "ground" pad of the same connector.
  4. If necessary, pass the wires through end of the connector that screws in to cover the connection (see below, in the g.USBamp Amplifier Plug).
g.USBamp Amplifier Plug

Each connector now has a signal and ground wire that can be connector to the g.USBamp inputs, or any amplifier that accepts the standard EEG connector.

Digital I/O Cable

Parts List:

  • 3 Colored Electrical Wires
  • 7-Pin Digital Connector from g.tec
  • Soldering Iron and Solder
  • Silicone or electrical tape for insulation
  • 1 Female Mono Audio Plug

In order to access the digital I/O pins on the g.USBamp, a connector needs to be assembled. The 7-pin digital I/O connector should be ordered through g.TEC, the manufacturer of the g.USBamp.

At least three wires are needed for the circuit, which connect to the digital output 1 pin, the digital input 0 pin, and the ground pin.

g.USBamp Digital I/O Pins
g.USBamp Digital I/O Pins
  1. Strip the ends of the 3 wires (~1/4 in)
  2. Take one wire, and solder the wire to the "signal" pad of the female mono audio connectors.
  3. Solder another wire to the "ground" pad of the same connector.
  4. If necessary, pass the wires through end of the connector that screws in to cover the connection (see below, in the g.USBamp Amplifier Plug).
  5. Solder the ground wire to the center ground pin on the digital I/O connector.
  6. Locate the top of the connector by locating the small plastic notch, and locate pins 3 and 4 relative to the notch.
  7. Solder the signal wire from the mono connector to pin 3 (digital out 1)
  8. Finally, solder the remaining wire (it has not been connected to anything yet) between pin 3 and pin 4. This connects the digital output directly to the digital input.
  9. It may be helpful to insulate the wires from the casing using either a piece of black electrical tape, or an insulating gel, like silicone. If a gel is used, be sure to let it set for several hours or overnight. It is also probably a good idea to check the electrical connection with a multimeter prior to insulating.
  10. Next, attach the rubber sleeve over the end of the part with the female threads, as shown in the panel.
  11. Slide the smaller metal sleeve over the wires, then slide the larger rubber sleeve over the wires.
g.USBamp Digital Wire Plug
  1. For the next step, attach the two metallic pieces that surround the cables, shown laying to the sides in the figure below.
  2. One of the metal pieces has a small groove in it, which should be placed over the notch on the connector; the other metal piece fits on the opposite side of the connector.
  3. Push the small metal sleeve up to the side piece with the groove, and rotate the sleeve until the notch fits in the groove.
  4. Place the end piece over the connector, with the red dot on top and aligned with the connector notch.
  5. Finally, slide the rubber sleeve over the connector, and screw the sleeve into the end piece.
g.USBamp Digital I/O Cable

The plug is assembled next, similar to the. Solder the signal wire from Pin 3 (Digital Out 1) on the digital I/O connector to the signal connector on the female mono plug, and solder the ground wire to the outer ground connector on the plug. The below figure shows this; in this figure, the red wire is the signal wire, and the brown wire is the ground.

Latest gTRIGbox

There are two ways to connect the output of the gTRIGbox to the g.USBamp, “Analog” or “Digital”, as indicated in the figure below. Pins connected by a orange dotted line denote connections that are necessary to measure the amplifier latency.

Connectors schematic.
gTrigBox output schematic.

Digital Configuration

The digital configuration uses digital I/O 1 on the back of the g.tec amplifier. It requires a specific 7 pin g.tec connector that can be ordered from g.tec, the manufacturer of the g.USBamp. Parts list for digital cable:

  1. One 15 pin D-Sub Type Female Connector
  2. One 7 pin Digital Pin Connector from g.tec
  3. Electrical Wire
  4. Soldering Iron and Solder

The D-Sub Type connector can be purchased such that you only need to strip the electrical wires and then insert them to receptacles that tighten with a screwdriver, thus reducing the need for any soldering. This makes the process of assembling the cable easier. In example of this type of connector is shown below:

15 Pin D-Sub Connector with Breakout Board.

In the gUSB Digital Out cables we have seen, here is the coloring of the wires:

DIGITAL I/O CABLE
gTRIGbox Wire Color g.USBamp Digital I/O 1
Blue 7
White 1
Brown 2
Green 3
Yellow 4
Pink 5 or 6
Grey 5 or 6


Procedure for Assembling Cable
  1. Cut 3 pieces of wire, each the same length (about 3 ft should be sufficient).
  2. Strip the ends of these pieces of wire, leaving about a quarter-inch of wire exposed.
  3. Start with the D-Sub connector. Connect each of the pins in gTRIGbox Output column of the table above to its own piece of wire.
  4. Solder the other ends of each of these wires to its appropriate location, as defined in the table above, on the g.USBamp digital I/O connector.
  5. If you also want to measure amplifier latency, strip and solder a piece of wire between pins 1 and 5 on the g.USBamp digital I/O connector.
  6. When assembling the digital I/O connector, it may be helpful to insulate the wires from the casing using either a piece of black electrical tape or an insulating gel, like silicone. If a gel is used, be sure to let it set for several hours or overnight. It is also probably a good idea to check the electrical connection with a multimeter prior to insulating.
  7. Next, attach the rubber sleeve over the end of the part with the female threads, as shown in the panel.
  8. Slide the smaller metal sleeve over the wires, then slide the larger rubber sleeve over the wires.

Analog Configuration

The analog configuration uses the analog inputs on the front on the amplifier to measure latency. To also measure the amplifier latency you will need a specific g.tec connector. Parts list to make analog cable to measure system latency:

  1. One 15 pin D-Sub Type Female Connector
  2. Five Safety Connector Wires (to USB amplifier input)

If you also want to measure amplifier latency, you will also need:

  1. One 7 pin Digital Pin Connector from g.tec
  2. One Additional Safety Connector Wire (to USB amplifier input)
  3. Two resistors for Voltage Divider
  4. Soldering Iron and Solder

If you use the D-Sub connector with a breakout board and do not want to measure amplifier latency, you will not need to do any soldering, making assembly quite easy. Below is a table of what pin connections must be made. The wiring diagram above provides a guide for what pins need to be connected.

Procedure for Assembling Cable
  1. Strip the ends of five wires with safety connectors at one end, leaving a quarter-inch exposed (if you want to measure amplifier latency, strip one additional wire).
  2. Start with the D-Sub connector. Connect each of the pins as shown in the connector schematic to its own piece of safety connector wire.
  3. If you also want to measure amplifier latency, a voltage divider is necessary to bring the 5V output at the gUSBamp's digital output at pin 5 down to a voltage less than 250mV (the max input range on the gUSBamp's analog inputs). Selecting a R1 and R2 such that the following equation is satisfied would result in a 100mV signal at the analog inputs:

0.02=R2R1+R2

  1. Assemble this voltage divider by soldering the resistors, pin 5 on the digital I/O, and additional safety connector together as detailed in the connectors schematic. You can also use a bread board to avoid soldering.
  2. When assembling the digital I/O connector, it may be helpful to insulate the wires from the casing using either a piece of black electrical tape or an insulating gel, like silicone. If a gel is used, be sure to let it set for several hours or overnight. It is also probably a good idea to check the electrical connection with a multimeter prior to insulating.
  3. Next, attach the rubber sleeve over the end of the part with the female threads, as shown in the panel.
  4. Slide the smaller metal sleeve over the wires, then slide the larger rubber sleeve over the wires.

Troubleshooting

Things to watch out for:

  • Parameter files for gUSBamp with channels greater than 16 have pre-defined amplifier IDs. Please set these to your amplifier IDs.
  • BCI2000Certification runs BCI2000 applications from the working directory: C:\BCI2000\tools\BCI2000Certification. The images folder there doesn't contain all required images for CursorTask or a sounds folder for "StimulusPresentation". If necessary, paste the images and sounds folder from the BCI2000/prog directory into the BCI2000/tools/BCI2000Certification directory - overwriting what is currently in that directory.

See also

User Reference:Module Command Line Options, User Reference:Operator Module

Retrieved from "http://www.bci2000.org/mediawiki/index.php?title=User_Reference:BCI2000Certification&oldid=11563"