VisuomotorJoystickTask
Synopsis
USBHIDJoystickTask is a BCI2000 application module that implements a center-out visuomotor reaching task controlled with a USB HID joystick. The task is designed to compare movements made under a normal joystick-to-cursor mapping with movements made under an inverted mapping.
During each trial, the participant moves a cursor to the center of the display, holds it there, waits through a warning period, and then moves to a peripheral target. In the automatic condition, joystick and cursor movements are congruent. In the controlled condition, joystick-to-cursor movement is inverted after the target appears.
The task is based on the visuomotor paradigm described by Neumann et al. (2018), which was developed to compare automatic and controlled movements and investigate their relationship to basal ganglia function and subthalamic deep brain stimulation in Parkinson's disease. The BCI2000 implementation adapts the original paradigm for USB joystick control and synchronized BCI2000 state recording.
Location
The source code for the Visuomotor Joystick Task is located in the BCI2000 private application directory:
http://www.bci2000.org/svn/trunk/src/private/Application/VisuomotorJoystickTask
The main application implementation is contained in:
USBHIDJoystickTask.cpp
with the corresponding header and build configuration in:
USBHIDJoystickTask.hCMakeLists.txt
Versioning
Author
Alexander Speer
Friedman Lab, Department of Neurosurgery
Washington University in St. Louis
Developed in the Friedman Lab.
Contact: speer@wustl.edu
Version History
The task was developed as a USB HID joystick implementation of the automatic/controlled visuomotor reaching paradigm described by Neumann et al. (2018).
The current implementation includes:
- Automatic and controlled visuomotor conditions.
- Ordered and pseudorandom experimental blocks.
- Configurable trial counts and trial timing.
- USB joystick input through BCI2000's input logging system.
- Selection of the participant display on multi-monitor Windows systems.
- BCI2000 state logging of task timing, cursor position, target position, condition, block structure, and trial events.
- A separate diagnostic window with trial information and manual trial reset.
- Trial-level reaction time, movement time, and total time measurements.
The files inspected in the current development checkout are located under src/custom and are not associated with SVN history in that checkout. Revision-specific history should therefore be updated once the contribution is placed in its final version-controlled location.
Source Code Revisions
- Initial development: NEEDS MANUAL INPUT
- Tested under: BCI2000 framework 3.6.9528.4, Release x64, MSVC 19.35
- Known to compile under: Visual Studio 2022 / MSVC 19.35 for a prior version of the current source
- Current source revision: NEEDS MANUAL INPUT
- Broken since: No known breaking revision
Functional Description
Scientific Basis
This task is a BCI2000 adaptation of the visuomotor task described by Neumann et al. (2018).
In the original experiment, participants controlled a cursor using a pen and digitizing tablet. Targets appeared around a central fixation point. In the automatic condition, movement of the pen and cursor was congruent. In the controlled condition, the pen-to-cursor mapping was inverted. Automatic targets were shown in green and controlled targets in red.
Trials in the original experiment began at a central fixation cross. Participants remained at the center for 3 seconds, after which a yellow warning cue appeared for 500 ms. The target appeared 500 ms after the warning cue. The original experiment used eight possible target positions arranged around the center and included both ordered condition blocks and a pseudorandom block.
The present BCI2000 implementation preserves the central features of this paradigm, including the automatic and inverted controlled mappings, green and red targets, 3-second default center hold, 500-ms warning period, 500-ms pre-target delay, and ordered and mixed experimental structures.
There are several implementation differences. The BCI2000 version uses a USB HID joystick rather than a digitizing tablet, allows the timing and number of trials to be configured, and generates targets over a continuous range of angles rather than restricting targets to eight positions. It also records task variables directly as synchronized BCI2000 states.
See References for the original study.
Experiment Structure
Four experiment modes are available:
| Value | Mode | Description |
|---|---|---|
| 1 | Automatic Only | Runs one block containing only automatic trials. |
| 2 | Controlled Only | Runs one block containing only controlled trials. |
| 3 | Ordered | Runs one automatic block and one controlled block. OrderedStartCondition determines which condition occurs first.
|
| 4 | Ordered + Pseudorandom | Runs an ordered automatic block, an ordered controlled block, and then a mixed pseudorandom block containing both conditions. |
In the mixed block, the application shuffles the configured numbers of automatic and controlled trials. The randomization attempts to avoid both strict alternation and excessively long runs of a single condition.
Trial Sequence
Each block begins with a 1.5-second announcement indicating whether the upcoming block is Automatic, Controlled, or Mixed.
A trial then proceeds as follows:
- The participant moves the cursor into the center region.
- Once centered, the participant must remain there for
CenterHoldSeconds. - A yellow warning marker appears for
WarningSeconds. - A pre-target interval lasting
PreTargetDelaySecondsfollows. - The target appears.
- The participant moves the cursor into the target.
- Target acquisition records the trial timing measurements and begins the feedback interval.
- After
FeedbackSeconds, the task advances to the next trial or block.
If the participant leaves the center during the center-hold, warning, or pre-target interval, the task returns to the waiting-for-center phase and the preparation sequence begins again.
A new target is generated when advancing to a new trial. Manually resetting a trial does not generate a new target.
Automatic Condition
During an automatic trial, cursor position follows the joystick using a direct mapping. Moving the joystick in one direction moves the cursor in the corresponding direction.
Automatic targets are displayed in green.
Controlled Condition
During a controlled trial, the normal direct mapping is initially used while the participant returns to and holds the center.
When the target appears, the current joystick position and cursor position are recorded as anchors. Cursor movement then becomes relative to those anchors and inverted. Moving the joystick away from its target-onset position causes the cursor to move in the opposite direction.
Conceptually:
cursor position = anchor cursor position - joystick displacement
The cursor is constrained to the participant display.
After completion of a controlled trial, the inverted mapping remains active until the joystick has returned close to the joystick position recorded at target onset. This prevents an abrupt cursor jump when returning from the controlled mapping to the normal mapping.
Controlled targets are displayed in red.
Joystick Input
Joystick input is provided through BCI2000's input logging system.
Joystick position states are expected to range from 0 through 32767. The application converts these values into internal task coordinates ranging from 0 through 1023.
The application reads:
JoystickXposJoystickYposJoystickButtons1
JoystickButtons1 is currently read by the task but is not used for trial progression or condition selection.
Joystick recording is normally enabled using the LogJoystick option. See LogJoystick for configuration details.
Target Generation
A new target angle is generated for each new trial.
The random target generator produces an integer between 0 and 3599. TargetAngle therefore represents tenths of a degree:
angle in degrees = TargetAngle / 10
This produces target angles from 0.0 through 359.9 degrees.
Targets are positioned at a fixed distance from the center using the generated angle.
Unlike the original Neumann et al. experiment, which used eight circularly arranged target locations, the BCI2000 implementation allows targets to appear over a continuous range of angles.
Trial Phases
The current phase of the task is recorded in TaskPhase.
| Value | Phase | Description |
|---|---|---|
| 0 | Block announcement | The upcoming block type is being displayed. |
| 1 | Waiting for center | The task is waiting for the cursor to enter the center region. |
| 2 | Holding center | The participant is completing the required center hold. |
| 3 | Warning | The yellow warning cue is visible. |
| 4 | Pre-target delay | The warning has ended and the task is waiting for target onset. |
| 5 | Target visible | The peripheral target is visible and movement may occur. |
| 6 | Feedback | The target has been acquired and the task is waiting before advancing. |
| 7 | Finished | All configured trials have been completed. |
A transition from TaskPhase=4 to TaskPhase=5 marks target onset.
Condition and Block Encodings
| State/Parameter | Value | Meaning |
|---|---|---|
TaskCondition
|
0 | Automatic |
TaskCondition
|
1 | Controlled |
TaskBlockType
|
0 | Ordered automatic block |
TaskBlockType
|
1 | Ordered controlled block |
TaskBlockType
|
2 | Pseudorandom mixed block |
OrderedStartCondition
|
0 | Automatic first |
OrderedStartCondition
|
1 | Controlled first |
Participant Display
The participant-facing ApplicationWindow contains:
- A light gray background.
- A black cursor.
- A black central fixation cross.
- A gray center marker.
- A larger yellow warning marker.
- A green target during automatic trials.
- A red target during controlled trials.
- A progress bar near the top of the display.
- Block announcements.
- An "ended" message when all trials are complete.
The StimulusDisplay parameter may be used to select the physical monitor on which the participant window appears.
Diagnostic Window
A separate diagnostic window is shown to the experimenter. It displays information including:
- Current condition and task phase.
- Current block and block type.
- Overall and condition-specific trial numbers.
- Target angle and coordinates.
- Raw joystick states and converted task coordinates.
- Cursor coordinates.
- Movement status.
- Reaction time, movement time, and total time.
- Completed-trial and reset counts.
- Mean timing measurements for automatic, controlled, and overall performance.
The diagnostic window also provides a button for manually resetting the current trial.
Trial Reset
A trial may be manually reset before it reaches the feedback phase.
Resetting a trial:
- Returns the task to the waiting-for-center phase.
- Clears the current movement and timing states.
- Generates a
TrialResetevent. - Increments the diagnostic reset count.
- Keeps the same trial number.
- Keeps the same target.
- Does not write a completed-trial row to the secondary CSV file.
A trial cannot be reset after it has already entered feedback.
Timing Measurements
The task calculates reaction time, movement time, and total time for each successfully completed trial.
Reaction time is the interval between target appearance and detected movement onset.
Movement time is the interval between detected movement onset and target acquisition.
Total time is the interval between target appearance and target acquisition.
Movement onset is detected when either joystick axis differs from the center-hold baseline by more than the task's joystick deadzone.
Timing is measured using a steady clock, but movement onset and target acquisition are detected during calls to Process(). The effective temporal resolution is therefore limited by the BCI2000 processing-block interval.
Data Recording
The task records its behavioral variables as BCI2000 states, allowing them to be synchronized with the normal BCI2000 .dat recording.
Joystick coordinates are also recorded when joystick input logging is enabled.
The current implementation additionally writes a secondary CSV file containing one row for each successfully completed trial:
trial,block,block_type,condition,reaction_time_ms,movement_time_ms,total_time_ms,success
Reset trials do not generate CSV rows.
Parameters
ExperimentMode
Selects the overall experimental structure.
Type: integer enumeration
Default: 1
Range: 1-4
Values:
1: Automatic only2: Controlled only3: Ordered automatic and controlled4: Ordered automatic, ordered controlled, and pseudorandom mixed
The application verifies during preflight that the trial-count parameters required by the selected mode are at least 1.
AutomaticTrials
Number of automatic trials used by experiment modes 1 and 3.
Type: integer
Default: 5
Declared range: 0-10000
When used by the selected experiment mode, the value must be at least 1.
ControlledTrials
Number of controlled trials used by experiment modes 2 and 3.
Type: integer
Default: 5
Declared range: 0-10000
When used by the selected experiment mode, the value must be at least 1.
OrderedStartCondition
Determines which condition is presented first in experiment mode 3.
Type: integer enumeration
Default: 0
Values:
0: Automatic first1: Controlled first
This parameter does not affect experiment modes 1, 2, or 4.
OrderedAutomaticTrials
Number of automatic trials in the ordered automatic block of experiment mode 4.
Type: integer
Default: 5
Declared range: 0-10000
When mode 4 is selected, the value must be at least 1.
OrderedControlledTrials
Number of controlled trials in the ordered controlled block of experiment mode 4.
Type: integer
Default: 5
Declared range: 0-10000
When mode 4 is selected, the value must be at least 1.
RandomAutomaticTrials
Number of automatic trials included in the pseudorandom block of experiment mode 4.
Type: integer
Default: 5
Declared range: 0-10000
When mode 4 is selected, the value must be at least 1.
RandomControlledTrials
Number of controlled trials included in the pseudorandom block of experiment mode 4.
Type: integer
Default: 5
Declared range: 0-10000
When mode 4 is selected, the value must be at least 1.
StimulusDisplay
Selects the physical display used for the participant-facing application window.
Type: integer enumeration
Default: 0
Values:
0: Use the normal/default ApplicationWindow placement.1and above: Select one of the displays detected when the parameters are published.
On Windows, available monitors are enumerated with their dimensions and primary-display status.
The application verifies during preflight that the selected display exists.
CenterHoldSeconds
Duration for which the participant must keep the cursor within the center region before the trial may continue.
Type: floating point
Default: 3.0 seconds
Range: 0.0-60.0 seconds
Leaving the center during this interval restarts the center-acquisition sequence.
WarningSeconds
Duration of the yellow warning cue.
Type: floating point
Default: 0.5 seconds
Range: 0.0-60.0 seconds
Leaving the center during the warning interval restarts the center-acquisition sequence.
PreTargetDelaySeconds
Delay between the end of the warning cue and target appearance.
Type: floating point
Default: 0.5 seconds
Range: 0.0-60.0 seconds
The participant must remain centered during this interval.
FeedbackSeconds
Interval between successful target acquisition and advancement to the next trial or block.
Type: floating point
Default: 0.5 seconds
Range: 0.0-60.0 seconds
ResultsFile
Specifies the file used for the secondary trial-level CSV output.
Type: string
Default:
block_visuomotor_results_usb_hid.csv
The file is opened when the application initializes and is overwritten if a file with the same name already exists. Initialization fails if the file cannot be created.
States
The following states are defined by USBHIDJoystickTask and are recorded with the normal BCI2000 data stream.
CursorX
10-bit state representing the current horizontal cursor coordinate.
Initial value: 512
Produced range: 0-1023
The state is updated continuously while the task runs.
CursorY
10-bit state representing the current vertical cursor coordinate.
Initial value: 512
Produced range: 0-1023
The state is updated continuously while the task runs.
TaskPhase
3-bit state containing the current phase of the task.
Initial value: 0
Values:
0: Block announcement1: Waiting for center2: Holding center3: Warning4: Pre-target delay5: Target visible6: Feedback7: Finished
A transition from 4 to 5 may be used to identify target onset during offline analysis.
TaskCondition
1-bit state identifying the current movement condition.
Initial value: 0
Values:
0: Automatic1: Controlled
TaskBlockType
2-bit state identifying the type of the current block.
Initial value: 0
Values:
0: Ordered automatic1: Ordered controlled2: Pseudorandom mixed
TargetAngle
12-bit state containing the target angle in tenths of a degree.
Initial value: 0
Generated range: 0-3599
To convert to degrees:
degrees = TargetAngle / 10
TargetX
10-bit state containing the horizontal coordinate of the current target.
Initial value: 512
Coordinate units correspond to the task's 0-through-1023 coordinate system.
TargetY
10-bit state containing the vertical coordinate of the current target.
Initial value: 512
Coordinate units correspond to the task's 0-through-1023 coordinate system.
TaskBlock
8-bit state containing the current one-based block number.
Initial value: 0
During a run, block numbering begins at 1.
TaskTrial
16-bit state containing the current one-based overall trial number.
Initial value: 0
Trial numbering continues across block boundaries. Manually resetting a trial does not increment TaskTrial.
MovementStarted
1-bit state indicating whether movement onset has been detected during the current trial.
Initial value: 0
The state changes to 1 when joystick displacement exceeds the movement deadzone after target onset. It remains high during the remainder of the movement and feedback period.
A rising transition from 0 to 1 may be used to identify movement onset.
TargetHit
1-bit event-like state indicating successful target acquisition.
Initial value: 0
The state is set to 1 when the cursor reaches the target and is cleared at the beginning of the following processing block. It therefore acts as a one-processing-block pulse.
TrialReset
1-bit event-like state indicating a manual trial reset.
Initial value: 0
The state is set to 1 when the experimenter successfully resets the current trial and is cleared at the beginning of the following processing block.
ReactionTimeMs
32-bit state containing reaction time in milliseconds.
Initial value: 0
Reaction time is measured from target appearance to detected movement onset. The value is assigned when the target is acquired and remains available through the feedback interval.
MovementTimeMs
32-bit state containing movement time in milliseconds.
Initial value: 0
Movement time is measured from detected movement onset to target acquisition.
TotalTimeMs
32-bit state containing total target-directed trial time in milliseconds.
Initial value: 0
Total time is measured from target appearance to target acquisition.
External Joystick States
The application also uses states provided by BCI2000's input logging system. These states are not defined by USBHIDJoystickTask itself.
| State | Description |
|---|---|
JoystickXpos
|
Horizontal joystick position. BCI2000 joystick input logging uses values from 0 through 32767. |
JoystickYpos
|
Vertical joystick position. BCI2000 joystick input logging uses values from 0 through 32767. |
JoystickButtons1
|
State of joystick button 1. The application reads this state but does not currently use it for task control. |
Running
|
Standard BCI2000 system state. The application sets it to 0 when all configured trials have been completed. |
For details about enabling joystick recording and the joystick states provided by BCI2000, see Logging Input.
Data Analysis Notes
The task states are intended to allow trial behavior to be reconstructed from the BCI2000 .dat file.
| Measurement/Event | State identification |
|---|---|
| Trial number | TaskTrial
|
| Block number | TaskBlock
|
| Automatic vs. controlled | TaskCondition
|
| Ordered vs. mixed block | TaskBlockType
|
| Target onset | Transition from TaskPhase=4 to TaskPhase=5
|
| Movement onset | Rising transition of MovementStarted
|
| Successful target acquisition | TargetHit=1
|
| Manual reset | TrialReset=1
|
| Reaction time | ReactionTimeMs
|
| Movement time | MovementTimeMs
|
| Total time | TotalTimeMs
|
| Cursor trajectory | CursorX and CursorY
|
| Target coordinates | TargetX and TargetY
|
| Target angle | TargetAngle
|
TargetHit and TrialReset are single-processing-block pulses. MovementStarted, in contrast, remains high after movement onset until the trial advances or is reset.
References
Neumann WJ, Schroll H, de Almeida Marcelino AL, Horn A, Ewert S, Irmen F, Krause P, Schneider GH, Hamker F, Kühn AA. Functional segregation of basal ganglia pathways in Parkinson's disease. Brain. 2018;141(9):2655-2669. doi:10.1093/brain/awy206.