Jump to content

Contributions:GazeMonitorFilter

From BCI2000 Wiki
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Synopsis

An application filter which greatly eases using the EyetrackerLogger extension which is capable of visualizing eye tracker states in real time, manages fixation enforcement, and provides functionality for re-alignment of the subject.

Location

http://www.bci2000.org/svn/trunk/src/contrib/Extensions/GazeMonitorFilter

Versioning

Authors

Griffin Milsap (griffin.milsap@gmail.com)

William Engelhardt (engelhardt@neurotechcenter.org)

Version History

  • 07/08/2011: Initial release
  • 07/02/2021: Addition of ChangeFixation and EyeLink eye tracker implementation
  • 02/17/2023: Use EnableGazeMonitorFilter flag to enable filter.

Source Code Revisions

  • Initial development: 3389
  • Tested under: 3389
  • Known to compile under: 6327
  • Broken since: --

Functional Description

GazeMonitorFilter is a "drop-in" application module extension which automatically performs tasks commonly associated with using an eye tracker. It has been designed for use with the EyetrackerLogger extension. When added into an application module, the GazeMonitorFilter is capable of drawing to the application window or in a layer on top of the application visualization screen.

Fixation Management

Even without eyetracker states, the GazeMonitorFilter is capable of drawing fixation images on the screen, but with Eyetracker<Left/Right>EyeGaze<X/Y> states, GazeMonitorFilter is capable of enforcing this fixation and setting the FixationViolated state which indicates violation of the fixation should it occur. The fixation is entirely parameterized and allows for detailed configuration of fixation management.

Visualization

GazeMonitorFilter is also capable of drawing a visualization of eye position (if it's logged) and gaze location (if it's logged) to the application window visualization screen. This enables the experimenter to monitor the eye tracker states visually in realtime for early detection (and later correction) of acquiring poor gaze data.

Subject Correction

With many eye trackers, accuracy diminishes as gaze diverts from the center of the screen. Even if the eye tracker is well calibrated, a sizable difference can exist between where the subject is actually looking and where the eye tracker gaze data reports the subject is looking. With Tobii eye trackers, this was seen more prominently as the subject's head changed position or orientation. GazeMonitorFilter provides functionality for giving the subject visual feedback and prompts which direct the subject to move back to the position they calibrated in.

If asked nicely, the GazeMonitorFilter may also brew you a cup of coffee.

Integration into BCI2000

Compile the extension into your application modules by enabling contributed extensions in your CMake configuration. You can do this by going into your root build folder and deleting CMakeCache.txt and re-running the project batch file, or by running cmake -i and enabling BUILD_GAZEMONITORFILTER.

In the batch file you run, append --EnableGazeMonitorFilter=1 to the Application module you are using, such as StimulusPresentation. Without this, the GazeMonitorFilter parameters will not be generated.

Parameters

The GazeMonitorFilter is configured in the Application tab within the GazeMonitor section. The configurable parameters are:

  • VisualizeGazeMonitorFilter - Enables/Disables visualization of eye tracker states on the application visualization window
  • EnforceFixation - Enables/Disables fixation management.
  • FixationX - Expression which defines requested fixation location X coordinate. (0.0 - 1.0)
  • FixationY - Expression which defines requested fixation location Y coordinate. (0.0 - 1.0)
  • BlinkTime - The amount of time allowed for which eye states reported by EyetrackerLogger can remain "invalid" before fixation has been violated.
  • SaccadeTime - The amount of time allowed for which valid gaze data reported by EyetrackerLogger can exist outside of the designated fixation before fixation has been violated.
  • FixationRadius - The maximum distance from the center of fixation (FixationX, FixationY) from which the gaze data can deviate without violating fixation.
  • FixationImage - The image displayed at the center of fixation while fixation is not violated.
  • FixationViolationImage - The image displayed at the center of fixation while fixation is violated.
  • FixationImageSize - The fixation image size in percentage of the screen height
  • FixationViolationSound - The sound played every time fixation has just been violated.
  • LogGazeInformation - Log information about eye validity and fixation enforcement to the ApplicationLog.
  • ChangeFixation - Allows for multiple fixations during one run.
  • Change Fixation parameter matrix. Any valid BCI2000 Expression can be used in the first column. If none of the expressions evaluate to true, then the Fixation center corresponds to the parameters FixationX and FixationY.
    Change Fixation parameter matrix. Any valid BCI2000 Expression can be used in the first column. If none of the expressions evaluate to true, then the Fixation center corresponds to the parameters FixationX and FixationY.

Notes

  • For FixationX and FixationY, the top left corner of the screen is (0.0, 0.0) and the bottom right corner of the screen is (1.0, 1.0)
  • Specifying large images for FixationImage and FixationViolationImage is not recommended as it may have a detrimental impact on overall system timing.
  • If either of the timing parameters (BlinkTime and SaccadeTime) trigger a fixation violation, fixation violation will not occur again when the other timing parameter triggers if fixation has not been acquired again. (Example: One second is allotted for BlinkTime and 4 seconds is allotted for SaccadeTime. Gaze moves outside of fixation and eyes close immediately thereafter and remain closed. FixationViolated is set after one second of the eyes being closed, and is not thrown again when the four seconds of gaze existing outside the allowed fixation radius elapse.)
  • A Fixation image can be drawn when there is no gaze data available to enforce the fixation. That said, FixationViolationImage, FixationViolationSound are not ever presented and the fixation timing and radius parameters remain unused due to lack of ability to determine if fixation has indeed been violated.
  • Real time visualization of gaze data on the application visualization screen is controlled by the VisualizeGazeMonitorFilter parameter declared by GazeMonitorFilter and the AppWindow<Spatial/Temporal>Decimation parameters declared by the ApplicationWindow class. Control the performance of your system by decimating how often and how large GazeMonitorFilter visualization updates are.

State Variables

  • FixationViolated - This state is set to "1" when the GazeMonitorFilter has determined that fixation is violated and is set back to 0 when fixation is acquired again.
  • GazeCorrectionMode - This state is monitored by GazeMonitorFilter. When this state is set to "1", GazeMonitorFilter will perform the gaze correction procedure. When the procedure has completed, the state is set by GazeMonitorFilter back to 0.
  • FixationDuration - 16-bit state that records the duration of the gaze being continuously fixated. It restarts from 0 every time the fixation is violated. Its units are the same as SourceTime.

Gaze Correction Mode

Gaze correction mode is initiated by setting the state GazeCorrectionMode to "1". GazeMonitorFilter will then perform the gaze correction procedure and set the state back to "0" when the procedure has concluded.

During the procedure, a prompt will be displayed to the subject on the application window at their current gaze location. The prompt position is "smoothed" to increase readability of the prompt and is not an accurate representation of where the subject's gaze is located at any particular frame.

Gaze Correction Procedure

  1. Subject is prompted to "Fixate". The subject will need to move his/her gaze into the fixation position/radius in order to continue in the procedure.
  2. Subject is prompted to move "Closer" to the screen or "Further" from the screen. The subject will need to move his/her eyes to between 550mm and 600mm away from the screen in order to continue in the procedure. Once the subject is fixated and at the correct distance from the screen, the fixation radius indicator will turn green.
  3. Subject must hold this position for four seconds. After four seconds (and a visual count down prompt) the procedure will be complete and the GazeCorrectionMode will be set back to "0"
  • Note -- Eye position data is displayed to the experimenter through the application visualization window through the use of the VisualizeGazeMonitorFilter parameter if and only if eye position data is being logged. If the subject calibrated with their eyes centered in the eye tracker's cameras, the experimenter can re-position the subject's head during this procedure based on the feedback provided by the application visualization window. This eye position data was not included explicitly as part of the gaze correction procedure for ease of use.

Integration of GazeMonitorFilter Capabilities

If you wish for your task to interact with the GazeMonitorFilter, you may need to make a few small source code modifications to your task.

Triggering Gaze Correction Mode from Within a Task

Integration of gaze correction mode into a task consists of setting the GazeCorrectionMode state to "1" if it exists, and waiting for the state to be set back to "0" before continuing with the run. This is done as follows:

  • Check for the existence of the GazeCorrectionMode state in Preflight()
void
MyTask::Preflight( const SignalProperties&, SignalProperties& ) const
{
  ...
  OptionalState( "GazeCorrectionMode" );
  ...
}
  • To turn on gaze correction mode
if( States->Exists( "GazeCorrectionMode" ) )
{
  State( "GazeCorrectionMode" ) = 1;
  // Disable drawing of your stimuli in here
}
  • To wait for gaze correction mode to complete
void
MyTask::Process( const GenericSignal&, GenericSignal& )
{
  ...
  if( !OptionalState( "GazeCorrectionMode", 0 ) )
  {
    // Update stimuli and continue performing task related things
  }
  ...
}

Triggering Gaze Correction Mode Manually

The BCI2000 Operator provides "function" buttons which can be scripted to modify the environment. The following changes are necessary to set one of the "function" buttons to modify the GazeCorrectionMode state in order to allow the experimenter to trigger this mode at will:

  • Open Operator.exe
  • File->Preferences
  • In the "Function Buttons" choose a button to modify and set the Name to "Correction" and the command to:
SET STATE GazeCorrectionMode 1

Reacting to Fixation Violation Within a Task

It may be the case that you want your task to react to a fixation violation event; for example, you may end a trial if fixation has been violated. To do this, make the following changes in your task's source code:

  • Check for the existence of the FixationViolated state in Preflight()
void
MyTask::Preflight( const SignalProperties&, SignalProperties& ) const
{
  ...
  OptionalState( "FixationViolated" );
  ...
}
  • Read the state this way wherever you need to react to it:
if( OptionalState( "FixationViolated", 0 ) )
{
  // React to the fixation being violated
}

See also

Contributions:EyetrackerLogger, Contributions:Extensions

Retrieved from "http://www.bci2000.org/mediawiki/index.php?title=Contributions:GazeMonitorFilter&oldid=10092"