Tobii EyeX

Forum for software developers to discuss BCI2000 software development
Post Reply
ageronimo
Posts: 24
Joined: 04 Dec 2012, 14:00

Tobii EyeX

Post by ageronimo » 15 Jun 2015, 14:40

Are there any resources for integration of the Tobii EyeX data into the BCI2000 platform for eye tracking purposes?

I saw that Tobii EyeX uses a different SDK than other eye trackers offered by the company, so the EyeTrackerLogger source module extension will not work.
http://www.bci2000.org/phpbb/viewtopic. ... obii#p7797

Would there possibly be a way to do it by accessing a the EyeX stream in Matlab?
http://developer.tobii.com/community/fo ... in-matlab/

Thanks for your insight!
-Andrew

pbrunner
Posts: 344
Joined: 17 Sep 2010, 12:43

Re: Tobii EyeX

Post by pbrunner » 16 Jun 2015, 15:18

Andrew,

the EyeTrackerLogger in BCI2000 currently is based on Tobii Analytics SDK v2.0 which supports Tobii X60/120, Tobii T60/120, Tobii T60 XL, Tobii TX300 and the Tobii X1/X2 devices.

For the Tobii EyeX to work in BCI2000 you would have to implement a new logger that utilizes the the Tobii EyeX SDK for C/C++. If you are savvy in C/C++ you could implement this logger in a day or two. In fact, Tobii provides a nice example online that provides the minimal functionality needed [1]. The BCI2000 wiki also provides info on how to implement a logger [2]. If this is a feasible option for you, I would like to suggest, that you get the minimal example to work on your Windows PC and then touch base again about writing the logger in BCI2000. I can walk you through this process.

The alternative solution would be to use the AppConnector to stream data from MATLAB into BCI2000 states [3].

In my opinion the cleanest and best solution is to write the BCI2000 logger for the Tobii EyeX SDK, as this would allow full integration into BCI2000, could utilize the GazeMonitorFilter to visualize the eye gaze on a copy of the user window, and would after checking it into the BCI2000 source code be available to other BCI2000 users.

Regards, Peter

[1] http://developer.tobii.com/c-sample-wal ... atastream/
[2] http://www.bci2000.org/wiki/index.php/P ... put_Logger
[3] http://www.bci2000.org/wiki/index.php/T ... _Connector

ageronimo
Posts: 24
Joined: 04 Dec 2012, 14:00

Re: Tobii EyeX

Post by ageronimo » 18 Jun 2015, 12:29

Hi Peter,
Thanks for the great suggestions. I will look into implementing a logger, although I am by no means savvy at C++. Ill be in touch with whatever progress I make.

ageronimo
Posts: 24
Joined: 04 Dec 2012, 14:00

Re: Tobii EyeX

Post by ageronimo » 18 Jun 2015, 17:16

Hi Peter,

I was able to compile the example gaze detector that Tobii provides in its SDK (MinimalGazeDataStream.c) in VC2008. I do think i will need help creating a BCI2000 logger from this. I tried to look at the EyeTrackerLogger files to get an idea of how this might work, but its complexity is a bit overwhelming, and not nearly as straightforward as the Logger programming tutorial on the BCI2000 wiki.

I will look into it more tomorrow (ive only skimmed the first two links you sent me), but further direction on how to transform MinimalGazeDataStream into an Extension like EyeTrackerLogger would be very helpful.
Thanks

ageronimo
Posts: 24
Joined: 04 Dec 2012, 14:00

Re: Tobii EyeX

Post by ageronimo » 23 Jun 2015, 10:16

Hi Peter,

I've now tried using the AppConnector method somewhat successfully. I was able to control the cursor for a CursorTask application using a modified version of this Matlab code found at http://www.ucl.ac.uk/~smgxprj/resources.html (Tobii EyeX Binding)

I compiled the myex.c file in section 1 of myex_v2_1.m, and modified the sections 1.1 - 3 so that they worked for me. (See my attached modified Matlab file myex_v2_AG.m)

While running section 2 of that file, i use the BCI2000 launcher to run SignalGenerator + DummySignalProcessing + CursorTask. In configuration, I set the amplitude of the generated signal and noise to 0mV, ConnectorInputFilter = "*", and ConnectorInputAddress = "localhost:20320". I positioned the BCI2000 application window fullscreen on my second monitor, where my EyeX is installed. With both BCI2000 and this Matlab script running, I am able to control the movement of the cursor with the EyeX interface.

I don't like this solution, because I have to be running both Matlab and BCI2000 at the same time, and Matlab occasionally crashes (??). Also, I haven't figured out how to save this input as its own state. Right now it is being input as Signal(0,0) and Signal(1,0), which get mapped to the cursor x and y positions and saved as those states. In what module can I create a state for this variable to be saved in the .dat file? This would be helpful, but I agree with you that it would be preferrable to use the logger option, but I'm not sure where to begin with that method.

Code: Select all

%myex_v2_AG.m
%% 1. compile
%
% Instructions for compiling "myex.c"
% -------------------------------
% Compiling is needed to turn "myex.c" into "myex.mexw32" (or myex.mexw64).
% When compiling, the compiler needs to be able to see the .dll and .lib
% file, and the .h files contained within "./eyex/". When running, the .mex
% file will still need to be able to see the .dll and .lib file (e.g., put
% them in the same local directory).
%
% - Compiling only needs to be done once, on first usage.
% - Must be run in a directory containing:
%       ./eyex (subdirectory containing EyeX.h, EyeXActions.h, etc.)
%       myex.c
%       Tobii.EyeX.Client.dll
%       Tobii.EyeX.Client.lib
% - Note that the .dll and .lib file are found inside the Tobii EyeX SDK
%   E.g., inside: TobiiEyeXSdk-Cpp-0.23.325\lib\x64
%       - Remember that you must use the appropriate .dll/.lib files (x64
%         if compiling for 64bit Matlab, x86 if compiling for 32bit Matlab
%         [even on a  64bit machine])
%       - Remember that the EyeX SDK is not the same as the Tobii SDK for
%         their other ('research grade') eye-trackers.
%
% 32 vs 64 bit
% -------------------------------
% - Note for 32-bit Matlab users:
%       This compiler *did* work: Microsoft Software Development Kit (SDK) 7.1 in C:\Program Files (x86)\Microsoft Visual Studio 10.0
%       The default compiler did *not* work: Lcc-win32 C 2.4.1 in C:\PROGRA~2\MATLAB\R2012b\sys\lcc 
%       (this is because the lcc compiler does not permit variable definition/initialisation on same line)
%       - You can change compiler using mex -setup
%       - You can download the visual studio compiler as part of the
%         Microsoft .Net dev kit (if my memory serves)
% - Note for 64-bit Matlab users:
%       If when you compile you get an error like this:
%           myex.obj : error LNK2019: unresolved external symbol __imp_txFormatObjectAsText referenced in function __txDbgObject 
%       Then you are trying to compile against the 32bit .dll/.lib files.
%       Replace these with the appropriate versions from the x64 SDK
%       directory (see above).
%
% Tobii EyeX engine number
% -------------------------------
% Both the Tobii EyeX Engine and the SDK are regularly updated. This can
% lead to various errors. For example, if you update the EyeX Engine, you
% may start to get this error:
%
%   The connection state is now SERVER_VERSION_TOO_HIGH: this application requires an older version of the EyeX Engine to run.
%
% Or if you try to compile "myex.c" against newer/older-than-expected
% versions of the SDK, you may get various "undeclared identifier errors".
% For example, between SDK_0.23 and SDK_1.3 the following things changed:
%
%   All variables starting "TX_INTERACTIONBEHAVIORTYPE_" now start "TX_BEHAVIORTYPE_"
%   "txInitializeSystem" => "txInitializeEyeX"
%   "TX_SYSTEMCOMPONENTOVERRIDEFLAG_NONE" => "TX_EYEXCOMPONENTOVERRIDEFLAG_NONE"
%
% In the "__precompiled_versions/" directory I include versions compiled
% using the following setups:
%
%   Tobii EyeX Engine (0.8.17.1196), Tobii EyeX Cpp SDK (0.23.325)
%   Tobii EyeX Engine (1.2.0.4583),  Tobii EyeX Cpp SDK (1.3.443)
%
% But as Tobii update their software, you may have to update "myex.c" and
% recompile it accordingly


% compile - only need to do this once
mex myex.c % compile to generate myex.mexw32 / myex.mexw64
% WaitSecs(.1);


%% 1.1 Preliminaries
%For cursor control, where the application window is fullscreen on a 
%second monitor, need to specify appropriate dimensions
MonitorNum = 2; %Which monitor being used
cursorgain = 5; %To modulate the speed of the cursor
yoff = -282;  %My second monitor has an offset of 282px down
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
monloc = get(0,'MonitorPositions');
xmin = monloc(MonitorNum,1); xwid = monloc(MonitorNum,3);
xmean = xmin+xwid/2;
ymin = monloc(MonitorNum,2)+yoff; yhig = monloc(MonitorNum,4);
ymean = ymin+yhig/2;


%% 2. run
% This requires Psychtoolbox to run: http://psychtoolbox.org/ 
%%%%%%%%%%%%%%%%%%%%%%%%%%%% AG 6/22/15 -- the functions WaitSecs and 
%KbCheck require this toolbox.  I dont believe they are necessary.
%Psychtoolbox not required for this modification.

% connect to EyeX Engine
myex('connect')

% clear any data in buffer
myex('getdata');


%Open UDP port for writing
if exist('u')
% echoudp('off')
fclose(u)
end
% echoudp('on',20320)
u = udp('127.0.0.1',20320);
fopen(u);
lastsave = [0 0];

% allow to track until key press
x_all = [];
% while ~KbCheck();
while size(x_all,1)<300
    x = myex('getdata');
    if ~isempty(x)
        % display distance
        z_mm = x(end,[8 11]);
        isvalid = x(end,[4 5])==1;
        isvalid = isvalid & (z_mm>0.001); % defensive (shouldn't be necessary)
        z_mm = z_mm(isvalid);
        z_mm = nanmean(z_mm);
        % add to store
        x_all = [x_all; x(end,:)]; %#ok<AGROW> This is innefficient memory-allocation, but ok for present purposes
    end
    
    
    if x(end,1)==0
        savedat = lastsave;
    else
        savedat = x(end,1:2);
    end

    fprintf(u,'%s\n',['Signal(0,0) ' num2str(cursorgain*(savedat(1)-xmean)/xwid)])
    fprintf(u,'%s\n',['Signal(1,0) ' num2str(-cursorgain*(savedat(2)-ymean)/yhig)])
    
    fprintf('%s\n',['Signal(0,0) ' num2str(cursorgain*(savedat(1)-xmean)/xwid)])
    fprintf('%s\n',['Signal(1,0) ' num2str(-cursorgain*(savedat(2)-ymean)/yhig)])

    lastsave = savedat;
    pause(.1)
%     WaitSecs(1/10);
end

% disconnect from EyeX Engine
% WaitSecs(.1);
myex('disconnect')

% echoudp('off')
fclose(u)
%Invert the second value of x_all (this is the y-position)
x_all(:,2) = -x_all(:,2);

 
%% 3. show results
close all
subplot(211); plot(x_all(:,1:2));
subplot(212); scatter(x_all(:,1),x_all(:,2),[],parula(size(x_all,1)));
monloc = get(groot,'MonitorPositions');
axis equal;
axis([xmin xmin+xwid -ymin-yhig -ymin])

pbrunner
Posts: 344
Joined: 17 Sep 2010, 12:43

Re: Tobii EyeX

Post by pbrunner » 23 Jun 2015, 16:22

Andrew,

I have put one of our summer students on implementing a logger for the basic eye gaze functionality in the TobiiEyeXSdk. This should provide a native solution for BCI2000. However as we don't have a Tobii EyeX device, I would need your help in testing and debugging the logger. Would you be willing to help with this?

Regards, Peter

ageronimo
Posts: 24
Joined: 04 Dec 2012, 14:00

Re: Tobii EyeX

Post by ageronimo » 24 Jun 2015, 09:04

Sure, no problem. Let me know what I can do to help.

pbrunner
Posts: 344
Joined: 17 Sep 2010, 12:43

Re: Tobii EyeX

Post by pbrunner » 16 Jul 2015, 11:32

The issue has been resolved by creating a BCI2000 logger that acquires eye-gaze from Tobii EyeX devices. The BCI2000 EyetrackerLoggerTobiiX extension is described on the BCI2000 wiki:

http://www.bci2000.org/wiki/index.php/C ... ggerTobiiX

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests