KeyStrokeFilter, AppConnector or BCI2000Remote?

Forum for software developers to discuss BCI2000 software development
Post Reply
pieterkubben
Posts: 22
Joined: 23 Jan 2012, 07:18

KeyStrokeFilter, AppConnector or BCI2000Remote?

Post by pieterkubben » 11 Jul 2012, 20:21

I developed an application to control a robot-arm and want to use EEG to send movement commands to the software. In this case, the command is simply a number (0-9) that corresponds to a particular movement direction.

Using BCI2000 for input leaves me 3 options, if I understand correctly:
(1) the KeyStrokeFilter(using the KeyStrokeStateName)
(2) AppConnector
(3) BCI2000Remote

If #1 is feasible it might be the most straightforward without the need for additional programming, but is it possible that way? Or do I misunderstand what it is meant for?

Regarding #2 and #3: I have been playing around with #2 but find working with the signals not intuitive (said otherwise: I don't understand it). Have to try #3 with these suggestions, after upgrading to BCI2000 3.0.5 (still on 3.0.4).

Any suggestions on the easiest / fastest way to send a key (0-9) to another application based on user actions (SMR is intended for this purpose).

Thanks,

Pieter

mellinger
Posts: 1065
Joined: 12 Feb 2003, 11:06

Re: KeyStrokeFilter, AppConnector or BCI2000Remote?

Post by mellinger » 12 Jul 2012, 08:46

ad 1)
The KeyStrokeFilter is exactly meant for such purposes, i.e. controlling an application outside of BCI2000 in a very generic way, via simulated key presses. However, in its present form, it has the drawback that the controlled application must have input focus. This means that the controlled application must be in the foreground, and process keypress events. If your robot arm software allows control via keypresses, then the KeyStrokeFilter approach should work. However, you must take care that no other windows come into the foreground during operation, so it is not a very robust solution. To make it more robust, you might modify the KeyStrokeFilter such that the user may specify a window title in a parameter, and the KeyStrokeFilter then sends keyboard events to that window rather than to the current foreground application. Such a modification would be straightforward to implement (using the Windows API FindWindow() and PostMessage() functions), and a welcome contribution to BCI2000.

ad 2) and 3)
In terms of signals and state variables, there is little difference between 2) and 3). BCI2000Remote allows you everything you can do via AppConnector, and a lot more, i.e. BCI2000Remote practically supersedes the AppConnector.
I have been playing around with #2 but find working with the signals not intuitive (said otherwise: I don't understand it).
From what you write regarding the KeyStrokeFilter, it seems that you have set up BCI2000 in such a way that you can specify an expression or state that goes into KeyStrokeStateName, and results in the desired robot arm command. When using BCI2000Remote, you have access to states and the control signal as well, so you may use this information to compute the robot arm command just like you would do it using the KeyStrokeFilter. Thus, I'm not sure whether I understand your remark regarding 2).

Regards,
Juergen

pieterkubben
Posts: 22
Joined: 23 Jan 2012, 07:18

Re: KeyStrokeFilter, AppConnector or BCI2000Remote?

Post by pieterkubben » 16 Jul 2012, 19:13

I suppose that the KeyStateFilter is the easiest way to start, get some more experience and later I can see how to improve the workflow and make it more error-proof. But how to start? I read the User Reference:States page, but do not really get it. I do not understand the DAT files when I open them in Notepad, nor what to type in the KeyStrokeStateName field in BCI2000's Application-tab (I suppose I should connect it to a State there, but how to instruct the system what keypress to perform).

I attached two screenshots of the AppConnector GUI with the state names I encounter when using the Emotiv device in combination with the Mu intro session. Can you give me an example on how to proceed, how to use them focusing on the KeyStrokeStateName field?

Suppose I want to associate "moving the mouse up" with sending key "0" to my robot arm control app, how should I start?

Many thanks,
Pieter
Attachments
AppConnStates02.png
AppConnStates01.png

mellinger
Posts: 1065
Joined: 12 Feb 2003, 11:06

Re: KeyStrokeFilter, AppConnector or BCI2000Remote?

Post by mellinger » 17 Jul 2012, 07:41

As a start, you might try this for KeyStrokeStateName:
(Signal(2,1)>0)?48:32

This is an expression (http://www.bci2000.org/wiki/index.php/U ... ion_Syntax) that evaluates to 48 (key '0') when the control signal is greater zero, and to 32 (space key) otherwise. The KeyPressFilter will thus send a series of '0' key presses while the cursor moves up, and a series of space key presses while the cursor moves down.

For a list of key codes, see
http://msdn.microsoft.com/en-us/library ... s.85).aspx

Regards,
Juergen

pieterkubben
Posts: 22
Joined: 23 Jan 2012, 07:18

Re: KeyStrokeFilter, AppConnector or BCI2000Remote?

Post by pieterkubben » 18 Jul 2012, 00:49

Okay, thanks - will try. Btw, I found this site on key codes more helpful. FYI...

How can I know what e.g. Signal(2,1) stands for? I understood that in Signal(A, B) the A-value might be related to a direction (X=0, Y-1, Z=2) but do not know whether that is correct (and what X=3 then means in my AppConnector output). Besides, how to know what the B-value means? E.g. why do you use Signal(2,1) for Cursor Up? And what do B=2 or B=3 mean then?

Thanks for your time and help!

Pieter

mellinger
Posts: 1065
Joined: 12 Feb 2003, 11:06

Re: KeyStrokeFilter, AppConnector or BCI2000Remote?

Post by mellinger » 18 Jul 2012, 08:50

The content of the control signal is defined by the LinearClassifier filter (http://www.bci2000.org/wiki/index.php/U ... Classifier). A signal's first index is called "channel", and its second index is called "element". The LinearClassifier does only write a single element into its output signal, so there is no element index other than "1".

Feedback-based application modules, such as the CursorTask or the FeedbackDemo, take channel index 1 to represent the x dimension, 2 to represent the y dimension, and 3 to represent the z dimension. In the tutorial, the brain signal is used to control the y dimension. Thus, Signal(2,1) is the only value that contains information about the brain signal. Also, the sign of that value determines the direction of cursor movement. For more information, please see the CursorTask documentation:
http://www.bci2000.org/wiki/index.php/U ... CursorTask

Regards,
Juergen

pieterkubben
Posts: 22
Joined: 23 Jan 2012, 07:18

Re: KeyStrokeFilter, AppConnector or BCI2000Remote?

Post by pieterkubben » 23 Jul 2012, 20:41

mellinger wrote:As a start, you might try this for KeyStrokeStateName:
(Signal(2,1)>0)?48:32
I tried using the Stimulus Presentation and Cursor Task paradigm, but get an error in the system log:

Code: Select all

KeystrokeFilter::Preflight: State "(Signal(2,1)>0)?48:32" is inaccessible.
Any suggestions?

Pieter

mellinger
Posts: 1065
Joined: 12 Feb 2003, 11:06

Re: KeyStrokeFilter, AppConnector or BCI2000Remote?

Post by mellinger » 24 Jul 2012, 11:00

Hi,

I'm sorry, I gave you a wrong advice because I incorrectly remembered parameterization of the KeystrokeFilter. However, I thought it would be useful if it worked that way, so I decided to add this functionality to the filter.

So, if you update to the current SVN version, and recompile, you may set the new parameter KeystrokeExpression to the value I suggested.

You might also be interested in checking the example on the Keystroke wiki page:
http://www.bci2000.org/wiki/index.php/U ... rokeFilter

Regards,
Juergen

pieterkubben
Posts: 22
Joined: 23 Jan 2012, 07:18

Re: KeyStrokeFilter, AppConnector or BCI2000Remote?

Post by pieterkubben » 28 Jul 2012, 21:44

mellinger wrote:So, if you update to the current SVN version, and recompile, you may set the new parameter KeystrokeExpression to the value I suggested.
Thanks a lot, will try after the weekend. To avoid having to compile all sources, what is the exact module I need to compile here?

Also thanks for expanding the wiki documentation, it's helpful!

Pieter

mellinger
Posts: 1065
Joined: 12 Feb 2003, 11:06

Re: KeyStrokeFilter, AppConnector or BCI2000Remote?

Post by mellinger » 31 Jul 2012, 09:00

Thanks a lot, will try after the weekend. To avoid having to compile all sources, what is the exact module I need to compile here?
You will need to recompile the application modules you are going use, which is CursorTask and DummyApplication.

-Juergen

pieterkubben
Posts: 22
Joined: 23 Jan 2012, 07:18

Re: KeyStrokeFilter, AppConnector or BCI2000Remote?

Post by pieterkubben » 01 Aug 2012, 15:50

mellinger wrote:You will need to recompile the application modules you are going use, which is CursorTask and DummyApplication.
All right... I downloaded the latest source code today from SVN, and found this folder (src > core > Application > Dummy) to be empty: just a CMakeLists (dated from June 2012) without any code files.

Did I misunderstand what you meant? Or is this folder supposed to be empty? (the KeyStrokeFilter code must be updated somewhere)

Thanks,
Pieter

mellinger
Posts: 1065
Joined: 12 Feb 2003, 11:06

Re: KeyStrokeFilter, AppConnector or BCI2000Remote?

Post by mellinger » 02 Aug 2012, 07:09

Information about how to build BCI2000 in its entirety, or parts of it, may be found here:
http://www.google.com/search?q=site%3Abci2000.org+build
http://www.bci2000.org/wiki/index.php/P ... tart_Guide
http://www.bci2000.org/wiki/index.php/P ... ild_System

In short, you need run CMake in order to create IDE project files, or makefiles. Then, you will be able to build a certain subproject.

Project folders for individual modules contain a CMake file, and additional code that is specific to that module. Due to its Dummy nature, the Dummy Application consists entirely of framework code, and has no specific code.

The KeystrokeFilter is part of the BCI2000 framework, and resides at BCI2000//src/shared/modules/application/human_interface_devices/KeystrokeFilter.

Regards,
Juergen

sunny
Posts: 4
Joined: 03 May 2013, 02:24

Re: KeyStrokeFilter, AppConnector or BCI2000Remote?

Post by sunny » 20 Dec 2014, 05:10

Hi,

i want to understand the concept of keystrokefilter inside any application. could you share how you designed how you developed it for your application?

Thanks

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest