Creating my own task application

[pjercic]

Hi everyone.

• I would like to create my own task application exe file, to run inside of the BCI2000, can you please guide me to the guide/tutorial/wiki/post explaining this. I got lost regarding this matter.
• Is it possible to run already a compiled exe application task (a small game made with Unity 3D) inside of the BCI2000, and if so how to do this? What interface does my game have to implement to support this?

//Petar

p.s. I have to say that I was pleasantly surprised with the speed I was able to connect to BioSemi system and get the cursor task and analysis running, excellent work on guides and tutorials!!!

[mellinger]

Hi Petar,

thanks for your feedback.

I guess you want to control a game using the SMR (mu rhythm) paradigm rather than an ERP/P300 paradigm. For this, you will use BCI2000's DummyApplication as an application module, and connect externally to the BCI2000 system.

In order to connect BCI2000 to an existing game, you might use the BCI2000Remote class which allows you to fully control BCI2000, and receive data from BCI2000. This is the best option if you are writing in C++.
This class has also been wrapped into a C library called BCI2000RemoteLib, for which bindings exist for Python, COM/OLE/ActiveX/Automation on Windows, and command line scripts on all supported platforms. With a little effort, you can also create bindings for C# (and we would be happy if you shared them with us).
http://www.google.com/search?q=site:bci ... 2000Remote

If none of these options are available to you, you can remotely control BCI2000 via the Operator module's telnet interface over TCP. This allows you to open a session, issue commands, and receive command results. This gives you all the options of the BCI2000Remote interface, but is a little more cumbersome to program because you need to compose commands, and parse results, on your own.
http://www.google.com/search?q=site:bci ... ine+telnet
http://www.google.com/search?q=site:bci ... +scripting

Finally, you can use the older AppConnector interface which allows communication over UDP ports. AppConnector messages are very easy to compose and parse, but the AppConnecter interface does not allow you to fully control BCI2000 from your game.
http://www.google.com/search?q=site:bci ... pConnector

Please let me know if you have any more questions.

Regards,
Juergen

[pjercic]

Thank you for helping me out on this one.

I wasn't able to find any relevant leads on using/exploting DummyApplication component, please can you point me to it since this seams like the best way to integrate an already existing game into the BCI2000. I HAVE found your quote on the BB (other thread) saying:
"It is not possible to start BCI2000 without an application module. However, you may easily create a dummy application module that only contains the two AppConnector filters. "
- mellinger
Does that mean I have to program my own DummyApplication using the one given as a template or what? I am kind of lost here o_O

On the other hand, since BCI2000AutomationLib has a .NET component, it was easy to reproduce the Visual Basic code in C# on using the BCI2000Remote class posted in http://www.bci2000.org/wiki/index.php/C ... Automation

Here's the code

Code: Select all

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using BCI2000AutomationLib;

namespace bci2000
{
    class bci2000_controller
    {
        // Instantiate a BCI2000Remote object
        BCI2000Remote bci = new BCI2000Remote();

        public void TestConnect() {

            bool ok_conn = false;

            // Start the Operator module, and connect to it
            ok_conn = bci.Connect();

            //If Not ok Then MsgBox bci.Result
            if(!ok_conn) Console.WriteLine(bci.Result);

            // Startup modules
            string[] modules = new string[3];
            modules[0] = "SignalGenerator --local";
            modules[1] = "DummySignalProcessing --local";
            modules[2] = "FeedbackDemo --local";
            ok_conn = bci.StartupModules(modules);

            //If Not ok Then MsgBox bci.Result
            if (!ok_conn) Console.WriteLine(bci.Result);

            // Load a parameter file, and set subject information
            ok_conn = bci.LoadParametersRemote("../parms/examples/CursorTask_SignalGenerator.prm");

            //If Not ok Then MsgBox bci.Result
            if (!ok_conn) Console.WriteLine(bci.Result);
            bci.SubjectID = "SUB";

            // Start a run
            ok_conn = bci.Start();

            // Display the feedback signal while BCI2000 is running
            string state = "Running";
            double value = 0;
            while (ok_conn && state=="Running") {
                ok_conn = bci.GetControlSignal(1, 1, value);
                Console.WriteLine(value);
                System.Threading.Thread.Sleep(500);
                ok_conn = bci.GetSystemState(state);
            }

            bci.Stop();

            // Shut down BCI2000
            bci.Disconnect();

            // Release and delete the BCI2000Remote object
            bci = null;
        }
    }
}

But I have a problem when trying to run the first simple demo from the tutorial.
The actual problem is that i get only one value from the system before

Code: Select all

ok_conn = bci.GetSystemState(state);

returns "Running" as a state of the system, but the returned bool value is false. My guess is that some module shut down???
Am I missing something here. I double checked against the remote class C++ code posted in

Code: Select all

http://www.bci2000.org/wiki/index.php/Programming_Reference:BCI2000Remote_Class

but I made all of the necessary steps.

//Petar

[mellinger]

Hi,

you need not create a DummyApplication module. It is now a standard part of BCI2000.

Regarding the return value of BCI2000Remote::GetSystemState(), you hit a bug. That function would always return false.
There was another bug in BCI2000Automation: It was not possible to change the Operator window's visibility by setting the BCI2000Remote::WindowVisible property.

I fixed both bugs, so you will get rid of them by updating your source code to the latest SVN version, and recompile BCI2000Automation.

If you don't want to recompile, use these workarounds:
*Don't test the return value of GetSystemState(),
*To show or hide the Operator window, use

Code: Select all

bci.Execute( "Show window" );
bci.Execute( "Hide window" );

In your example code, you should make the window visible after bci.Connect(). Otherwise, you will not be able to suspend BCI2000, and your code will run forever.

Regards,
Juergen

[pjercic]

Hi mellinger.

I still don't know how to use/exploit the DummyApplication module. Is there some kind of a guide/wiki about it.
After clicking on a help button in the Configuration>Application tab i get to the http://www.bci2000.org/wiki/index.php/U ... rokeFilter
which explains about the keystroke filter and how are keystrokes simulated using BCI2000 states. If I got correctly how this module works?
Now a question is how do these keystrokes reach my application (game), do you have to specify an .exe somewhere or just have it running to accept any keystroke.

On the other matter, I implemented your bug workaround notes and the application works like a charm. Now I am thinking, since the game is implemented in C#, to implement BCI2000 inside of the game and have the control signal value accessed inside of the game. I will keep you posted on this.

Thanks for all the help. I am in your care,

//Petar

[mellinger]

The DummyApplication module is just an application module that does nothing.
In order to use BCI2000 with your game application, you might proceed as follows:
1) Follow the mu rhythm tutorial until you achieved reliable control in 1D.
2) A normalizer parameter file will have been saved in the same folder as the data file.
3) From your game, start up BCI2000, using the same source and signal processing modules, but the DummyApplication module.
4) From your game, load the Normalizer parameters mentioned before, then set the Adaptation parameter to a list of three zeros.
5) Run your game with BCI control. Expect the control signal to have zero mean and unit variance, i.e. expect it to range mainly between -0.7 and +0.7, though it may leave that range temporarily.
6) After some time, the original adaptation will be valid no longer. You need to run a CursorTask feedback session again, as is part of the mu rhythm tutorial. Then, proceed from 2).

HTH,
Juergen

[pjercic]

I will try and follow your guidance slowly. I actually got the BCI2000 starting up inside of the game, a promising start.
The first issue is that the previously posted code was compiled in Visual Studio 2010 and it works perfectly, now porting to a game engine (e.g. Unity 3D) requires some stricter C# code since it uses Mono C# compiler. So I had to change the following 3 lines:
From:

Code: Select all

ok_conn = bci.StartupModules(modules);
ok_conn = bci.GetControlSignal(1, 1, value);
bci.GetSystemState(state);

To:

Code: Select all

ok_conn = bci.StartupModules(ref modules);
ok_conn = bci.GetControlSignal(1, 1, ref value);
bci.GetSystemState(ref state);

Now the error concerning the first line

Code: Select all

ok_conn = bci.StartupModules(ref modules);

appears, regarding array conversion. More specifically:
The best overloaded method match for 'BCI2000AutomationLib.IBCI2000Remote.StartupModules(ref System.Array)' has some invalid arguments
Argument 1: cannot convert from 'ref string[]' to 'ref System.Array'

Can you give an advice on how to rewrite this code block to a more strict coding to resolve the stated error:

Code: Select all

// Startup modules
string[] modules = new string[3];
modules[0] = "SignalGenerator --local";
modules[1] = "DummySignalProcessing --local";
modules[2] = "DummyApplication --local";

ok_conn = bci.StartupModules(ref modules);

//Petar

[pjercic]

Here's further input on this issue. I managed to rewrite the piece of code creating the compilation error. So now the same code looks like this:

Code: Select all

// Startup modules 
Array modules = new string[3] 
{
    "SignalGenerator --local",
    "DummySignalProcessing --local",
    "DummyApplication --local"
};
ok_conn = bci.StartupModules(ref modules); 

And it compiles in VS2010 and Mono C# compilers, weeeee.

Now I hit another problem, when I import that same code to the Unity 3D game engine, I get the following runtime error (the Operator module is displayed, but no other modules comes alive and the system is in Idle mode):

ArgumentException: Value does not fall within the expected range.
System.Runtime.InteropServices.Marshal.ThrowExceptionForHR (Int32 errorCode)
(wrapper cominterop) BCI2000AutomationLib.BCI2000RemoteClass:StartupModules (System.Array&)
(wrapper cominterop-invoke) BCI2000AutomationLib.BCI2000RemoteClass:StartupModules (System.Array&)

Plese can you advise me on this error or explain it, I want to know how to proceed on this matter forward?

[mellinger]

Congratulations that you made it that far.

Unfortunately, the combination of Mono and COM is outside my area of expertise, but it seems that you need to manually convert the strings to BSTRings as expected by the COM object. From the Mono-COM-interop page (http://www.mono-project.com/COM_Interop), it seems that there exists a function for this purpose:

Code: Select all

Marshal::StringToBSTR (string s);

I guess you just need to call this function on all the elements in the array.

HTH,
Juergen

[eman]

Hi,,

I'm pretty new into BCI2000 I am developing an application using C# that will call p300 matrix which will display bitmap pictures instead of letters. I manage to run the P300 matrix using the above code and get brain signals using emotiv headset. I have two questions here:

1- is there any way to get target selected picture using BCI2000Remoot object?
2- how to stop P300(time out) after first selection?

any help will be appreciated.

[mellinger]

Hi,

1- is there any way to get target selected picture using BCI2000Remoot object?
2- how to stop P300(time out) after first selection?

this can be done by reading and writing state variables through BCI2000Remote.

For details, check out description of the following state variables on the P3SpellerTask wiki page:
SelectedRow
SelectedColumn
SelectedTarget
PauseApplication

Regards,
Juergen

[eman]

unfortunately, always I get the value= zero. I assume it didn't get any value. this is the code:

Code: Select all

 // Instantiate a BCI2000Remote object
        BCI2000Remote bci = new BCI2000Remote();

            bool ok_conn = false;

            // Start the Operator module, and connect to it
            ok_conn = bci.Connect();

            //If Not ok Then MsgBox bci.Result
            if (!ok_conn)
                Console.WriteLine(bci.Result);
            // Startup modules
            string[] modules = new string[3];
            modules[0] = "Emotiv --local";
            modules[1] = "P3SignalProcessing --local";
            modules[2] = "P3Speller --local";
            ok_conn = bci.StartupModules(modules);

            //show operator module 
            bci.Execute("Show window");

            //If Not ok Then MsgBox bci.Result
            if (!ok_conn)
                Console.WriteLine(bci.Result);

        
            // Load a parameter file, and set subject information
            ok_conn = bci.LoadParametersRemote("C:\\BCI2000\\parms\\p3_tutorial\\AA_test.prm");

            //If Not ok Then MsgBox bci.Result
            if (!ok_conn)
                Console.WriteLine(bci.Result);
            bci.SubjectID = "SUB";

            // Start a run
            ok_conn = bci.Start();


            // Display the feedback signal while BCI2000 is running
           string state = "Running";
            double value =0;
            string ST = "SelectedTarget";
            while (ok_conn && state == "Running")
            {
                ok_conn = bci.GetStateVariable(ST, value);
                Console.WriteLine(value);
                System.Threading.Thread.Sleep(500);
                ok_conn = bci.GetSystemState(state);
            }
           
            bci.Stop();

            // Shut down BCI2000
            bci.Disconnect();

            // Release and delete the BCI2000Remote object
            bci = null;

should I add some code to read from emotiv headset??

[mellinger]

Hi,

the SelectedTarget state will only be nonzero during a single data block after classification. Presumably, your data blocks are much shorter than your sleeping time of 500ms, so you will miss the SelectedTarget state.
When polling for a state value, you need to make sure that polling frequency is at least twice as high as the frequency at which changes may occur. In other words, make sure that the Sleep() call's argument is less than half the block duration, minus any processing time spent in between Sleep() calls.

HTH,
Juergen