bci2000.org BBS

Posted: **17 Feb 2012, 10:56**

Dear all,
I'm in the need of using P300Speller tightly connected with a custom application, treating the P300Speller as an "input engine".
I am thinking about graphically integrate the P300Speller in a black background application, positioning and shaping the P300Speller interface so that it might leaves some space for my custom application in background. This because my application should provide certain graphical and audio output that are not easy (or not reasonable) to integrate in the P300Speller code.

For this reason, I'd like to know:
1) what is the best way to do this
2) if there is a way to communicate (via socket - I suppose) from an external application which should be able to "suspend" and "unsuspend" the P300Speller, after each sequence (to let the external application handling the choice)
3) if it could be possible to modify the "TIME OUT" and "WAITING TO START" messages that comes respectively after the "suspend" and "unsuspend" actions have been performed
4) if the only way to set the status to be invisible is to reduce its dimensions to zero.

Thanks for your attention

Kind regards

Paolo

Posted: **19 Feb 2012, 14:05**

5) which UDP message should I have the external application to look for in order to have it "realizing" that a new selection has been performed (thought simply to look for "SelectedTarget X" but such message is continuously broadcasted), in order to send some specific messages to stop for a while the P300SPeller, perform some actions and then "restart" the P300SPeller.

Posted: **20 Feb 2012, 07:04**

Hi Paolo,

my suggestion is to proceed along the following path:
1) Integrate the P3Speller window into a separate application. This may be done by "hijacking" its window from another application. You may even draw on top of the P3Speller window then. For a suggestion how that may be done, see this thread:
http://www.bci2000.org/phpbb/viewtopic.php?f=5&t=1083
That thread shows an example how to do it in C# but it will work similarly with a different windowing toolkit.
2) Read/write BCI2000 state values from the external application. This can be done using the AppConnector interface:
http://www.bci2000.org/wiki/index.php/T ... _Connector
3) Slightly modify P3Speller such that it listens to a new state "paused":
* Add
BEGIN_STATE_DEFINITION
"Paused 1 0 0 0",
END_STATE_DEFINITION
to the constructor.
* To the header file, add a new member function with the following name and signature:
void Process( const GenericSignal&, GenericSignal& );
* Add an implementation of Process() to the cpp file, that looks like:
void P3SpellerTask::Process( const GenericSignal& Input, GenericSignal& Output )
{
if( !State( "Paused" ) )
StimulusTask::Process( Input, Output );
}
Now you can control the P3Speller via the AppConnector interface. It also will not be "suspended" the normal way, so you need not bother with the respective messages.
4) To read P3Speller selection output from your external application, you have two options:
*Read the "SelectedTarget" state through the AppConnector interface. It will give you the row number of the selected target in the TargetDefinitions parameter. There are also "SelectedRow" and "SelectedColumn" states, in case you like these better -- these give you the row and column of the selected target in the current speller matrix.
*Use the DestinationAddress parameter to specify a UDP address into which the P3Speller will write its selection result. In this case, you will receive the selected letter (or command) as specified in the TargetDefinitions parameter's "Enter" column.
5) Set the StatusBarHeight parameter to zero in order to hide the status bar.

Best regards,
Juergen

Posted: **21 Feb 2012, 18:26**

Dear Juergen,
thanks a lot for your thorough description. Unfortunately, it seems that there is still something I'm missing. After having implemented the changes you suggested, I tried the P300Speller_SignalGenerator batch example with a simple C# code (see below). What I would like to do was simply to wait for every selection performed (the results are sent to the destinationAddress), put the P300Speller in paused state, do something and then "unpause" the P300Speller and wait for the next selection.

So in the Config I set

ConnectorOutput = localhost:9050
ConnectorInput = localhost:9051
ConnectorInputFilter = *
DestinationAddress = localhost:9052

However, when I run the following code, everything goes fine until I send the "Paused 0" to the ConnectorInput in order to "unpause" the P300Speller process. from that moment on, the console simply reports some message coming from the ConnectorOutput socket, then an interesting "Paused 0", and then the P300SPeller process gets stucked and I have to kill it badly unless I cannot run again the batch file.

Any hots are verrrrrrrrry welcome

Paolo

Code: Select all

using System;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading;

namespace BCI2000_P300_Paused
{
    class P300_SimplePausedTest
    {
        public static void Main()
        {
            int recv;
            byte[] data = new byte[1024];
            string PauseOn = "Paused 1";
            byte[] dataPauseOn = Encoding.ASCII.GetBytes(PauseOn);
            string PauseOff = "Paused 0";
            byte[] dataPauseOff = Encoding.ASCII.GetBytes(PauseOff);

            IPEndPoint ConnectorOutput = new IPEndPoint(IPAddress.Any, 9050);
            UdpClient ConnectorInput = new UdpClient("127.0.0.1", 9051);

            IPEndPoint DestinationAddress = new IPEndPoint(IPAddress.Any, 9052);
            Socket FromConnectorOutput = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
            Socket FromDestinationAddress = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);

            FromConnectorOutput.Bind(ConnectorOutput);
            FromDestinationAddress.Bind(DestinationAddress);

            Console.WriteLine("Waiting for Speller to start...");
            IPEndPoint sender = new IPEndPoint(IPAddress.Any, 0);
            EndPoint Remote = (EndPoint)(sender);

            //Let's wait for some data on the ConnectorOutput socket
            recv = FromConnectorOutput.ReceiveFrom(data, ref Remote);
            Console.WriteLine("Message received from {0}:", Remote.ToString());
            Console.WriteLine(Encoding.ASCII.GetString(data, 0, recv));

            try
            {
                while (true)
                {

                    //I received some data on the ConnectorSocket, let's wait for the result code
                    recv = FromDestinationAddress.ReceiveFrom(data, ref Remote);
                    Console.WriteLine("Message received from {0}:", Remote.ToString());
                    Console.WriteLine(Encoding.ASCII.GetString(data, 0, recv));

                    //Let's put P300SPeller in the state "Paused 1" 
                    //data = Encoding.ASCII.GetBytes(PauseOn);
                    ConnectorInput.Send(dataPauseOn, dataPauseOn.Length);

                    //let's wait for a second before restart the P300Speller
                    for (int i = 0; i < 10; i++)
                    {
                        Console.WriteLine(i.ToString() + Environment.NewLine);
                        Thread.Sleep(100);
                    }

                    //Let's restart the P300Speller
                    //data = Encoding.ASCII.GetBytes(PauseOff);
                    ConnectorInput.Send(dataPauseOff, dataPauseOff.Length);

                
                    //let's wait for the "Paused 0" condition

                    while (true)
                    {
                        recv = FromConnectorOutput.ReceiveFrom(data, ref Remote);
                        string received = Encoding.ASCII.GetString(data, 0, recv);
                        Console.WriteLine(received);
                        if(received.Contains("Paused 0"))
                            break;
                    }
                
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("Error : " + ex.Message.ToString());
            }
        }
    }
}

Posted: **22 Feb 2012, 12:24**

Hi Paolo,

as described on the AppConnector page, each message must be terminated with a newline character (0x0a). Writing

Code: Select all

string PauseOn = "Paused 1\n";

(and correspondingly for PauseOff), should do the trick.
Thanks for your hint regarding the blocked connector. I will make ConnectorInput more robust, so it can handle this type of input.

Regards,
Juergen

Posted: **22 Feb 2012, 12:59**

nooooooo... that simple.... ok.... I know... I should have read the manual more carefully...

Thanxalot

paolo

Posted: **23 Feb 2012, 18:52**

Just in case, I wrote another C# version with a Form and using async UDP sockets instead of reaading the socket via blocking calls. I thought it could be an interesting end for this thread.
Thnx again to Juergen (and all the BCI2000 guys) for the great support they provide.

In order to use the code, simply create a new form with three textboxes (textBox1, textBox2 and textBox3) for the output. Then cut and paste the following code. Although this code seems to work fine, any further comment or improvement is always very welcome.

Paolo

Code: Select all

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.Net;
using System.Net.Sockets;
using System.Threading;
using System.Media;

namespace BCI2000_eBrainBridge
{
    public partial class Form1 : Form
    {

        byte[] databuffer = new byte[1024];

        string PauseOn = "Paused 1\n";
        string PauseOff = "Paused 0\n";

        UdpClient UDPClientToConnIn;

        Socket SkFromConnOut = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
        Socket SkFromDestAddr = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);

        byte[] dataPauseOn, dataPauseOff;

        bool bFirstDataReceived = false;

        public Form1()
        {
            InitializeComponent();

            CheckForIllegalCrossThreadCalls = false;

            dataPauseOn = Encoding.ASCII.GetBytes(PauseOn);
            dataPauseOff = Encoding.ASCII.GetBytes(PauseOff);

            SkFromConnOut.Bind(new IPEndPoint(IPAddress.Any, 9050));
            SkFromDestAddr.Bind(new IPEndPoint(IPAddress.Any, 9052));

            UDPClientToConnIn = new UdpClient("127.0.0.1", 9051);

            EndPoint Remote = new IPEndPoint(IPAddress.Any, 0);
            SkFromConnOut.BeginReceiveFrom(databuffer, 0, databuffer.Length,
                SocketFlags.None, ref Remote, new AsyncCallback(OnReceiveFromConnectorOutput), SkFromConnOut);
        }

        private void OnReceiveFromConnectorOutput(IAsyncResult ar)
        {
            try
            {
                Socket recvSock = (Socket)ar.AsyncState;
                EndPoint epSender = new IPEndPoint(IPAddress.Any, 0);

                int msgLen = SkFromConnOut.EndReceiveFrom(ar, ref epSender);

                byte[] localMsg = new byte[msgLen];
                Array.Copy(databuffer, localMsg, msgLen);

                string outcome = Encoding.ASCII.GetString(localMsg, 0, msgLen);
                textBox1.Text = outcome;

                if (!bFirstDataReceived)
                {
                    bFirstDataReceived = true;

                    UDPClientToConnIn.Send(dataPauseOn, dataPauseOn.Length);

                    DoSomething();

                    UDPClientToConnIn.Send(dataPauseOff, dataPauseOff.Length);

                    EndPoint newClientEP = new IPEndPoint(IPAddress.Any, 0);
                    SkFromDestAddr.BeginReceiveFrom(databuffer, 0, databuffer.Length,
                        SocketFlags.None, ref newClientEP, new AsyncCallback(OnReceiveFromDestinationAddress), SkFromDestAddr);
                }
                else
                {
                    if (outcome.Contains("Paused 0"))
                    {
                        EndPoint newClientEP = new IPEndPoint(IPAddress.Any, 0);
                        SkFromDestAddr.BeginReceiveFrom(databuffer, 0, databuffer.Length,
                            SocketFlags.None, ref newClientEP, new AsyncCallback(OnReceiveFromDestinationAddress), SkFromDestAddr);
                    }
                    else
                    {
                        EndPoint newClientEP = new IPEndPoint(IPAddress.Any, 0);
                        SkFromConnOut.BeginReceiveFrom(databuffer, 0, databuffer.Length,
                            SocketFlags.None, ref newClientEP, new AsyncCallback(OnReceiveFromConnectorOutput), SkFromConnOut);
                    }
                    this.Invalidate();
                }
            }
            catch (Exception ex)
            {
                MessageBox.Show(ex.Message, "BCI2000_eBrainBridge - OnReceiveFromConnectorOutput",
               MessageBoxButtons.OK, MessageBoxIcon.Error);
            }
        }


        private void OnReceiveFromDestinationAddress(IAsyncResult ar)
        {
            try
            {
                Socket recvSock = (Socket)ar.AsyncState;
                EndPoint epSender = new IPEndPoint(IPAddress.Any, 0);

                int msgLen = recvSock.EndReceiveFrom(ar, ref epSender);
                byte[] localMsg = new byte[msgLen];
                Array.Copy(databuffer, localMsg, msgLen);

                textBox2.Text = Encoding.ASCII.GetString(localMsg, 0, msgLen);

                DoSomething();

                EndPoint newClientEP = new IPEndPoint(IPAddress.Any, 0); 
                SkFromConnOut.BeginReceiveFrom(databuffer, 0, databuffer.Length,
                    SocketFlags.None, ref newClientEP, new AsyncCallback(OnReceiveFromConnectorOutput), SkFromConnOut);
            }
            catch (Exception ex)
            {
                MessageBox.Show(ex.Message, "BCI2000_eBrainBridge - OnReceiveFromDestinationAddress",
             MessageBoxButtons.OK, MessageBoxIcon.Error);
            }
        }

        private void DoSomething()
        {
            UDPClientToConnIn.Send(dataPauseOn, dataPauseOn.Length);
            System.Media.SystemSounds.Asterisk.Play();

            //let's wait for a second before restart the P300Speller
            for (int i = 0; i < 5; i++)
            {
                Console.WriteLine(i.ToString() + Environment.NewLine);

                textBox3.Text = i.ToString();
                this.Refresh();
                Thread.Sleep(500);
            }

            UDPClientToConnIn.Send(dataPauseOff, dataPauseOff.Length);
        }
    }
}

bci2000.org BBS

"Including" P300Speller in a custom application

"Including" P300Speller in a custom application

Re: "Including" P300Speller in a custom application - (add-o

Re: "Including" P300Speller in a custom application

Re: "Including" P300Speller in a custom application

Re: "Including" P300Speller in a custom application

Re: "Including" P300Speller in a custom application

Re: "Including" P300Speller in a custom application