Contributions:CommunicationTask

Synopsis

The hyperScanning task is a turn-based communication game that is played between two clients. The two clients exchange information through the hyperScanning backend which uses a client-server model to share the states. Each client has a separate state machine, the server only forwards the information between two clients. Shared states are converted to BCI2000 states asynchronously through events using the HyperscanningApplicationBase class which is the parent class of the application module. For more information, please refer to the hyperScanning back end wiki page: https://www.bci2000.org/mediawiki/index.php/BCI2000_Hyperscanning.

Timeline

• 
  Figure 1: Single trial timeline.

Visual Representation

• 

Versioning

Authors

Huiling Huang (huiling@neurotechcenter.org) Max Marcus (bigmaxmarcus@gmail.com)

Parameters

There are two kinds of parameters: shared and local. The shared parameter is the one you want to synchronize between two clients, like the stimulus sequence, the sequence of break trials you wish to take a break after this trial, etc. It should be saved on the server as a parameter file and passed to the clients through messages as a parameter. The local parameters are that you want to customize the parameter locally, like GUI. The local parameter is the normal BCI2000 parameter file saved under the local BCI2000 project folder.

Shared Parameters

You must upload the parameters below to the server as a parameter file. When the client connects to the server, it will automatically download the parameter file from the server and save it locally. You can indicate the download path in the Application tab, ParamterPath. Please do not modify the parameter file. Otherwise, the synchronization between two clients can't be guaranteed.

StimuliMatrix

Shared parameter, matrix of stimulus information, including:
1. ImageReceiver and ImageSender(the path of images).
2. Sender(the role of players),1 is the sender), 2 is the receiver.
3. Jitter(the duration to present the role).
4. CorrectResponse(1: left, 2:right).
5. Training(1: training, 0: main task).
6. Category(1: object, 2: animal).
7. Difficulty(1: easy, 2: moderate, 3: hard).
8. Dimension1a and Dimension1b(bar#1 left text and right text).
9. Dimension2a and Dimension2b(bar#2 left text and right text).
10. Option1 and Option2(left option text and right option text).

BreakTrialSequece

The sequence of break trials, a break will happen after the break trial.

FeedbackDuration

Duration of feedback(ITI). Indicate the receiver make a correct or incorrect guess.

The server will generate the InitialTrialNumber and StimuliSequence, and save them into a parameter file named by IP addresses on the server. You don't need to worry about this two parameters. Different with the parameters described above, they are saved as a file in the server rather then in the local computers.

InitialTrialNumber

The trial number to begin with. We provide the resume option, allowing clients to pause and resume the game from where they stop. After one of the clients quits the game, the server will discover it and save the current trial number as well as the current trial sequence. When the same clients(same IP address) reconnect the server again, the server will send the previous trial number to them, otherwise the InitialTrailNumber will be initialize by 0. Usually you don't need to touch this paramter.

StimuliSequence

The sequence of stimuli(images). For every pair of client(indentify by the IP address), the server will generate the ramdom stimuli sequence.

Local Parameters

StimuliWidth

StimulusWidth in percent of screen width (zero for original pixel size).

InstructionImagesSeque

The sequence of instruction images path.

InstructionWidth

Instruction width in percent of screen width (zero for original pixel size).

SliderWidth

SliderWidth in percent of screen width (zero for original pixel size).

BarColor

Color of the slider bar.

AxeActiveColor

Color of active slider axe.

AxeInActiveColor

Color of inactive slider axe.

DimensionFontColor

Color of the text under the slider.

DimensionHeight

Height of text in percent of the screen height.

OptionFontActiveColor

Color of the active options.

OptionFontInActiveColor

Color of the inactive options.

OptionHeight

Height of text in percent of the screen height.

RoleFontColor

Color of the role text.

RoleHeight

Height of text in percent of the screen height.

FeedbackWidth

Width of feedback in percent of screen width.

PhotoDiodePatch

Display a photodiode patch on the stimulus window. Recording from a photodiode located on that patch will allow triggering on actual stimulus delivery (see User_Reference: P3TemporalFilter#OnsetExpression).

PhotoDiodePatchHeight, PhotoDiodePatchWidth

Photodiode patch height/width in relative coordinates (between 0 and 1).

PhotoDiodePatchLeft, PhotoDiodePatchTop

Photodiode patch left/top position in relative coordinates (between 0 and 1).

PhotoDiodePatchShape

Photodiode patch shape: 0 rectangle, 1 ellipse.

PhotoDiodePatchActiveColor

Photodiode patch color when active (RGB color in format 0xrrggbb).

PhotoDiodePatchInactiveColor

Photodiode patch color when inactive, (RGB color in format 0xrrggbb, use 0xff000000 for transparent).

States

Shared states are converted to BCI2000 states asynchronously through events using the HyperscanningApplicationBase class. We declared shared states in the batch file as "--SharedStates=state1_name,state1_size&state2_name,state2_size&...", which is appended to the end of the application module.

PhaseNumber

The index of phase in each trial. (8 bits) 0: instruction; 1: wait_to_begin; 2: role_text; 3: image_present; 4: feedback; 5: break; 6: end.

TrialNumber

Index of trials. (8 bits)

BarOneValue

Value of slider#1. (32 bits)

BarTwoValue

Value of slider#2. (32 bits)

BarOneActive, BarTwoActive

0: slider is not selected. 1: slider is selected. (8 bits)

ResponseValue

1: object. 2: animal. (8 bits)

SenderLock, ReceiverLock

0: client has locked the answer. 1: client is moving the slider/making choose. (8 bits)

isReadyStart0, isReadyStart1

0: client isn't ready to start/resume/continue the game. 1: client is ready to start/resume/continue the game. (8 bits)

ClientNumber

Client index, either 0 or 1. (8 bits)

Example

The hyperscanning application module is almost the same as a BCI2000 application module, except that the typical BCI2000 module is inherited from ApplicationBase, and the hyperscanning application module is inherited from HyperscanningApplicationBase. Basically, the HyperscanningApplicationBase class is the same as ApplicationBase class with adding the communication with the server. So instead of using Publish, Preflight, Process, etc. We are using SharedPublish, SharedPreflight, SharedProcess, etc. These methods behave in the exact same way, except they also call the client loops which interact with the server.

Communication_task_dualTask.h:

#include "ApplicationBase.h"
#include "Slider.h"
#include <map>
#include <vector>
#include <string>
#include "TextStimulus.h"
#include "ImageStimulus.h"
#include "Shapes.h"
#include "HyperscanningApplicationBase.h"

class Communication_task_dualTask : public HyperscanningApplicationBase
{
 public:
  Communication_task_dualTask();
  ~Communication_task_dualTask();
  void SharedPublish() override;
  void SharedPreflight( const SignalProperties& Input, SignalProperties& Output ) const override;
  void SharedInitialize( const SignalProperties& Input, const SignalProperties& Output ) override;
  void SharedAutoConfig( const SignalProperties& Input ) override;
  void SharedStartRun() override;
  void SharedProcess( const GenericSignal& Input, GenericSignal& Output ) override;
  void SharedStopRun() override;
  void SharedHalt() override;

 private:
   //define your own private members here
   ApplicationWindow& mrDisplay;

   //define your own private methods here

};

You have to redefine the keyboard or joystick states here.

void
Communication_task_dualTask::SharedPublish()
{


     BEGIN_STATE_DEFINITIONS
     "KeyDown 32 0 0 0",
     "JoystickButtons1 8 0 0 0",
     "JoystickButtons2 8 0 0 0",
     "JoystickButtons3 8 0 0 0",
     "JoystickButtons4 8 0 0 0",
     "JoystickButtons9 8 0 0 0",
     "JoystickButtons10 8 0 0 0"
     END_STATE_DEFINITIONS
}

Manipulate the shared state is the same as the BCI2000 state. "isReadyStart1" is a shared state.

   State("isReadyStart1") = 1; // assign 1 to "isReadyStart1"

Get the value of the shared state "isReadyStart1"

   //Both of the two players are ready, move to the next phase
   if (State("isReadyStart0") && State("isReadyStart1")) {
            m_instruction_container->Conceal();
            m_role_txt->Conceal();
            my_phase = role_text;
            m_block_in_phase = 0;
   }