View source for Contributions:CommunicationTask

==Synopsis==
The hyperscanning task is a turn-based communication game between two clients. This game aims to guess what the subject is in Sender's computer without talking. For each trial, the two players will alternate between the roles of Sender and Receiver. The Sender will describe traits of the subject presented by adjusting the two axes(Noisy or silent/Big or small). Then, the Receiver will guess if the Sender is describing the animal or object. After the Receiver makes a guess, both players receive feedback on whether the Receiver makes the correct guess. This marks the end of a trial.

The two clients exchange information through the hyperScanning backend, which uses a client-server model to share the states. Each client has a separate state machine. The server only forwards the information between two clients. Shared states are converted to BCI2000 states asynchronously through events using the HyperscanningApplicationBase class, which is the parent class of the application module. For more information, please refer to the hyperScanning framework wiki page: https://www.bci2000.org/mediawiki/index.php/BCI2000_Hyperscanning.

==Timeline==
<gallery mode="packed" widths=600px heights=500px>
File:Hyperscanning pipe line.png|Figure 1: Single Trial Timeline.
</gallery>

== Visual Representation ==
<gallery mode="packed" widths=750px heights=421px>
File:Hyperscanning_1.gif|Figure 2: Visual Representation
</gallery>

==Versioning==
===Authors===
Huiling Huang (huiling@neurotechcenter.org)<br>
Max Marcus (bigmaxmarcus@gmail.com)

==Set up==
===Deploy the Server===
You will need a Linux or macOS computer to run the server. Currently, the Windows OS is not supported. The following steps are for deploying the server on the local network. If you want to deploy the server on AWS, please refer to https://www.bci2000.org/mediawiki/index.php/BCI2000_Hyperscanning#Important_Notes_2.<br>
First, please ensure all the computers are connected to the same local network. Then open the terminal(Mac OS or Linux) and run the command below to clone the server code from GitHub.
<pre>git clone https://github.com/MaxwellMarcus/hyperscanning-backend</pre>
Go the hyperscanning directory
<pre>cd hyperscanning</pre>
Compile the project by
<pre>make -B</pre>
<gallery mode="packed"  widths=600px heights=350px>
File:CommunicationTask server set up.png|Figure 4: Download and Compile the Server.
</gallery>
Run the server with
<pre>./application</pre>
<gallery mode="packed" widths=600px heights=100px>
File:CommunicationTask_server_launched.png|Figure 5: Launch the Server.
</gallery>
The application will only run as long as the experiment and will have to be restarted.

===Batch File===
The batch file is located in ./batch/CommunicationTask.bat. 
<gallery mode="packed" widths=600px heights=400px>
File:CommunicationTask batch.png|Figure 3: Batch File.
</gallery>

Unlike the normal bci2000 task, we define the shared states in the batch file(line 35) by beginning with"--SharedStates=". The form is:
<pre>--SharedStates=<state-1>,<state-1-size>&<state-2>,<state-2-size>&<state-3>,<state-3-size>...</pre>
IPAddress and Port indicate the server computer.


===Run the experiment on the client computer===
Go to the./batch, find the "CommunicationTask.bat", and double click it. After pushing "Set Config," the BCI2000 will try to connect the server. If it successfully connects to the server, the System Log will print "Connected to server." If only one client connects to the server, the "Start" button in the operator will turn gray until another client connects.

<gallery mode="packed" widths=600px heights=400px>
File:Launch_the_experiment.gif|Figure 6: Launch the BCI2000.
</gallery>

==Parameters==
There are two kinds of parameters: shared and local. The shared parameter is the one you want to synchronize between two clients, like the stimulus sequence, the sequence of break trials you wish to take a break after this trial, etc. It should be saved on the server as a parameter file and passed to the clients through messages as a parameter. The local parameter is that you want to customize the parameter locally, like GUI. The local parameter is the normal BCI2000 parameter file saved under the local BCI2000 project folder.


===Shared Parameters===
You must upload the parameters below to the server as a parameter file. When the client connects to the server, it will automatically download the parameter file from the server and save it locally. You can indicate the download path in the Application tab, ParamterPath. Please do not modify the parameter file. Otherwise, the synchronization between two clients can't be guaranteed.

====StimuliMatrix====
Shared parameter, matrix of stimulus information, including:<br>
1. ImageReceiver and ImageSender(the path of images).<br>
2. Sender(the role of players),1 is the sender), 2 is the receiver.<br>
3. Jitter(the duration to present the role).<br>
4. CorrectResponse(1: left, 2:right).<br>
5. Training(1: training, 0: main task).<br>
6. Category(1: object, 2: animal).<br>
7. Difficulty(1: easy, 2: moderate, 3: hard).<br>
8. Dimension1a and Dimension1b(bar#1 left text and right text).<br>
9. Dimension2a and Dimension2b(bar#2 left text and right text).<br>
10. Option1 and Option2(left option text and right option text).

====BreakTrialSequece====
The sequence of break trials, a break will happen after the break trial.

====FeedbackDuration====
Duration of feedback(ITI). Indicate the receiver to make a correct or incorrect guess.<br><br>

The server will generate the InitialTrialNumber and StimuliSequence, and save them into a parameter file named by IP addresses on the server. You don't need to worry about these two parameters. Unlike the parameters described above, they are saved on the server rather than on local computers.

====InitialTrialNumber====
The trial number to begin with. We provide the resume option, allowing clients to pause and resume the game from where they stop. After one of the clients quits the game, the server will discover it and save the current trial number and sequence. When the same clients(same IP address) reconnect the server again, the server will send the previous trial number to them. Otherwise the InitialTrailNumber will be initialized by 0. Usually, you don't need to touch this parameter.

====StimuliSequence====
The sequence of stimuli(images). The server will generate a random stimuli sequence for every pair of clients (identified by the IP address).

===Local Parameters===

====StimuliWidth====
StimulusWidth in percent of screen width (zero for original pixel size).

====InstructionImagesSeque====
The sequence of instruction images path.

====InstructionWidth====
Instruction width in percent of screen width (zero for original pixel size).

====SliderWidth====
SliderWidth in percent of screen width (zero for original pixel size).

====BarColor====
Color of the slider bar.

====AxeActiveColor====
Color of active slider axe.

====AxeInActiveColor====
Color of inactive slider axe.

====DimensionFontColor====
Color of the text under the slider.

====DimensionHeight====
Height of text in percent of the screen height.

====OptionFontActiveColor====
Color of the active options.

====OptionFontInActiveColor====
Color of the inactive options.

====OptionHeight====
Height of text in percent of the screen height.

====RoleFontColor====
Color of the role text.

====RoleHeight====
Height of text in percent of the screen height.

====FeedbackWidth====
Width of feedback in percent of screen width.

====PhotoDiodePatch====
Display a photodiode patch on the stimulus window. Recording from a photodiode located on that patch will allow triggering on actual stimulus delivery (see User_Reference: P3TemporalFilter#OnsetExpression).

==== PhotoDiodePatchHeight, PhotoDiodePatchWidth====
Photodiode patch height/width in relative coordinates (between 0 and 1).

====PhotoDiodePatchLeft, PhotoDiodePatchTop ====
Photodiode patch left/top position in relative coordinates (between 0 and 1).

====PhotoDiodePatchShape====
Photodiode patch shape: 0 rectangle, 1 ellipse.

====PhotoDiodePatchActiveColor====
Photodiode patch color when active (RGB color in format 0xrrggbb).

====PhotoDiodePatchInactiveColor====
Photodiode patch color when inactive, (RGB color in format 0xrrggbb, use 0xff000000 for transparent).

==States==
Shared states are converted to BCI2000 states asynchronously through events using the HyperscanningApplicationBase class. We declared shared states in the batch file as "--SharedStates=state1_name,state1_size&state2_name,state2_size&...", which is appended to the end of the application module.

===PhaseNumber===
The index of phase in each trial. (8 bits) <br>  
[0] indicates the instruction at the beginning of the experiment. (When the 10 slides instruction images are present on the screen)<br> 
[1] indicates the client is waiting for the other client to push the button to start the game. ( We are waiting for another player; once she/he is ready, we're ready to go!)<br> 
[2] When the role text is present on the screen.(Sender or Receiver).<br> 
[3] When the sliders and stimulus are present on the screen.<br> 
[4] Feedback: After the receiver locks the answer, it comes to phase [4], showing whether the answer is correct.<br> 
[5]Break: (Well done! Are you ready for the next block? \n\n Press SPACEBAR)<br> 
[6]End: (Well done! You have finished all the trials.)

===TrialNumber===
Index of trials. (8 bits)

===BarOneValue===
Value of slider#1. (32 bits)

===BarTwoValue===
Value of slider#2. (32 bits)

===BarOneActive, BarTwoActive===
0: slider is not selected.  1: slider is selected. (8 bits)

===ResponseValue===
1: object.    2: animal. (8 bits)

===SenderLock, ReceiverLock===
0: client is moving the slider/making a guess. 1: client has locked the answer. (8 bits)

===isReadyStart0, isReadyStart1===
0: client isn't ready to start/resume/continue the game(not yet pushing the button).  1: client is ready to start/resume/continue the game(push the button). (8 bits)

===ClientNumber===
The client’s ID, is assigned by the server. Either 0 or 1. (8 bits)

==Example==
The hyperscanning application module is almost the same as a BCI2000 application module, except that the typical BCI2000 module is inherited from ApplicationBase, and the hyperscanning application module is inherited from HyperscanningApplicationBase. Basically, the HyperscanningApplicationBase class is the same as ApplicationBase class with adding the communication with the server. So instead of using  Publish, Preflight, Process, etc. We are using SharedPublish, SharedPreflight, SharedProcess, etc. These methods behave in the exact same way, except they also call the client loops which interact with the server.


Communication_task_dualTask.h:
<pre>
#include "ApplicationBase.h"
#include "Slider.h"
#include <map>
#include <vector>
#include <string>
#include "TextStimulus.h"
#include "ImageStimulus.h"
#include "Shapes.h"
#include "HyperscanningApplicationBase.h"

class Communication_task_dualTask : public HyperscanningApplicationBase
{
 public:
  Communication_task_dualTask();
  ~Communication_task_dualTask();
  void SharedPublish() override;
  void SharedPreflight( const SignalProperties& Input, SignalProperties& Output ) const override;
  void SharedInitialize( const SignalProperties& Input, const SignalProperties& Output ) override;
  void SharedAutoConfig( const SignalProperties& Input ) override;
  void SharedStartRun() override;
  void SharedProcess( const GenericSignal& Input, GenericSignal& Output ) override;
  void SharedStopRun() override;
  void SharedHalt() override;

 private:
   //define your own private members here
   ApplicationWindow& mrDisplay;

   //define your own private methods here

};
</pre>


Manipulate the shared state is the same as the BCI2000 state. "isReadyStart1" is a shared state.
<pre>
   State("isReadyStart1") = 1; // assign 1 to "isReadyStart1"
</pre>

Get the value of the shared state "isReadyStart1"
<pre>
   //Both of the two players are ready, move to the next phase
   if (State("isReadyStart0") && State("isReadyStart1")) {
            m_instruction_container->Conceal();
            m_role_txt->Conceal();
            my_phase = role_text;
            m_block_in_phase = 0;
   }
</pre>