Audio Stimuli: nested matrices

[Francesco]

Hi,

I would like to create a P300 speller of audio stimuli composed by one array (row or column) of n icons.

First question: is it possible in one stimuli array not consider the whole row or column stimulus (flashing)? I don't want the whole row or column is not flashing.

Second question: when one icon flashes then a sound, randomly chosen from a vector of sound, has to be played. Is it possible to do that with BCI2000?

I have to do that in Copy Mode and Online Free Mode

Thanking you in advance for your help, I look forward to your reply.

Francesco

[mellinger]

Hi,

re your first question: When you have only a single row or column, flashing of the entire row/column will not occur in recent versions of BCI2000 (unless there is a bug, which will then be fixed).

Re your second question: This is not possible without modifying the source code of the P3SpellerTask class. Basically, what you need to do is in the to modify the so-called "associations" between multiple stimuli and targets, and to modify the way how the sequence of stimulus codes is created.

At the end of P3SpellerTask::LoadMenu(), your modified code should iterate over all existing items in the Associations() map, and duplicate these by assigning them to a new stimulus code, such that you have a separate stimulus code for each sound alternative. Then, iterate over the original and the newly created associations, and add a new AudioStimulus object to each, which plays the desired sound file. For the stimulus codes of the newly created associations, you should choose values that are composed of the original stimulus code in the lower bits, and the index of the current alternative in the upper bits. This will make it easier to implement the change to the sequence of stimulus codes: After that sequence has been created, randomly modify the upper bits of the stimulus codes in the sequence.

HTH,
Juergen

[Francesco]

Hi,

"re your first question: When you have only a single row or column, flashing of the entire row/column will not occur in recent versions of BCI2000 (unless there is a bug, which will then be fixed)."

I've downloaded the latest release of BCI2000 but the problem occurs again. I've downloaded also the latest source code (version 3.0.5) and I've compiled it but the problem is there.

Thanks for the second reply, I'll try to modify the code. I will let you know

Francesco

[mellinger]

Hi,

it seems I intended to apply that modification at some time but then was distracted. It's now in the P3Speller source code, so it should work as intended when you update to the latest version.

Regards,
Juergen

[Francesco]

Hi,

"At the end of P3SpellerTask::LoadMenu(), your modified code should iterate over all existing items in the Associations() map, and duplicate these by assigning them to a new stimulus code, such that you have a separate stimulus code for each sound alternative. Then, iterate over the original and the newly created associations, and add a new AudioStimulus object to each, which plays the desired sound file". Does it mean:

for( int i = 0; i < AudioStimuliRowsFiles->NumRows(); ++i )
{
int row = ::atoi( AudioStimuliRowsFiles->RowLabels()[ i ].c_str() );
AudioStimulus* pStimulusNEW = new AudioStimulus;
pStimulusNEW->SetSound( AudioStimuliRowsFiles( i, 1 ) );
if( !pStimulusNEW->Error().empty() )
bcierr << "AudioStimulusRowsFiles(" << i << ", " << 1 << "): "
<< pStimulusNEW->Error()
<< endl;
ioStimuliNEW.Add( pStimulusNEW );
ioAssociationsNEW[ row ].Add( pStimulusNEW );
}

Thank you
Francesco Ferracuti

[mellinger]

Sorry for being so unclear. ioAssociations() is a container, a so-called map, which uses stimulus codes as indices, and contains objects of type Association:
typedef std::map<int, Assocation> AssociationMap;

An Association object may contain any number of stimuli, and any number of targets. Whenever a certain stimulus code is active, ioAssociations() is used to look up the corresponding Association object, and then all stimuli are presented which are contained there. On classification, responses to stimuli are mapped to targets, again using ioAssocations() to find the targets corresponding to certain stimulus codes.

So the code might look like

Code: Select all

int maxExisting = ioAssociations().rbegin()->first;
for( AssocationMap::iterator i = ioAssociations().begin(); i->first <= maxExisting; ++i )
  for( int j = 1; j < numberOfAlternatives; ++j )
    ioAssocations()[j * maxExisting + i->first] = i->second;
// Then either:
for( AssocationMap::iterator i = ioAssociations().begin(); i != ioAssociations.end(); ++i )
{
  int baseCode = i->first % maxExisting,
       alternative = i->first / maxExisting;
  AudioStimulus* pStimulus = new AudioStimulus;
  pStimulus->SetSound( Whatever( baseCode, alternative ) );
  ...
  ioStimuli.Add( pStimulus );
  i->second.Add( pStimulus );
}
// Or:
for( int i = 0; i < numRows; ++i )
  for( int j = 0; j < numberOfAlternatives; ++j )
  {
    ...
    ioAssociations[j * maxExisting + i + 1].Add( pStimulus );
  }
for( int i = 0; i < numCols; ++i )
  for( int j = 0; j < numberOfAlternatives; ++j )
  {
    ...
    ioAssociations[j * maxExisting + numRows + i + 1].Add( pStimulus );
  }

HTH,
Juergen

[Francesco]

Now the last version works, thank you.

Ok, I solved the problem by creating new Associations() objects and calling them randomly:

Code: Select all

void
StimulusTask::OnStimulusBegin( int inStimulusCode )
{
int p=rand() % 4+ 1;
switch (p) {
case 1:
Associations1()[ inStimulusCode ].Present();
break;
case 2:
Associations2()[ inStimulusCode ].Present();
break;
case 3:
Associations3()[ inStimulusCode ].Present();
break;
case 4:
Associations4()[ inStimulusCode ].Present();
break;
}
}

but now when I train the SWLDA classificator I receive the message: Dataset is corrupted. Try another dataset. I saw that the P300classificator doesn't work for every one vector Speller.
I checked and the file is not corrupted and the warning is in the function GetScore:

///////////////////////////////////////////////////////////////////////
// Section: Create a matrix with the scores for each sequence

Code: Select all

pscore.setbounds(0, NumberOfChoices-1, 0, (NumberOfEpochs*NumberOfSequences)-1);
for (int i=0; i<NumberOfEpochs; i++)
{
for (size_t j=0; j<trial.size(); j++)
{
if (trial[j] == i+1)
range.push_back(static_cast<int>(j));
}
if ((range.size() != 0) && (range.size() == NumberOfSequences*NumberOfChoices))
{
for (int k=0; k<NumberOfChoices; k++)
{
for (size_t j=0; j<range.size(); j++)
{
if (Code(range[j]) == k+1)
code_indx.push_back(range[j]);
}
for (size_t j=0; j<code_indx.size(); j++)
{
if (code_indx.size() == NumberOfSequences)
pscore(k,static_cast<int>((i*NumberOfSequences)+j)) = score(code_indx[j]);
}
code_indx.clear();
}
flag = true;
}
else
{
flag = false;
break;
}
range.clear(); 
}

return flag;

/////////////////////////////////////////////////////////////////////////

I see that (range.size() != NumberOfSequences*NumberOfChoices) because NumberOfChoices=parms.NumMatrixRows + parms.NumMatrixColumns for 6x6 matrix is 12 (that's correct) but for one vector 1x6 is 7(that isn't correct). I tried to force NumberOfChoices=6 but I receive a fatal error from P300_Classify function.
How can I use P300Classificator with one column or row vector Speller Menu?

Thank you
Francesco Ferracuti

[mellinger]

Hi Francesco,

I fixed the P300Classifier tool. It should now work with single-column and single-row data.

Regards,
Juergen

[Francesco]

Hi,

I tried, but I received this Windows message error:

Microsoft Visual c++ Debug Library:
Debug assertion failed!

Program:
...BCI2000src\tools\P300Classifier\P300Classifier.exe
File: c:\Program Files(x86)\Microsoft Visual Studio\10.0\VC\include\vector Line:932

Expression: vector subscript out of range.

and the P300classifier crashes.

Best regards,
Francesco

[mellinger]

Hi,

it worked fine when I tested with simulated data. Could you send me a data file that causes the crash, so I can fix P300Classifier?

(Use the "send email" button next to this post.)

Regards,
Juergen

[mellinger]

Hi,

thanks for sending the example data file. The bug has been fixed.

Best regards,
Juergen

[Francesco]

Hi,

now it works very good!! Thank you for all your assistance

Best regards,
Francesco