bci2000.org BBS

Hi everybody,

I would like to be able to read 64-bit floats data format from the buffer of FieldTripBufferSource.
I noticed that currently, the only supported data formats that can be grabbed from the buffer are 16-bit integers, 32-bit integers, and 32-bit floats.
However, I wonder if someone is developping or has already developped this functionnality ?
Otherwise, is there possible to help me to integrate it? I have already done some modifications but I am in difficulty to do some changes in some files...

Thank you very much for your help



This is the changes I did :

FieldTripBufferADC.cpp
…
{
SignalType::Type sType;

switch(hdr.data_type) {
case DATATYPE_INT32:
sType = SignalType::int32;
break;
case DATATYPE_INT16:
sType = SignalType::int16;
break;
case DATATYPE_FLOAT32:
sType = SignalType::float32;
break;
case DATATYPE_FLOAT64:
sType = SignalType::float64;
break;
default:
bcierr << "Datatype : '" << hdr.data_type << "' not supported yet" << endl;
return;
}
…
switch(datadef.data_type) {
case DATATYPE_INT16:
{
INT16_T *src = (INT16_T *) databuf.data();

for( int sample = 0; sample < Output.Elements(); ++sample )
for( int channel = 0; channel < Output.Channels(); ++channel )
Output( channel, sample ) = *src++;
}
break;
case DATATYPE_INT32:
{
INT32_T *src = (INT32_T *) databuf.data();

for( int sample = 0; sample < Output.Elements(); ++sample )
for( int channel = 0; channel < Output.Channels(); ++channel )
Output( channel, sample ) = *src++;
}
break;
case DATATYPE_FLOAT32:
{
FLOAT32_T *src = (FLOAT32_T *) databuf.data();

for( int sample = 0; sample < Output.Elements(); ++sample )
for( int channel = 0; channel < Output.Channels(); ++channel )
Output( channel, sample ) = *src++;
}
break;
case DATATYPE_FLOAT64:
{
FLOAT64_T *src = (FLOAT64_T *) databuf.data();

for( int sample = 0; sample < Output.Elements(); ++sample )
for( int channel = 0; channel < Output.Channels(); ++channel )
Output( channel, sample ) = *src++;
}
break;
}

____________________________________________________________________________

SignalType.h
…
typedef enum Type
{
none = -1,
int16 = 0,
float24,
float32,
int32,
float64,

numTypes,
defaultType = float32

} Type;
…
____________________________________________________________________________
SignalType.cpp
…
SignalTypeProperties[] =
{
{ SignalType::int16, "int16", 2, - ( 1 << 15 ), ( 1 << 15 ) - 1 },
{ SignalType::float24, "float24", 3, - numeric_limits<float>::max(), numeric_limits<float>::max() },
{ SignalType::float32, "float32", 4, - numeric_limits<float>::max(), numeric_limits<float>::max() },
{ SignalType::float64, "float64", ?, - numeric_limits<double>::max(), numeric_limits<double>::max() },
#if defined( __BORLANDC__ ) && ( __BORLANDC__ <= 0x0560 ) // bcc32 5.5.1 does not have LL type
{ SignalType::int32, "int32", 4, - ( 1i64 << 31 ), ( 1i64 << 31 ) - 1 },
#else // __BORLANDC__
{ SignalType::int32, "int32", 4, - ( 1LL << 31 ), ( 1LL << 31 ) - 1 },
#endif // __BORLANDC__
};
…
static const bool conversionTable[ numTypes ][ numTypes ] =
{
/* int16 float24 float32 int32 float64*/
/* int16 */ { true, true, true, true, ? },
/* float24 */ { false, true, true, false, ? },
/* float32 */ { false, false, true, false, ? },
/* int32 */ { false, false, false, true, ? },
/* float64 */ { ? , ? , ? , ? , ? },
};
…
____________________________________________________________________________
BCI2000OutputFormat.cpp
…
switch( inProperties.Type() )
{
case SignalType::int16:
case SignalType::int32:
case SignalType::float32:
case SignalType::float64:
/* These types are OK */
break;

default:
bcierr << inProperties.Type().Name()
<< " data type unsupported for BCI2000 files"
<< endl;
}
…
BCI2000OutputFormat::Write( ostream& os, const GenericSignal& inSignal, const StateVector& inStatevector )
{
switch( mInputProperties.Type() )
{
case SignalType::int16:
PutBlock<SignalType::int16>( os, inSignal, inStatevector );
break;

case SignalType::float32:
PutBlock<SignalType::float32>( os, inSignal, inStatevector );
break;

case SignalType::int32:
PutBlock<SignalType::int32>( os, inSignal, inStatevector );
break;

case SignalType::float64:
PutBlock<SignalType::float64>( os, inSignal, inStatevector );
break;

default:
bcierr << "Unsupported signal data type" << endl;
}
}
…
____________________________________________________________________________
BCI2000FileReader.cpp
…
if( isBigEndian )
{
switch( mSignalType )
{
case SignalType::int16:
value = ReadValue_SwapBytes<int16_t>( address );
break;
case SignalType::int32:
value = ReadValue_SwapBytes<int32_t>( address );
break;
case SignalType::float32:
value = ReadValue_SwapBytes<float32_t>( address );
break;
case SignalType::float64:
value = ReadValue_SwapBytes<float64_t>( address );
break;
default:
break;
}
}
else
{ // Little endian machine
switch( mSignalType )
{
case SignalType::int16:
value = ReadValue<int16_t>( address );
break;
case SignalType::int32:
value = ReadValue<int32_t>( address );
break;
case SignalType::float32:
value = ReadValue<float32_t>( address );
break;
case SignalType::float64:
value = ReadValue<float64_t>( address );
break;
default:
break;
}
}
…
____________________________________________________________________________
defines.h
…
typedef float float32_t;
typedef double float64_t;

// Backward compatibility (for new code, use stdint types):
#if 1
typedef uint8_t uint8;
typedef int8_t sint8;
typedef uint16_t uint16;
typedef int16_t sint16;
typedef uint32_t uint32;
typedef int32_t sint32;
typedef uint64_t uint64;
typedef int64_t sint64;
typedef float32_t float32;
typedef float64_t float64;
#endif
…
____________________________________________________________________________
message.h
Do we have to do changes in this file?
____________________________________________________________________________
topsocket.c
Do we have to do changes in this file?
____________________________________________________________________________
swapbytes.c
Do we have to do changes in this file?
____________________________________________________________________________
swapbytes.h
Do we have to do changes in this file?
____________________________________________________________________________
GenericSignal.h
…
template<> void
GenericSignal::PutValueBinary<SignalType::int16>( std::ostream&, size_t, size_t ) const;
template<> void
GenericSignal::PutValueBinary<SignalType::int32>( std::ostream&, size_t, size_t ) const;
template<> void
GenericSignal::PutValueBinary<SignalType::float24>( std::ostream&, size_t, size_t ) const;
template<> void
GenericSignal::PutValueBinary<SignalType::float32>( std::ostream&, size_t, size_t ) const;
template<> void
GenericSignal::PutValueBinary<SignalType::float64>( std::ostream&, size_t, size_t ) const;

template<> void
GenericSignal::GetValueBinary<SignalType::int16>( std::istream&, size_t, size_t );
template<> void
GenericSignal::GetValueBinary<SignalType::int32>( std::istream&, size_t, size_t );
template<> void
GenericSignal::GetValueBinary<SignalType::float24>( std::istream&, size_t, size_t );
template<> void
GenericSignal::GetValueBinary<SignalType::float32>( std::istream&, size_t, size_t );
template<> void
GenericSignal::GetValueBinary<SignalType::float64>( std::istream&, size_t, size_t );
…
____________________________________________________________________________
GenericSignal.cpp
I don’t know what it should be modified.

Fanny,

this is an excellent posting. I assume that for your particular application it is necessary to keep the double precision before submitting the data to the FieldTripBuffer, as you otherwise could just convert to single precision.

Reading through your code snippets you already got very far. A few additions to your question marks for the SignalType.cpp. The size of float64 is 8 bytes, hence the 64 bits. The conversionTable shows the permissible conversions between data types, i.e., whether the largest positive/negative number of one type still fits in the other data type. For float64 none of these conversions are permissible, while every other data type fits nicely into float64.

Can you please try to compile and test your changes to see whether they achieve the intended results. If it compiles and does not crash, please verify the double precision to ensure that there is no casting or conversion happening somewhere in the code. If it does not work, please report back and I will see how we can debug the issue.

Once the modifications work without any side effect, I can help with committing the changes to the BCI2000 source code on SVN.

____________________________________________________________________________
SignalType.cpp
Regards, Peter

Hi Peter,

Thank you very much for your reply and your help. I did the modifications you told me. However, it doesn't work because some modifications have to be done in GenericSignal.cpp and I am a little lost to implement codes about the reading of float64 in memory (in GenericSignal.cpp)
I did the followed changes in GenericSignal.cpp (and what I supposed to write to read float64 in memory):

...
ostream&
GenericSignal::WriteBinary( ostream& os ) const
{
Type().WriteBinary( os );
LengthField<2> channelsField( Channels() ),
elementsField( Elements() );
channelsField.WriteBinary( os );
elementsField.WriteBinary( os );
switch( Type() )
{
case SignalType::int16:
for( int i = 0; i < Channels(); ++i )
for( int j = 0; j < Elements(); ++j )
PutValueBinary<SignalType::int16>( os, i, j );
break;

case SignalType::float24:
for( int i = 0; i < Channels(); ++i )
for( int j = 0; j < Elements(); ++j )
PutValueBinary<SignalType::float24>( os, i, j );
break;

case SignalType::float32:
for( int i = 0; i < Channels(); ++i )
for( int j = 0; j < Elements(); ++j )
PutValueBinary<SignalType::float32>( os, i, j );
break;

case SignalType::int32:
for( int i = 0; i < Channels(); ++i )
for( int j = 0; j < Elements(); ++j )
PutValueBinary<SignalType::int32>( os, i, j );
break;

case SignalType::float64:
for( int i = 0; i < Channels(); ++i )
for( int j = 0; j < Elements(); ++j )
PutValueBinary<SignalType::float64>( os, i, j );
break;

default:
os.setstate( os.failbit );
}
return os;
}

istream&
GenericSignal::ReadBinary( istream& is )
{
SignalType type;
LengthField<2> channels,
elements;
type.ReadBinary( is );
channels.ReadBinary( is );
elements.ReadBinary( is );
SetProperties( SignalProperties( channels, elements, type ) );
switch( Type() )
{
case SignalType::int16:
for( int i = 0; i < Channels(); ++i )
for( int j = 0; j < Elements(); ++j )
GetValueBinary<SignalType::int16>( is, i, j );
break;

case SignalType::float24:
for( int i = 0; i < Channels(); ++i )
for( int j = 0; j < Elements(); ++j )
GetValueBinary<SignalType::float24>( is, i, j );
break;

case SignalType::float32:
for( int i = 0; i < Channels(); ++i )
for( int j = 0; j < Elements(); ++j )
GetValueBinary<SignalType::float32>( is, i, j );
break;

case SignalType::int32:
for( int i = 0; i < Channels(); ++i )
for( int j = 0; j < Elements(); ++j )
GetValueBinary<SignalType::int32>( is, i, j );
break;

case SignalType::float64:
for( int i = 0; i < Channels(); ++i )
for( int j = 0; j < Elements(); ++j )
GetValueBinary<SignalType::float64>( is, i, j );
break;

default:
is.setstate( is.failbit );
}
return is;
}

ostream&
GenericSignal::WriteValueBinary( ostream& os, size_t inChannel, size_t inElement ) const
{
switch( Type() )
{
case SignalType::int16:
PutValueBinary<SignalType::int16>( os, inChannel, inElement );
break;

case SignalType::float24:
PutValueBinary<SignalType::float24>( os, inChannel, inElement );
break;

case SignalType::float32:
PutValueBinary<SignalType::float32>( os, inChannel, inElement );
break;

case SignalType::int32:
PutValueBinary<SignalType::int32>( os, inChannel, inElement );
break;

case SignalType::float64:
PutValueBinary<SignalType::float64>( os, inChannel, inElement );
break;

default:
os.setstate( os.failbit );
}
return os;
}

istream&
GenericSignal::ReadValueBinary( istream& is, size_t inChannel, size_t inElement )
{
switch( Type() )
{
case SignalType::int16:
GetValueBinary<SignalType::int16>( is, inChannel, inElement );
break;

case SignalType::float24:
GetValueBinary<SignalType::float24>( is, inChannel, inElement );
break;

case SignalType::float32:
GetValueBinary<SignalType::float32>( is, inChannel, inElement );
break;

case SignalType::int32:
GetValueBinary<SignalType::int32>( is, inChannel, inElement );
break;

case SignalType::float64:
GetValueBinary<SignalType::float64>( is, inChannel, inElement );
break;

default:
is.setstate( is.failbit );
}
return is;
}
...
I assume the following code is to read data type in memory ? I don't know if what I wrote is correct...

template<>
void
GenericSignal::PutValueBinary<SignalType::int16>( std::ostream& os, size_t inChannel, size_t inElement ) const
{
int value = static_cast<int>( Value( inChannel, inElement ) );
os.put( value & 0xff ).put( value >> 8 );
}

template<>
void
GenericSignal::GetValueBinary<SignalType::int16>( std::istream& is, size_t inChannel, size_t inElement )
{
signed short value = is.get();
value |= is.get() << 8;
SetValue( inChannel, inElement, value );
}

template<>
void
GenericSignal::PutValueBinary<SignalType::int32>( std::ostream& os, size_t inChannel, size_t inElement ) const
{
signed int value = static_cast<signed int>( Value( inChannel, inElement ) );
PutLittleEndian( os, value );
}

template<>
void
GenericSignal::GetValueBinary<SignalType::int32>( std::istream& is, size_t inChannel, size_t inElement )
{
signed int value = 0;
GetLittleEndian( is, value );
SetValue( inChannel, inElement, value );
}

template<>
void
GenericSignal::PutValueBinary<SignalType::float24>( std::ostream& os, size_t inChannel, size_t inElement ) const
{
GenericSignal::ValueType value = Value( inChannel, inElement );
int mantissa,
exponent;
if( value == 0.0 )
{
mantissa = 0;
exponent = 1;
}
else
{
exponent = static_cast<int>( ::ceil( ::log10( ::fabs( value ) ) ) );
mantissa = static_cast<int>( value / ::pow( 10.0, exponent ) ) * 10000;
exponent -= 4;
}
os.put( mantissa & 0xff ).put( mantissa >> 8 );
os.put( exponent & 0xff );
}

template<>
void
GenericSignal::GetValueBinary<SignalType::float24>( std::istream& is, size_t inChannel, size_t inElement )
{
signed short mantissa = is.get();
mantissa |= is.get() << 8;
signed char exponent = is.get();
SetValue( inChannel, inElement, mantissa * ::pow( 10.0, exponent ) );
}

template<>
void
GenericSignal::PutValueBinary<SignalType::float32>( std::ostream& os, size_t inChannel, size_t inElement ) const
{
bciassert( numeric_limits<float>::is_iec559 && sizeof( uint32_t ) == sizeof( float ) );
union { float f; uint32_t i; } value;
value.f = static_cast<float>( Value( inChannel, inElement ) );
PutLittleEndian( os, value.i );
}

template<>
void
GenericSignal::GetValueBinary<SignalType::float32>( std::istream& is, size_t inChannel, size_t inElement )
{
bciassert( numeric_limits<float>::is_iec559 && sizeof( uint32_t ) == sizeof( float ) );
union { float f; uint32_t i; } value;
GetLittleEndian( is, value.i );
try
{
SetValue( inChannel, inElement, value.f );
}
catch( ... )
{
is.setstate( is.failbit );
}
}

template<>
void
GenericSignal::PutValueBinary<SignalType::float64>( std::ostream& os, size_t inChannel, size_t inElement ) const
{
bciassert( numeric_limits<double>::is_iec559 && sizeof( uint32_t ) == sizeof( double) );
union { double f; uint32_t i; } value;
value.f = static_cast<double>( Value( inChannel, inElement ) );
PutLittleEndian( os, value.i );
}

template<>
void
GenericSignal::GetValueBinary<SignalType::float64>( std::istream& is, size_t inChannel, size_t inElement )
{
bciassert( numeric_limits<double>::is_iec559 && sizeof( uint32_t ) == sizeof( double) );
union { double f; uint32_t i; } value;
GetLittleEndian( is, value.i );
try
{
SetValue( inChannel, inElement, value.f );
}
catch( ... )
{
is.setstate( is.failbit );
}
}


template<typename T>
void
GenericSignal::PutLittleEndian( std::ostream& os, const T& inValue )
{
T value = inValue;
for( size_t i = 0; i < sizeof( T ); ++i )
{
os.put( value & 0xff );
value >>= 8;
}
}

template<typename T>
void
GenericSignal::GetLittleEndian( std::istream& is, T& outValue )
{
outValue = 0;
for( size_t i = 0; i < sizeof( T ); ++i )
outValue |= is.get() << ( i * 8 );
}

Moreover, I don't understand why a float64 value is considered as a uint32_t in message.h :
#define DATATYPE_CHAR (UINT32_T)0
#define DATATYPE_UINT8 (UINT32_T)1
#define DATATYPE_UINT16 (UINT32_T)2
#define DATATYPE_UINT32 (UINT32_T)3
#define DATATYPE_UINT64 (UINT32_T)4
#define DATATYPE_INT8 (UINT32_T)5
#define DATATYPE_INT16 (UINT32_T)6
#define DATATYPE_INT32 (UINT32_T)7
#define DATATYPE_INT64 (UINT32_T)8
#define DATATYPE_FLOAT32 (UINT32_T)9
#define DATATYPE_FLOAT64 (UINT32_T)10

Indeed, the data I want to submit to the FieldTripBuffer is the data type number 10 that is to say "DATATYPE_FLOAT64 (UINT32_T)10".

Thank you for your help,

Fanny

Fanny,

I have noticed that in your added PutValueBinary/GetValueBinary you only copy half of the data. The changes in red below should fix this issue.

Your changes in message.h are correct. No need to worry about the UINT32_T type, as this is applies to the codes and not to the values.

Can you please apply these modification and report back?

Regards, Peter


...

template<>
void
GenericSignal::PutValueBinary<SignalType::float64>( std::ostream& os, size_t inChannel, size_t inElement ) const
{
bciassert( numeric_limits<double>::is_iec559 && sizeof( uint64_t ) == sizeof( double) );
union { double f; uint64_t i; } value;
value.f = static_cast<double>( Value( inChannel, inElement ) );
PutLittleEndian( os, value.i );
}

template<>
void
GenericSignal::GetValueBinary<SignalType::float64>( std::istream& is, size_t inChannel, size_t inElement )
{
bciassert( numeric_limits<double>::is_iec559 && sizeof( uin64_t ) == sizeof( double) );
union { double f; uint64_t i; } value;
GetLittleEndian( is, value.i );
try
{
SetValue( inChannel, inElement, value.f );
}
catch( ... )
{
is.setstate( is.failbit );
}
}

...

Peter,

The modifications work ! Thank you very much for your help.
How can I commit the changes to the BCI2000 source code ? And how can I change the functional description (with my name) about FieldTripBufferSource?

Thank you,

Fanny

Fanny,

I am glad to hear that the modifications work for you. For the next two steps:

1) Documentation: You should be able to edit the relevant BCI2000 wiki pages [1,2] after logging in with your BCI2000 account.

2) Code Commit: We typically enable write access for developers for certain branches of the source code. In your case this would be the two FieldTripBuffer* folders in the src directory. However, for your modifications you also needed to also commit a few changes to the core, which we try to keep in the hands of our core developer. For that reason I would suggest that you send me the modified files and I will check them in. If you like to help maintain the FieldTripBuffer in BCI2000, I can enable write access for your BCI2000 account to these folders.

Regards, Peter

[1] http://www.bci2000.org/wiki/index.php/C ... TripBuffer
[2] http://www.bci2000.org/wiki/index.php/C ... fferSource