Writing variables to file

[gsudre]

Hello,

I was wondering what is the recommended way to save variables that change during runtime for offline analysis. More specifically, I have a few float variables that get updated per block, and I would like to save them for offline analysis.

I tried saving them to the State vector but it is limited to integers. I thought about saving them to a separate file or buffering them and saving at the end of a trial or run, but I wonder what will be the implication to the system performance by doing that.

Thank you,

Gus

[mellinger]

Gus,

there is no "official" way to do this. Basically, you have the following options:
1) Transform your float values into an integer range suited to represent all relevant information.
2) Treat the binary representation as an integer value, and store it as such.

Regards,
Juergen

[gsudre]

Juergen,

Thanks for the answer. It seems like the second option would work better for us, because our range of values can be big (e.g. 0.002 to 2000). Do you have a good way to do the transfer from float to bit then to int, in C++? All the code I've seen and tried so far has not been successful to represent the smaller floats in bits.

Also, do you have any idea of how much the performance would be affected if we saved these values to a file in the HD? Maybe at the end of every trial, or even run...

Thanks,

Gus

[mellinger]

Gus,

to get access to a float value's binary representation, use a pointer cast:

Code: Select all

 float value = M_PI;
 assert( sizeof( int ) == sizeof( value ) );
 int& intValue = *reinterpret_cast<int*>( &value );
 bool fifthBit = ( intValue >> 4 ) & 1;
 char secondByte = ( intValue >> 8 ) & 0xff;

Note that, when transferring data between machines, you will need to make sure that float values are represented identically on both sides. The type named "float" uses IEEE 754 representation on most machines.

I don't know whether I understand your performance/timing question. When writing data to an external file, performance is certainly not the issue; however, you will probably find it cumbersome to deal with information stored across multiple files.
When storing the values in BCI2000 state variables, they will be present in the data files themselves, but they will be stored per sample, possibly wasting quite some space. Still, given the large amount of storage available today, this might not be a concern.

Regards,
Juergen

[gsudre]

Juergen,

Thanks for the code. It worked great when I tested it, but when I try to use it with BCI2000, I get an error related to the State vector length.

I tracked it down to the function statevector::setstatevalue,. Because I need to output a float, I converted its binary representation to an int to be stored in the state vector (like you suggested before). To do that, I declared a state vector variable with 32 bits length. I thought that would be OK because it says in the wiki that this is the limit for state vector variables. However, when I do that the first if statement in the method mentioned above becomes true and the module is terminated.

Do you have any ideas of how to fix that?

Thanks,

Gus

[mellinger]

Gus,

make sure that, in your state definition, you allocate 32 bits for the state variable in question:

Code: Select all

MyState 32 0 0 0

If the error message persists, this is a bug, and will be fixed.

Regards,
Juergen

[gsudre]

Juergen,

Yes, that's exactly what I have in my code. Please let us know when it's fixed. For now, I thought about just commenting out this if statement. Can you think of any other implications it will have in the overall funtioning of the program?

Thanks,

Gus

[mellinger]

Gus,

I thought about just commenting out this if statement. Can you think of any other implications it will have in the overall funtioning of the program?

the check is there to avoid loss of data from inappropriate state variable definitions. Removing it will not have any side effects.

Thanks for reporting this issue,
Juergen

[mellinger]

Gus,

due to the wonders of integer arithmetic, there was actually a bug in the code that checks whether a state is wide enough to hold a given value.
I replaced it with a more robust formulation.

Regards,
Juergen