EventOffset out of range in P300 Memory Game application

Forum for discussion on different user applications
Locked
corbitv
Posts: 5
Joined: 18 Jul 2011, 09:23

EventOffset out of range in P300 Memory Game application

Post by corbitv » 22 May 2012, 11:06

My research team has created a memory game application for BCI2000 using BCPy2000, such that the matrix contains images of "cards" instead of letters as in the P3Speller. The matrix size was 3x4.

The first few times the game was run, it operated fine, though with some inaccuracies in choices.
After the first couple times of use, however, we started getting error messages halfway through gameplay. A summary of the error messages seen in the BCI2000 System Log and in PythonApp is below:

Game 1: After ~8 picks-->
BCI2000 System Log: DateIOFilter: EvaluateTiming: Roundtrip time consistently exceeds block duration (currently 153.59%)
The whole program froze, so we closed and restarted.

Game 2: After very first card pick -->
BCI2000 System Log: DataIOFilter: EvaluateTiming: Roundtrip time approaches block duration (currently 75.16%)
PythonApp:
Prompt: Card Selected: Card-12
Warning: EventOffset out of range (offset=1670, to be coded in 10-bit state)
Prompt: Next Card
Warning: multiple phase transitions per packet

The game continued to run, but after ~32 picks, the matrix stopped flashing, so we suspended the game. This message was showing:
PythonApp:
2012-04-24 18:02:19 - run stopped
data file: C:\Documents and Settings\This PC\Desktop\BCI2000\data\Memory Game Study\A\A002\A002\AS002R03.dat

debug warnings in self.db
{'BadEventOffsets': 34, 'state collision in EventOffset': 2220, "Multiple Transitions"; 3}




It appeared that accuracy decreased as the game proceeded, correlating with the appearance of these errors.

Any help you could give as to what kind of issue this is or how to fix it would be great.
Thank you!

jhill
Posts: 31
Joined: 17 Nov 2009, 15:15

Re: EventOffset out of range in P300 Memory Game application

Post by jhill » 22 May 2012, 12:47

The first thing to look at would be:
http://bci2000.org/downloads/BCPy2000/Timing.html

The most important point to understand is that you as a programmer, whether you're operating in C++, Python or Matlab, have the power to mess up BCI2000's timing performance to an arbitrary extent. The cascade of Process() calls (in the C++, Python or Matlab code of each filter in the chain) operates synchronously, and the whole cascade must finish by the time the next SampleBlock is due to arrive: otherwise the real-time constraint is broken. BCI2000's Timing window allows you to assess how close you are to breaking the realtime constraint—when you're close, you'll get a warning, and when you really break it, you get that fatal error message from the source module ("Roundtrip time consistently exceeds....").

Typically you'll break real-time if you create a Process() call that does too much processing, or does it too inefficiently. The simplest demonstration of how to break real-time would be to implement a Process() method that just sleep()s for longer than the SampleBlock duration. But note that the same constraints may apply indirectly via other threads and processes: in BCPy2000, that means if you use too much CPU in Phases(), Transition() or Frame(), then there may be insufficient CPU resources for Process() to complete whatever it is supposed to do by the the deadline.

"EventOffset out of range" is also a sign that calls are blocking for too long (possibly in your Transition() implementation) and "multiple phase transitions per packet" may be a knock-on effect from that (or it may not, but let's fix the biggest problem first).

The first thing to do is to profile your code (maybe using the crude simple approach of querying self.PrecisionTime() before and after each line and printing the differences ). Find out what is taking too long. Then, if you absolutely must do something asynchronous (something that takes longer than a small proportion of a SampleBlock duration) then you have to create a new thread in Python and manage it accordingly.

corbitv
Posts: 5
Joined: 18 Jul 2011, 09:23

Re: EventOffset out of range in P300 Memory Game application

Post by corbitv » 04 Jun 2012, 12:14

Thanks for the suggestions!

We tried to time the Transition process, and it showed up as being 0 most of the time.

We investigated the "Timing" graph while the program was running, and noticed that at the end of each flashing sequence, there was a huge spike in all three lines (Block, Roundtrip, and Stimulus). Our application is a memory game, so the user picks a "card", and then the flashing sequence happens again, the user picks a second card, and the program determines if a match was made or not made. If a match was not made, the program replaces the two selections ("picked cards") with the "card back" image. When this happens (the two replacements), an even bigger spike occurs. (screenshot attached)
When a pair is found (and therefore no images need to be replaced), no spike occurs.
We're using VisionEgg.Textures.Texture(image) to load the images.

We read on the BCI2000 Wiki that the Roundtrip time needs to stay below Block time (which makes sense, considering the error messages we've received). When the spikes occur, all three lines spike, so seemingly Roundtrip goes above Block.

We're thinking, then, that the spikes are correlated with the use of images, and thus that the slow processing has something to do with our images.

Does that seem plausible? Would replacing our images with less complex or lower resolution images be a solution? Is there another solution that would allow us to keep the images we're using now?


Thank you again!
Attachments
stimulus_screenshot.JPG

boulay
Posts: 382
Joined: 25 Dec 2011, 21:14

Re: EventOffset out of range in P300 Memory Game application

Post by boulay » 04 Jun 2012, 22:52

I have only done a small amount of game programming but in my experience it is better to have all of your visual assets in-memory all the time and then simply toggle whether or not they are visible as needed.

Start with 12 image assets: the card backs.

Then add 12 more assets: the card faces and each one will be positioned the same as the card backs. Make these 12 assets not visible by default. Then when a card is selected you simply make the card visible. A few ways to handle visibility would be to use the z-value, alpha-value, or some other visibility toggle.

Be sure to load and position all of the assets during the Initialize hook. Then you will only need to change the visibility of a single card face during Process or Transition. Personally I would prefer to use Transition for this. I try to only use Process for things that depend on block-to-block calculations and saving some calculated value into a state variable which I would then use in Transition.

If, in the future, you ever want to gradually change the alpha value of your card faces as a form of continuous feedback then you would be better off controlling the alpha value in the Process hook.

jhill
Posts: 31
Joined: 17 Nov 2009, 15:15

Re: EventOffset out of range in P300 Memory Game application

Post by jhill » 05 Jun 2012, 03:09

When following chad's recommendation above, ie to preload all images, I've never seen a huge spike like that caused purely by hiding one image and showing another—even when they're big images. But you can always test your hypothesis by substituting a smaller image. Generally it is easier to develop by moving steadily forward from simple cases that work, and testing frequently before moving forward to each more-complicated or potentially more resource-hungry step.

Re-reading what you've written, I suspect you're doing something in Process, under some particular contingency in the logic of the game, that is taking more time than you should ever spend in Process. Profile and debug Process() to track down the problem.

corbitv
Posts: 5
Joined: 18 Jul 2011, 09:23

Re: EventOffset out of range in P300 Memory Game application

Post by corbitv » 06 Jun 2012, 13:29

Thank you very much for all the suggestions and advice.
We replaced with smaller images and the spikes went away. There are some erratic spikes (both in Block and in Roundtrip), but they don't seem to affect the program so we're letting it go.

About you what said about Process ... I don't believe we (I didn't actually code the program, a past student from my research team did) adjusted anything in the Process portion of the code. I think they used a general BCPy2000 code and simply adjusted what was necessary for our game. This is all the Process part of the code says:

def Process(self, sig):
# process the new signal packet
x = sig[0,0]
stimulusCodeRes = self.states['StimulusCodeRes']
if stimulusCodeRes != 0:
self.validStimCode = True
self.classRes.append([stimulusCodeRes, x])
else:
self.validStimCode = False


I was wondering ... we are having severe accuracy problems (0% accuracy) with our program, even after the timing issues were fixed. Would this "Process" aspect of the code have anything to do with that? If not, that is fine, I will post another topic about the accuracy issues with further explanation.

Locked

Who is online

Users browsing this forum: No registered users and 1 guest