Hello,
I am using RDAClient as a source module. It seems to work fine, except that quite often (say 1-3 times a session), just after having "resumed" to start a new run, BCI2000 complains about the number of channels being less than expected (8 rather than the actual 60). Of course the configuration is the very same that worked for the previous run.
Before I post a complete bug report, I wonder if someone else has already experienced this behavior and has maybe already found a solution or a workaround.
Also, suggestions on what would I should not be missing in my bug report would be appreciated.
Best
febo
RDAClient
-
febo
- Posts: 9
- Joined: 17 Sep 2003, 03:24
Dear Juergen,
thanks for your reply.
I am pretty sure that there are no more than one instance of the Recorder running. Since you mentioned, I will check with the Task Manager, but there is no evidence of it in the Taskbar.
Also, we usually start the Recorder, check the impedences and let it run minimized until the end of the session (unless the number of crashes of the kind we are talking about suggests to restart everything). It happened that we have changed the display filters in the Vision workspace between runs, but I cannot tell if this increases the likeliness of errors.
Apart from the specific answers, I infer from your questions that you have never faced this issue, so I shoud go through some specific test to guess what is going wrong.
Please find below the session log and the "Source" section of the PRM file. Of course I do not understand the exact meaning of the error message (since I have not gone through the RDAClient code, at least so far), but it seems to me that the cause could be in the Recorder software, which restarts sending out less channels that it did during the previous run. Could you suggest me any procedure to check this or other hypotheses, so that I can try to help it out myself or simply ask the right question to the right person?
Thanks
febo
thanks for your reply.
I am pretty sure that there are no more than one instance of the Recorder running. Since you mentioned, I will check with the Task Manager, but there is no evidence of it in the Taskbar.
Also, we usually start the Recorder, check the impedences and let it run minimized until the end of the session (unless the number of crashes of the kind we are talking about suggests to restart everything). It happened that we have changed the display filters in the Vision workspace between runs, but I cannot tell if this increases the likeliness of errors.
Apart from the specific answers, I infer from your questions that you have never faced this issue, so I shoud go through some specific test to guess what is going wrong.
Please find below the session log and the "Source" section of the PRM file. Of course I do not understand the exact meaning of the error message (since I have not gone through the RDAClient code, at least so far), but it seems to me that the cause could be in the Recorder software, which restarts sending out less channels that it did during the previous run. Could you suggest me any procedure to check this or other hypotheses, so that I can try to help it out myself or simply ask the right question to the right person?
Thanks
febo
Code: Select all
=== STARTSESSIONLOG =================================
07/06/2004 15:02:21 - BCI2000 started
07/06/2004 15:02:22 - Executing script after all modules connected ...
07/06/2004 15:02:22 - Could not open script file C:\data\Bci2000\src\test.scr ...
07/06/2004 15:03:49 - Operator set configuration
07/06/2004 15:03:50 - User Application confirmed new parameters ...
07/06/2004 15:03:50 - Signal Processing confirmed new parameters ...
07/06/2004 15:03:50 - Source confirmed new parameters ...
07/06/2004 15:03:54 - Operator started operation
07/06/2004 15:06:57 - Operator suspended operation
07/06/2004 15:08:44 - Operator resumed operation
07/06/2004 15:08:44 - RDAClientADC::Preflight: The SoftwareCh parameter must equal the number of channels in the recording software plus one (9)
07/06/2004 15:08:44 - RDAClientADC::Preflight: The number of values in the SourceChOffset parameter must equal the number of channels in the recording software plus one (9)
07/06/2004 15:08:44 - RDAClientADC::Preflight: The number of values in the SourceChGain parameter must equal the number of channels in the recording software plus one (9)
07/06/2004 15:08:44 - RDAClientADC::Preflight: The SourceChGain values for the first 8 channels must match the channel resolutions settings in the recording software (1000)
07/06/2004 15:08:44 - RDAClientADC::Preflight: The SamplingRate parameter must match the setting in the recording software (100)
07/06/2004 15:08:44 - Error in Source!
07/06/2004 15:08:44 - Error (re)configuring Source
=== ENDSESSIONLOG ==================================
Code: Select all
=== STARTPRM =======================================
Source int SampleBlockSize= 8 20 1 128 // the number of samples transmitted at a time, incoming blocks are always 40ms
Source int SamplingRate= 200 250 1 4000 // the sample rate
Source int SoftwareCh= 61 33 1 129 // the number of digitized and stored channels including marker channel
Source int TransmitCh= 58 4 1 128 // the number of transmitted channels
Source intlist TransmitChList= 58 15 16 17 18 19 20 21 23 24 25 26 27 28 29 34 35 36 37 38 39 40 1 2 3 4 5 6 7 8 9 10 11 12 13 14 22 30 33 41 42 43 44 45 46 47 48 49 50 51 52 54 56 57 58 59 60 53 55 1 1 128 // list of transmitted channels (# of channels MUST equal TransmitCh)
Source string HostName= localhost 0 0 0 // the name of the host to connect to
=== SENDPRM ========================================
-
mellinger
- Posts: 1341
- Joined: 12 Feb 2003, 11:06
Febo,
in the Recorder software, there is an option that allows you to switch between a "demo" input (sine waves with a frequency proportional to channel number) and the true A/D input. To me, the values mentioned in the error message (8 Ch, 100 Hz Sampling rate, 1000 nV/AD unit) look like the ones sent for the demo input.
To find out whether the error is on the VisionRecorder or on the RDAClient side, it would be helpful to know answers to some further questions:
If the problem happens and you restore the Recorder window, what does the signal look like? Is it EEG or the test signal?
If you restart BCI2000 once the error occurred, and not the Recorder program, will the error persist?
Which version of the VisionRecorder software are you using?
What version of MS Windows are you using?
Best,
Jürgen
in the Recorder software, there is an option that allows you to switch between a "demo" input (sine waves with a frequency proportional to channel number) and the true A/D input. To me, the values mentioned in the error message (8 Ch, 100 Hz Sampling rate, 1000 nV/AD unit) look like the ones sent for the demo input.
To find out whether the error is on the VisionRecorder or on the RDAClient side, it would be helpful to know answers to some further questions:
If the problem happens and you restore the Recorder window, what does the signal look like? Is it EEG or the test signal?
If you restart BCI2000 once the error occurred, and not the Recorder program, will the error persist?
Which version of the VisionRecorder software are you using?
What version of MS Windows are you using?
Best,
Jürgen
-
febo
- Posts: 9
- Joined: 17 Sep 2003, 03:24
Dear Juergen,
I tend to exclude that the simulated amplifiers option turns on on by its own. At least I can see no sign that suggests this on the Recorder window, which keeps on showing the real EEG potentials, regardless of the BCI2000 complaints.
I could find no deterministic behavior regarding the need to restart the Recorder in order to have the BCI2000 working.
Actually, what happens is that most of the time it is enough to restart the BCI2000 in order to restore functionality. When it crashes two times in a row (meaning that functionality could not be restored by simply restarting BCI2000) we restart the Recorder. This happens about one time every 4-6 crashes. Seldom, we obtain no advantage from restarting the Recorder, so we restart Windows.
The procedure is quite crude, but is the fastest we could find to avoid the subject idling for a long time while we get the system working.
We use the BrainVision Recorder v1.02, with Windows 2000 (no SP, I think), running on a Pentium 4 (the speed is around 2GHz, I do not recall) with 512Mb RAM.
The amplifiers and the PCI board have been recently (May) firmware updated by the manifacturer.
Is there any Debug version of the Source module that we can run in order to have more debug information on the log window? This would of course help me very much in tackling the error.
Let me put forward another possibility. What happened to me sometimes running a RDAClient under Matlab is that I did not receive a "Info" packed as a first packet, but rather a "Data" packet. Of course this unexpectedly crashed my program until I found out what was going on. I actually attributed this behavior to some delay introduced by the Matlab or to the roughness of the first implementations of my program, so I did not mention it before. Is it possible that if the Recorder misses sending the Info packed to your Source module before the data, the latter will assume some default parameters (i.e., 8 chans, 100Hz, etc). If so, would it be possible to send out a line when the Source connects to the server and receives the Info packet, so that we can test this hypothesis.
Best
febo
I tend to exclude that the simulated amplifiers option turns on on by its own. At least I can see no sign that suggests this on the Recorder window, which keeps on showing the real EEG potentials, regardless of the BCI2000 complaints.
I could find no deterministic behavior regarding the need to restart the Recorder in order to have the BCI2000 working.
Actually, what happens is that most of the time it is enough to restart the BCI2000 in order to restore functionality. When it crashes two times in a row (meaning that functionality could not be restored by simply restarting BCI2000) we restart the Recorder. This happens about one time every 4-6 crashes. Seldom, we obtain no advantage from restarting the Recorder, so we restart Windows.
The procedure is quite crude, but is the fastest we could find to avoid the subject idling for a long time while we get the system working.
We use the BrainVision Recorder v1.02, with Windows 2000 (no SP, I think), running on a Pentium 4 (the speed is around 2GHz, I do not recall) with 512Mb RAM.
The amplifiers and the PCI board have been recently (May) firmware updated by the manifacturer.
Is there any Debug version of the Source module that we can run in order to have more debug information on the log window? This would of course help me very much in tackling the error.
Let me put forward another possibility. What happened to me sometimes running a RDAClient under Matlab is that I did not receive a "Info" packed as a first packet, but rather a "Data" packet. Of course this unexpectedly crashed my program until I found out what was going on. I actually attributed this behavior to some delay introduced by the Matlab or to the roughness of the first implementations of my program, so I did not mention it before. Is it possible that if the Recorder misses sending the Info packed to your Source module before the data, the latter will assume some default parameters (i.e., 8 chans, 100Hz, etc). If so, would it be possible to send out a line when the Source connects to the server and receives the Info packet, so that we can test this hypothesis.
Best
febo
-
mellinger
- Posts: 1341
- Joined: 12 Feb 2003, 11:06
Dear Febo,
thank you very much for your detailed description of the problems you experienced with the VisionRecorder's interpretation of the RDA protocol.
I checked the RDA client code for what it would do under the circumstances you described, and found that it would act exactly as you suggested, producing the output you obtained.
Not checking for this type of error condition but instead assuming some defaults was clearly an error in the RDA client code, leading to the misleading error messages -- thank you for reporting it.
I'll send you a current version of the RDA client source module that will also report more specific errors if the connection with the VisionRecorder fails.
We are using version 1.01b of the VisionRecorder, and have not experienced the protocol violation you report for 1.02. If your hardware runs with 1.01b, you might try this until they get 1.02 fixed at BrainProducts -- it is their own protocol specification that is violated by their product, so they should fix it.
From your experiences with the Matlab client, can you tell whether there will always be an info packet after some time? This would be helpful to design a work-around if 1.01b is no option for you.
Best,
Juergen
thank you very much for your detailed description of the problems you experienced with the VisionRecorder's interpretation of the RDA protocol.
I checked the RDA client code for what it would do under the circumstances you described, and found that it would act exactly as you suggested, producing the output you obtained.
Not checking for this type of error condition but instead assuming some defaults was clearly an error in the RDA client code, leading to the misleading error messages -- thank you for reporting it.
I'll send you a current version of the RDA client source module that will also report more specific errors if the connection with the VisionRecorder fails.
We are using version 1.01b of the VisionRecorder, and have not experienced the protocol violation you report for 1.02. If your hardware runs with 1.01b, you might try this until they get 1.02 fixed at BrainProducts -- it is their own protocol specification that is violated by their product, so they should fix it.
From your experiences with the Matlab client, can you tell whether there will always be an info packet after some time? This would be helpful to design a work-around if 1.01b is no option for you.
Best,
Juergen
-
febo
- Posts: 9
- Joined: 17 Sep 2003, 03:24
Dear Juergen,
So, the good news is that it seems that we have now some information about when my error is generated.
The bad news is that it does not seem to be a systematic error, since we have also been using version 1.01b for a long time and we hoped to overcome the bug by upgrading to 1.02.
I can try a downgrade to 1.01b to make some more systematic test, but I am not very optimistic.
In my Matlab implementation I decided to abort the connection after such an error, and to retry the connection. So I cannot help you with your question, but it is a good point and I can try to play with it in the next days.
Anyway, a possible workaround would be to do what we usually do by hand - disconnect from the socket and reconnect, for a number of times, before complaining (and having the user restart all things). I think anyway that I should investigate some more to suggest at least a smarter workaround.
The most discouraging thing for my lab has come up in the last two days (which have been quite busy for the system). Sometimes in the second half of the training session, the BCI2000 reported a loss of connection with the Vision.
In my opinion, this actually moves the focus to the particular installation and on the operating system, rather than on the software packages. In this sense I look forward very much to having the possibility of getting some enhanced log. This way I could send it to the Brainproducts and ask for their support to fix the installation.
Best
febo
So, the good news is that it seems that we have now some information about when my error is generated.
The bad news is that it does not seem to be a systematic error, since we have also been using version 1.01b for a long time and we hoped to overcome the bug by upgrading to 1.02.
I can try a downgrade to 1.01b to make some more systematic test, but I am not very optimistic.
In my Matlab implementation I decided to abort the connection after such an error, and to retry the connection. So I cannot help you with your question, but it is a good point and I can try to play with it in the next days.
Anyway, a possible workaround would be to do what we usually do by hand - disconnect from the socket and reconnect, for a number of times, before complaining (and having the user restart all things). I think anyway that I should investigate some more to suggest at least a smarter workaround.
The most discouraging thing for my lab has come up in the last two days (which have been quite busy for the system). Sometimes in the second half of the training session, the BCI2000 reported a loss of connection with the Vision.
In my opinion, this actually moves the focus to the particular installation and on the operating system, rather than on the software packages. In this sense I look forward very much to having the possibility of getting some enhanced log. This way I could send it to the Brainproducts and ask for their support to fix the installation.
Best
febo
-
febo
- Posts: 9
- Joined: 17 Sep 2003, 03:24
Dear Juergen,
I disappered for a while since I left to a conference (where I am now) and I could not directly follow the problem any more.
What happened before I left is that we could reproduce the crash on a "clean" machine.
We installed the Vision, the BCI2000, chose to use simulated amplifiers, loaded the usual Recorder workspace, loaded the usual BCI2000 parameters. As soon as we run it, it crashed (usual message).
I could not make any more tests on other machines since I left the dongle in Rome, but al least this allows me to say that we can try to reproduce the problem.
Would you agree to follow the same procedure described above, in order to check if you can replicate the crash on a computer of your lab. The only thing you need is the dongle and tree files (batch, parameters and workspace) that do not come with the standard installation of the two softwares.
I have put these files in the repository of this forum, in the folder "febo".
The second news is that here at the conference I talked to Ingmar Gutberlet (BrainProducts) and I described him the problem. He said he has no past experience of lost packets on the server side. He tends to believe that it can be a matter of delays introduced by the operative system, that may lead the Client to behave weirdly unless these problems are directly addressed. Particularly, he warned from fetching data from a not completely initialized server. He experienced this kind of problems on Windows 2000 systems, which would fit with my case.
He has given me a fancier version (with respect to the one distributed in the CD) of the RDAClient Demo (executable plus source in Visual Basic), where all the issues he knows have been addressed. The file repository refused to upolad this file (2.8Mb?), so if you think you need it I can send it to you in some other way.
Tomorrow I will borrow their dongle and try to check on my laptop if I can reproduce the error on a WindowsXP machine.
Best
febo
I disappered for a while since I left to a conference (where I am now) and I could not directly follow the problem any more.
What happened before I left is that we could reproduce the crash on a "clean" machine.
We installed the Vision, the BCI2000, chose to use simulated amplifiers, loaded the usual Recorder workspace, loaded the usual BCI2000 parameters. As soon as we run it, it crashed (usual message).
I could not make any more tests on other machines since I left the dongle in Rome, but al least this allows me to say that we can try to reproduce the problem.
Would you agree to follow the same procedure described above, in order to check if you can replicate the crash on a computer of your lab. The only thing you need is the dongle and tree files (batch, parameters and workspace) that do not come with the standard installation of the two softwares.
I have put these files in the repository of this forum, in the folder "febo".
The second news is that here at the conference I talked to Ingmar Gutberlet (BrainProducts) and I described him the problem. He said he has no past experience of lost packets on the server side. He tends to believe that it can be a matter of delays introduced by the operative system, that may lead the Client to behave weirdly unless these problems are directly addressed. Particularly, he warned from fetching data from a not completely initialized server. He experienced this kind of problems on Windows 2000 systems, which would fit with my case.
He has given me a fancier version (with respect to the one distributed in the CD) of the RDAClient Demo (executable plus source in Visual Basic), where all the issues he knows have been addressed. The file repository refused to upolad this file (2.8Mb?), so if you think you need it I can send it to you in some other way.
Tomorrow I will borrow their dongle and try to check on my laptop if I can reproduce the error on a WindowsXP machine.
Best
febo
-
febo
- Posts: 9
- Joined: 17 Sep 2003, 03:24
This is really bad! In fact I am using the executables extracted from the installation file dated May 20 (which is still the latest); moreover I am trying to do some experiments on my laptop, compiling the source code I downloaded on February 13 (obtaining the same kind of problems). So, I am afraid that it is not very sensitive on the version. Anyway, to answer your request, I will not send the executable, since the ones we run now in the lab are the very same that you can find on the ftp site. Of course, if you think that it can help anyway, I will try to upload them (hoping that the size will not be an obstacle). Reading my previous mail, I noticed that I described the crash as happening at the first run, which is what actually happened in the test I described, but is not the usual behavior. The fastest way to have the error reproduced, is to repeat starting the acquisition and quitting the program for a number of times, which is mostly below three-five.
My laptop runs WinXP, so the hint that I received by the guys at BrainProducts ("try to avoid Win2k") does not seem to apply.
What I did at the moment is to try to bypass the problem, plugging the actually utilized parameter values into the headerfile where the "guess" parameters are defined.
I left a bciout to check whether the problem is still there (and it is), but now it does not stop the acquisition. If the trick will also work on the acquisition computer in the lab, it will at least be a relief and will allow me to take all the time it needs to find out the real cause.
I actually have no more ideas of why the problem does only happen to me. I have tried it on at least three different computers, I have minimized the action that bring to the replication of the effect ... I can really see nothing meaningful that differs in our respective computers (I mean, I do not think that the language of Windows matters!)
I have tried to go the other way around, thus trying to get myself into the source code, but some parts requre an overview of the project I actually miss, and that I do not expect I can gain in a reasonable time with a reasobable effort. I can now trap the problem in the prefetch fuction, but I do not dare try to tackle the queue methods. Have you got any suggestion for me?
Cheers
febo
My laptop runs WinXP, so the hint that I received by the guys at BrainProducts ("try to avoid Win2k") does not seem to apply.
What I did at the moment is to try to bypass the problem, plugging the actually utilized parameter values into the headerfile where the "guess" parameters are defined.
I left a bciout to check whether the problem is still there (and it is), but now it does not stop the acquisition. If the trick will also work on the acquisition computer in the lab, it will at least be a relief and will allow me to take all the time it needs to find out the real cause.
I actually have no more ideas of why the problem does only happen to me. I have tried it on at least three different computers, I have minimized the action that bring to the replication of the effect ... I can really see nothing meaningful that differs in our respective computers (I mean, I do not think that the language of Windows matters!)
I have tried to go the other way around, thus trying to get myself into the source code, but some parts requre an overview of the project I actually miss, and that I do not expect I can gain in a reasonable time with a reasobable effort. I can now trap the problem in the prefetch fuction, but I do not dare try to tackle the queue methods. Have you got any suggestion for me?
Cheers
febo
-
mellinger
- Posts: 1341
- Joined: 12 Feb 2003, 11:06
Dear Febo,
the source code from Feb 13 is quite likely to be the same that is contained in the May 20 executable. Of course, if it is the version from the installation program, you need not send it to me.
Have you tried the RDAClient.exe version I sent you per e-mail about a week ago? It should give you a clearer error message if there occurs a protocol violation.
If sending a message via bciout will actually remove the problem then it might be related to network timing and multi-threading issues in the module code. I've put some work into the general module code recently, and there is a new version coming soon, i.e. within the next few days (watch the developer mailing list for my announcement).
I think it might be reasonable to wait until then, and try the latest version of BCI2000, before putting any more time into this issue.
Best regards,
Juergen
the source code from Feb 13 is quite likely to be the same that is contained in the May 20 executable. Of course, if it is the version from the installation program, you need not send it to me.
Have you tried the RDAClient.exe version I sent you per e-mail about a week ago? It should give you a clearer error message if there occurs a protocol violation.
If sending a message via bciout will actually remove the problem then it might be related to network timing and multi-threading issues in the module code. I've put some work into the general module code recently, and there is a new version coming soon, i.e. within the next few days (watch the developer mailing list for my announcement).
I think it might be reasonable to wait until then, and try the latest version of BCI2000, before putting any more time into this issue.
Best regards,
Juergen
-
febo
- Posts: 9
- Joined: 17 Sep 2003, 03:24
Dear Juergen,
Unfortunately I did not receive any mail wil the "debug version" of the RDAClient, so I could not test it. I will send you an email so that you can just reply.
I completely agree that since we now managed to bypass the problem, it might be worth checking whether the general improvement of the tool you are working on will fix the problem by itself. (By the way, the workaround was to provide the values we actually use for the parameters in the initialization, so that whatever the preflight reported, the correct values are always there; the bciout is just a memento to remind us how many crashes we should have born if we had not helped it.)
I will get back on the topic as soon as I test the new version, hopefully to say that everything is fine.
Best
febo
Unfortunately I did not receive any mail wil the "debug version" of the RDAClient, so I could not test it. I will send you an email so that you can just reply.
I completely agree that since we now managed to bypass the problem, it might be worth checking whether the general improvement of the tool you are working on will fix the problem by itself. (By the way, the workaround was to provide the values we actually use for the parameters in the initialization, so that whatever the preflight reported, the correct values are always there; the bciout is just a memento to remind us how many crashes we should have born if we had not helped it.)
I will get back on the topic as soon as I test the new version, hopefully to say that everything is fine.
Best
febo
-
mellinger
- Posts: 1341
- Joined: 12 Feb 2003, 11:06
Who is online
Users browsing this forum: No registered users and 0 guests