Programming Howto:Quickstart Guide

From BCI2000 Wiki
Jump to: navigation, search

This tutorial walks you through the process of obtaining the BCI2000 source distribution, and using it to build and test your own custom filters, implemented in C++ inside your own custom core module. It assumes that you have a good working knowledge of the C++ language, and basic familiarity with the compiler/IDE that you are going to use.

Many of the specific instructions below will assume that you are on a 32-bit Windows system, are using Microsoft's free Visual C++ 2012 Express compiler, and have checked out the BCI2000 distribution to a location C:\BCI2000\3.x on your hard-drive. However, the same steps are valid for other supported setups, with the appropriate setup-specific changes.

Prerequisites

All Operating Systems

  1. First, you need a complete BCI2000 distribution that includes the src and build directories. If you already have this, use SVN to make sure it is updated to a recent stable version. If you do not already have the source distribution, here is how you obtain it:
    1. If you do not already have a bci2000.org username, get one here.
    2. To download the BCI2000 source-code, you will need an SVN client. If you do not have one, for Windows we recommend TortoiseSVN, which can be downloaded here. The BCI2000 wiki contains more documentation about how to set up and use SVN.
    3. Use SVN, with your username and password, to check out the latest BCI2000 source-code-included distribution, the location of which is http://www.bci2000.org/svn/tags/releases/current (note this is the current release version, not the current source code -- for the current source code, check out from http://www.bci2000.org/svn/trunk). We will assume that you checked out to a location C:\BCI2000\3.x on your hard-drive. Wherever you see this path below, adjust it so that it reflects the location you actually used. More information about the layout of the resulting distribution can be found here.
  2. Next you will need to download and install CMake if you do not already have it (at least version 3.5.2 is recommended). CMake can be downloaded here. Under Windows, make sure you select the option add to PATH for all users when installing.
  3. Finally you will need a C++ compiler. We will proceed on the assumption that you are on 32-bit Windows and using the free Visual C++ 2012 Express For Windows Desktop environment which can be downloaded [1] (choose either the Visual C++ web-installer or, for something that can be installed offline, the file called marked "VS2012_WDX_ENU.iso" - this will give you an .iso file which can be mounted as a virtual drive containing the installer and all necessary files.)

Consult the documentation for the BCI2000 build system for information on other supported compilers, and more about CMake.

You should now have everything you need in order to build and customize BCI2000.

Prerequisites (Linux)

  1. C++ compiler
  2. Subversion
  3. CMake
  4. Qt libraries and header files (Qt-dev)

Then follow the "unix command-line equivalent" instructions from a terminal.

Prerequisites (OS X)

  1. Apple's Developer Tools (includes a C/C++ compiler and Subversion)
  2. A CMake binary installer (.dmg file) appropriate to your platform, version 2.8.3 or later.
  3. Qt libraries (You only need the libraries, not the full SDK. We have performed successful builds with v4.7.2. Your mileage may vary according to the version you use.)

Then follow the "unix command-line equivalent" instructions from the Terminal.app command-line.

NOTE: CMake may be used to create an XCode project file for BCI2000, using the appropriate generator option. However, it seems that XCode builds of BCI2000 do not always succeed, even if there is no problem with Unix makefile builds on the same machine. Thus, you should always try a command-line build when building fails under XCode.

Unix command-line equivalent:

$ cd ~  # or wherever
$ svn checkout http://www.bci2000.org/svn/trunk  bci2000   # (or call it whatever you want)

Setup

  1. Use CMake to build a project file. This is done by going to C:\BCI2000\3.x\build (which is the main workbench from which all build operations occur) and launching one of the "Make ... .cmd" files. If you are using Visual C++ 2012 Express, the appropriate batch file is Make VS2012 Project Files.cmd . It may ask you several questions: feel free to answer "y" to building the "tools", "contrib" and "BCPy2000" components. For now, for simplicity, you may wish to say no to framework extensions. Also say no to modules that require MFC (unless you are using a commercial version of Visual C++ Studio that includes MFC). If successful, you should see a long list of sub-projects being created, and after several seconds of this, "Configuring done" followed by "Generating done".
  2. Open the resulting solution file, BCI2000.sln , in Visual C++ Express. The file should be located in C:\BCI2000\3.x\build .
  3. Ensure that you are in "Release" mode rather than "Debug" (in Visual C++, this is a drop-down menu in a toolbar near the top)
  4. You probably do not need to build all of BCI2000 at once. To use BCI2000, the minimum you will need consists of the Operator, plus at least one SignalSource module, at least one SignalProcessing module, and at least one Application module. To build a single module in Visual C++ Express, right-click on the module's name in the list on the left, and select "Build". The initial build will typically take a few minutes per module. If you do not intend to modify a given module, you can always skip building it by copying the corresponding ready-built binary (for example, Operator.exe) from the prog subdirectory of a BCI2000 v.2 or v.3 binary distribution, and paste it into the corresponding directory of the distribution in which you are working, i.e. C:\BCI2000\3.x\prog . (To download the latest binary distribution, you will need to supply the same username and password that you used for SVN download.) Let's assume you have built, or copied, the following modules: SignalGenerator, SpectralSignalProcessing, CursorTask and Operator, as well as the necessary OperatorLib shared library. Test the modules, i.e. run C:\BCI2000\3.x\batch\CursorTask_SignalGenerator.bat .
  5. Build the NewBCI2000Module, NewBCI2000Filter and NewBCI2000FilterTool targets for use in the subsequent steps. The result will be new executables, NewBCI2000Module.exe, NewBCI2000Filter.exe and NewBCI2000FilterTool.exe, in the C:\BCI2000\3.x\build directory.
Unix command-line equivalent:

$ cd ~/bci2000  # or wherever
$ cd build
$ ./Make\ Unix\ Makefiles.sh
$ make NewBCI2000Module NewBCI2000Filter NewBCI2000FilterTool

Creating a project for a new BCI2000 module, from a template

  1. Navigate to C:\BCI2000\3.x\build
  2. Find NewBCI2000Module.exe there, assuming you have already built it (see "Setup", above).
  3. Launch it and answer the three questions it asks:
    1. What type of module are you creating? Enter "1" for a SignalSource module, "2" for a SignalProcessing module, or "3" for an Application module.
    2. What should be the name of the new module? This should be single word, without spaces or punctuation (For example: VeryNiceSignalProcessing. Or, at your option, something even more informative.) This name will be used as the name of a new directory that will contain the project source information, as well as for the resulting binary (.exe file).
    3. Where (i.e. inside what parent directory) should the new project directory be created? You can express this as a relative path, in which case it is interpreted relative to the build directory where NewBCI2000Module.exe itself is running. To accept the default answer, which is ..\src\custom, just press return. This location (or some subdirectory of it) is a good choice: it maps to C:\BCI2000\3.x\src\custom which is an area reserved for your own projects.
  4. Now re-run CMake (see step 1 of "Setup", above). You should see your new project being added at the end of the long list of projects. Re-open the project file (step 2 of "Setup") and you should see it listed in the alphabetical list of projects.
  5. If you have just created a new SignalSource project, a specialized source-acquisition filter (called an Analogue-to-Digital Converter or ADC) will already have been added to the project. If your project was called Foo or FooSource this will be instantiated in files called FooADC.cpp and FooADC.h. You can find this by expanding the tree (click on the plus sign) next to your project name in Visual C++. Find the files under "Source" (or "Headers") and then "Project". Double-click them to edit. You should already be able to compile the module: try it.
  6. If you have created a SignalProcessing or Application project, you may be able to compile it, but it will not do anything until you add at least one filter to it (see next section).
Unix command-line equivalent:

$ cd ~/bci2000  # or wherever
$ cd build
$ ./NewBCI2000Module 2 VeryNiceSignalProcessing ../src/custom
$ ./Make\ Unix\ Makefiles.sh
$ make VeryNiceSignalProcessing


Adding a new filter to a BCI2000 module, from a template

  1. Navigate to C:\BCI2000\3.x\build
  2. Find NewBCI2000Filter.exe there, assuming you have already built it (see "Setup", above).
  3. Launch it and answer the three questions it asks:
    1. What type of filter are you creating? Enter "1" for a subclass of BufferedADC (although if you created a SignalSource project with the NewBCI2000Module tool, an ADC of this kind will already have been created for you), enter "2" for a subclass of GenericFilter, or "3" for a subclass of ApplicationBase.
    2. What should be the name of the filter class? This should be a legal name for a new C++ class (no spaces or punctuation). If you enter, for example, "FooFilter", then a class of this name will be implemented in two new files, FooFilter.cpp and FooFilter.h .
    3. To which project (i.e. in which project directory, relative to build) should the filter be added? For example: ..\src\custom\VeryNiceSignalProcessing (as in all of the BCI2000 framework, directory slashes are allowed to go this / way or that \ way regardless of whether you are on Windows or not)
  4. Again, re-run CMake and re-open the master project file.
  5. Find your new files under the Source/Project and Headers/Project trees by clicking on the + sign next to the project name. Double-click them to edit. Read the comments in the file for help as to how to flesh out the various filter methods.
  6. Try building the project (right-click on the project and select "build"). SignalSource and Application projects may already build successfully. SignalProcessing projects, or any project to which you have added a new GenericFilter subclass, will have at least one deliberate #error in the code. When the build attempt finishes, double-click on the error message to go to the offending line. Read the comments in the file, and you will see that the error is there to force you to think about the ordering of the filters in your module. Once you have resolved this issue, simply remove the #error line and select "build" again (the process will be quicker this time: only previously unbuilt files or newly modified files will be compiled).
Unix command-line equivalent:

$ cd ~/bci2000  # or wherever
$ cd build
$ ./NewBCI2000Filter 2 FooFilter ../src/custom/VeryNiceSignalProcessing
$ ./Make\ Unix\ Makefiles.sh
$ make VeryNiceSignalProcessing


Running and testing the result

  1. Locate successfully built modules, which will appear as .exe files inside the top-level prog directory of the BCI2000 distribution, for example C:\BCI2000\3.x\prog
  2. The most basic way to launch BCI2000 is to double-click on each module in turn. Start with Operator.exe, then launch one SignalSource module, one SignalProcessing module, and one Application module (your new module will fill one of these three roles, but all three are required).
  3. If you have a firewall running on your machine, dialogs may open for any modules that the firewall has never seen before. If so, click "unblock" to proceed, for each one. You should only ever need to do this once per newly-built module. (Check that your firewall is not configured to block network connections without telling you.)
  4. In the Config dialog, under the Visualize tab, you should see a check-box for visualizing the output of all the filters in the chain, including your newly created filter. You may find it useful, whenever you are developing a new filter, to visualize your filter input and output simultaneously (i.e. visualize both the new filter and the filter that immediately precedes it in the chain).

Configuring your new filter for offline use

This is an optional step which many developers of new custom filters will find useful. Filters may be compiled singly as standalone executables to allow them to be tested and used for data analysis offline. The resulting "filter tools" can be used either from the DOS/Unix command-line or, more comfortably, from the Matlab command-line.

  1. Navigate to C:\BCI2000\3.x\build
  2. Find NewBCI2000FilterTool.exe there, assuming you have already built it (see "Setup", above).
  3. Launch it, and tell it which existing C++ file contains the filter definition (as always, specify the path relative to the build directory—for example, ..\src\custom\VeryNiceSignalProcessing\FooFilter.cpp)
  4. The tool will perform for you the necessary alterations to the CMakeLists files. (The procedure, details of which can be found here, involves creating a subdirectory called cmdline inside your project directory, because specifying a new executable target in the same CMakeLists.txt file as your module would cause a misconfiguration problems for either the module or the filter-tool. )
  5. Close BCI2000.sln, re-run CMake, and re-open BCI2000.sln. You should now have a new target, whose name is simply the name of the filter (e.g. FooFilter).
  6. Right-click on the new target and select "build". The resulting binary will appear as (for example) ..\tools\cmdline\FooFilter.exe
  7. If you have Matlab, consult User_Reference:Matlab_Tools to see how to proceed. Otherwise see User_Reference:Command_Line_Processing. (In the Matlab version, the "Hello World" messages of the default template filter, and any other bciout debugging outputs you specify, will appear in the .ShellOutput field.)
Unix command-line equivalent:

$ cd ~/bci2000  # or wherever
$ cd build
$ ./NewBCI2000FilterTool  ../src/custom/VeryNiceSignalProcessing/FooFilter.cpp
$ ./Make\ Unix\ Makefiles.sh
$ make FooFilter
$ ../tools/cmdline/FooFilter --help
$ make bci_dat2stream bci_stream2mat  # you'll also need these

Exercises

Exercise 1: RMSFilter

  1. Use NewBCI2000Module to create a new SignalProcessing module.
  2. Use NewBCI2000Filter to create a new filter called RMSFilter in your new module.
  3. NewBCI2000Module and NewBCI2000Filter report that they are creating and altering various files and directories. Write down what you think is the purpose of each new directory, new file, or alteration to an existing file.
  4. Write a filter called RMSFilter, which takes in multiple signal channels, and outputs a single channel containing the root-mean-square signal across all input channels. (Advanced variant: introduce a parameter which allows the user to specify groups of channels; then output one RMS signal per group.) Test the filter using the SignalGenerator module as an input, configuring the SignalGenerator to respond to mouse movement as described in the documentation. First visualize just the input of your new filter (the output of the previous filter, which will presumably be the TransmissionFilter). Then draw on paper what you would expect to see as an output of your filter in response to different mouse positions/actions. Finally, visualize your filter's output in order to verify that it matches your expectations.
  5. Visualization allows you to test your code's behaviour qualitatively to some extent. In what various different ways could you verify, in a more precise, quantitative way, that your implementation is correct? Under what circumstances should you spend the extra time to do this? (hint: the answer rhymes with "hallways")

Exercise 2: Debugging

  1. Write a batch file to launch your combination of modules. You may wish to use one of the existing files in the top-level batch directory as a template. A batch file will allow you to go around each edit-compile-debug cycle much faster and more reliably. It will also allow you to pass useful command-line flags to the modules as you start them. And finally, if you need to set parameters in a certain way on each launch, we strongly recommended that you take advantage of operator scripting in your batch file to ensure that the required parameters are loaded automatically from a file on each launch. Inconsistent behaviour from one debug cycle to the next can often be attributed to having forgotten to perform the menial task of loading parameters manually.
  2. You can also use the Visual C++ debugger as follows:
    1. In your batch file, comment out or remove the line that launches the module you want to debug.
    2. Again in your batch file, assuming you are using the SignalGenerator, FilePlayback, or some other source module which does not have to run in real-time, add the flag --EvaluateTiming=0 to the call that launches the source module (see example snippet below). Putting a debug breakpoint in your Process() method will slow the system to below real-time, and we do not want the framework's real-time check to terminate the debug session for this reason.
    3. Launch the batch file, thereby launching all the necessary modules except one, and loading any parameters needed for the debug session.
    4. Note that there are different "build modes" for the BCI2000 solution, with names like "Release" and "Debug". In order to debug a particular module, you will need to ensure that the module is built in either "Debug" mode or "RelWithDebInfo" mode. If you are in the wrong mode for debugging, select the correct build mode from the drop-down menu on Visual C++ Express's toolbar, then build your module again. Note that the BCPy2000 modules (PythonSource, PythonSignalProcessing and PythonApplication) cannot be built in Debug mode. Also, Debug mode can sometimes mask dangerous bugs, so you may experience the frustration of trying to investigate a crash only for it to stop happening when you switch from Release to Debug. Finally, a BCI2000 module built in Debug mode may exhibit poorer timing performance. For these reasons it is not generally advisable to use Debug mode as the default mode of your project, and RelWithDebInfo is often advisable when debugging (even though the compiler optimizations may lead to a less logical-seeming debugging experience).
    5. In Visual C++, set a breakpoint in the source file you want to debug.
    6. Although CMake directs Visual C++ to create the module in the correct directory, C:\BCI2000\3.x\prog, it is unable to set the working directory in which the debug instance runs to the same value. Therefore, if your module needs to load resources like image or sound files, and expects to find these at the end of a path that is expressed relative to prog, you will need to set the module's working directory by hand so that it runs in prog: Right-click on the module -> Properties -> Configuration Properties -> Debugging -> Working Directory
    7. In Visual C++, right-click on the module, then select "Debug" followed by "Start New Instance" (if you find this too fiddly to do with the mouse, right-click, then press "g", then press "s" ). If the build is not up-to-date, Visual Studio will prompt you: in this case, allow it to build the module.
    8. You are now debugging BCI2000. Verify that the debugger stops your module at the breakpoint you specified and at the time that you expected. Browse the available variables and their properties.
start SignalGenerator.exe --LogMouse=1 --EvaluateTiming=0

Exercise 3: Assembling a filter chain

  1. Write a filter called DifferenceFilter, in which each output channel contains the discrete difference, from each time-sample to the next, of the corresponding input channel (NB: we'll avoid the name "DiffFilter" because it may create a name conflict with another contributed component of BCI2000). Note that your Process() method only sees one discrete chunk (or SampleBlock) of signal at a time. Is this a problem? Use private member variables of your filter instance as a "scratch-pad" where necessary.
  2. Assemble the pre-existing ExpressionFilter, followed by your DifferenceFilter, followed by your RMSFilter (from exercise 1), in that order within one SignalProcessing module. The ExpressionFilter implementation is already part of the framework, so all you need to do is uncomment the #include and Filter() statements that correspond to it in PipeDefinition.cpp. Read the comments in PipeDefinition.cpp, and the Programming_Reference:Filter_Chain wiki page, for more information about linking filters in a chain.
  3. Make sure your SignalSource module is started with the --LogMouse=1 flag (see the code snippet above).
  4. Set the Expressions parameter (found in the Filtering tab of the Config dialog) such that it has two rows and one column, and contains the expressions "MousePosX" and "MousePosY".
  5. Write down a full description of how you think the SignalProcessing module will process the signal, from start to finish, and exactly how you expect it to respond to different kinds of mouse movement.
  6. Verify that the module behaves the way you expect, under all relevant input conditions.

Exercise 4: Offline filter-chain reconstruction in Matlab

  1. Take one or more of the filters you have written (RMSFilter, DifferenceFilter), or create a new one which also does something whose numerical output can be checked very easily (examples: squaring the input signal, or doubling it, or taking the absolute value).
  2. Use NewBCI2000FilterTool to create a command-line "filter tool" executable as described above.
  3. Also ensure that the bci_dat2stream and bci_stream2mat targets have been built.
  4. Learn how to use the matlab function bci2000chain and the supporting Matlab tools.
  5. Use bci2000chain to run your filter on some example data, or even on some toy data that you create using create_bcidat. Then, separately, write a Matlab function for performing the same numerical operation that you believe your filter performs. Compare the two outputs and verify that your filter does exactly what it is intended to do, at least to within some very small numerical tolerance. The maximum absolute difference between the two outputs should be very small (say, 1e-10 or less, depending on the operations and on the magnitude of the input signal).

Further reading

Refer to the main Programming_Reference:Contents page here on http://doc.bci2000.org for a list of topics on which you can find more in-depth documentation. In particular, you may wish to work through the various tutorials at the bottom of the page.