# Contributions:StatisticsFilter

## Contents

- 1 Synopsis
- 2 Location
- 3 Versioning
- 4 Methods
- 5 Functional Description
- 5.1 User-defined Objects
- 5.1.1 Observers and ChannelSets
- 5.1.2 Views
- 5.1.3 Output format
- 5.1.4 Computation of weighted statistics
- 5.1.5 Expressions
- 5.1.5.1 RSquared( obs1, obs2, idx1, idx2 )
- 5.1.5.2 ZScore( obs1, obs2, idx1, idx2 )
- 5.1.5.3 obs.Count()
- 5.1.5.4 obs.Mean()
- 5.1.5.5 obs.Variance()
- 5.1.5.6 obs.Covariance(idx1,idx2)
- 5.1.5.7 obs.Correlation(idx1,idx2)
- 5.1.5.8 obs.Skewness(), obs.Kurtosis()
- 5.1.5.9 obs.CentralMoment(n)
- 5.1.5.10 obs.Quantile(0>=p>=1)
- 5.1.5.11 obs.QQuantiles(n)
- 5.1.5.12 obs.Histogram(center, resolution, numBins)

- 5.1 User-defined Objects
- 6 Parameters
- 7 States
- 8 See also

## Synopsis

The StatisticsFilter is able to compute a number of statistical measures over a signal it is observing. Statistics are computed over decaying time windows of past data, and are thus updated continuously. Amongst other purposes, the StatisticsFilter may be used to compute feature maps displaying correlation between task and features online, during a measurement.

## Location

http://www.bci2000.org/svn/trunk/src/contrib/SignalProcessing/Statistics

## Versioning

### Author

juergen.mellinger@uni-tuebingen.de

### Source Code Revisions

- Initial development: 3758
- Tested under: 4151
- Known to compile under: 4164

## Methods

## Functional Description

The StatisticsFilter allows to statistically observe incoming data, and to compute various statistical measures over these data. Data observation may be restricted to boolean conditions involving state variables. Also, when computing statistics, older data may be weighted less, allowing for continuously updated statistics.

In addition to its input signal, the StatisticsFilter is able to include its own output into observation.

### User-defined Objects

Configuration of the StatisticsFilter is done by defining objects which belong to one of the following classes: **ChannelSets,** **Observers,** and **Views.**
For each class, there exists a parameter containing definitions of objects that belong that class. In these matrix-valued parameters, columns vary according to class, but each row corresponds to exactly one object. Objects must be uniquely named, and object names are specified by setting the corresponding row's label (double-click the row's header).

#### Observers and ChannelSets

**Observers**are the central components of the StatisticsFilter. They "observe" their inputs over time, and may provide statistical information based on their observations.- For each observer, there exists an observation condition in a column labelled "observe when." There, a boolean expression is expected, such as "TargetCode==1". No observation is made while the observation condition evaluates to "false".
- Observers also have a typical observation duration in the column labelled "observe over". When computing statistics, data are exponentially weighted according to their age. An observer's "observe over" entry defines for which age a data point's weight has decayed to a factor of exp(-1). An observer's data will only age when new data is being observed, i.e. while its observation condition evaluates to "true".
- Observers keep their observation data in between runs, but may be reset at any time by specifying an appropriate expression in their "reset when" column.
**ChannelSets**serve to define observers' inputs. Besides ChannelSets, BCI2000 expressions may be specified, e.g. in order to set a BCI2000 state to be the input to an observer. An observer's input is specified in the column labelled "observe what".

#### Views

**Views**define the output of statistical information. A view is a mathematical expression, which may contain general mathematical functions such as log(), sqrt(), +, *, / as well as statistical functions operating on observer objects. E.g., to get the mean value of the signal observed by observer "obs1", one defines a view computing the expression "obs1.mean()".- To compute the squared correlation coefficient ("Determination coefficient, r^2") between two conditions, one needs to define two observers, one for each condition. Into each observer's "observe when" column, one enters the desired condition, e.g. "TargetCode==1" and "TargetCode==2". Then, one creates a view to compute the r^2 value. For two observers "obs1" and "obs2", the corresponding expression would be "RSquared( obs1, obs2 )".
- To compute correlation between two signals, one defines a single observer with multivariate input. E.g., to compute a continuously updated correlation between the "TargetCode" state, and an input signal, one creates an observer with "true" as observation condition, i.e. an observer that observes continuously. For this observer, two entries are then specified under "observe what": First, "TargetCode", and then "AllChannels" (or the name of a different ChannelSet). This observer's input then has two dimensions: TargetCode and signal. In an output view, one may then write "obs.Correlation(1,2)", or short "obs.Correlation()", to view the 1,2 entry of the correlation matrix.
- Just like ChannelSets, views may be specified as input to observers. This allows for multi-level statistics, where statistical measures themselves become subject of statistical observation.

#### Output format

A view's output channels are the outer product of the input channels of the observers involved. Only "compatible" observers may be combined in a view's expression. When an observer has only a single ChannelSet in its input (as has been the case in all examples so far), then output channels directly correspond to input channels. When an observer has multivariate input, and there are ChannelSets associated with two of its input dimensions, then output channels correspond to pairwise combinations of input channels.

Output channel labels are constructed by combining input channel labels, such that is possible to identify each single combination in the output.

#### Computation of weighted statistics

In order to derive formulae for weighted statistics, one proceeds as if computing statistics from a histogram (or from pairs of histograms, in case of the RSquared() and ZScore() functions). First, the total weight of data points is normalized to 1. Then, statistical measures are expressed in terms of power sums of observed values. Finally, each contributing term in each power sum is weighted by the normalized weight of its corresponding data point.

#### Expressions

For the syntax of boolean and mathematical expressions, see: http://www.bci2000.org/wiki/index.php/User_Reference:Expression_Syntax

In addition to standard functions, the following statistical functions may be used when defining views:

##### RSquared( obs1, obs2, idx1, idx2 )

Computes the value (Determination coefficient, squared Pearson's r) between two sets of observations. For univariate observers, index arguments may be omitted. Otherwise, they specify a pair of dimensions to combine between first and second observer. For more information about the determination coefficient, see here.

##### ZScore( obs1, obs2, idx1, idx2 )

Computes a z score between two sets of observations, taking the first observer as a reference. As for the RSquared() function, indices refer to observer input dimensions, and may be omitted in case of two univariate observers.

##### obs.Count()

Computes the effective number of observations entering into statistics for observer "obs". The effective number of observations takes the weight decay of older observations into account, and approaches a constant for finite entries in the "observe over" column.

##### obs.Mean()

The mean of observations for observer "obs".

##### obs.Variance()

The variance of observations for each dimension of observer "obs".

##### obs.Covariance(idx1,idx2)

The covariance of observations for observer "obs". For a two-dimensional observer, indices may be omitted, resulting in the 1,2 entry of the covariance matrix. Diagonal elements of covariance are identical to the elements of variance.

##### obs.Correlation(idx1,idx2)

The correlation matrix for the dimensions of observer "obs". For a two-dimensional observer, indices may be omitted, resulting in the 1,2 entry of the correlation matrix. Diagonal elements of correlation are always 1.

##### obs.Skewness(), obs.Kurtosis()

The skewness and kurtosis for each dimension of observer "obs". Kurtosis is computed such that it becomes zero for Gaussian distributed data.

##### obs.CentralMoment(n)

The n-th central moment for each dimension of observer "obs".

##### obs.Quantile(0>=p>=1)

The value corresponding to the the given relative frequency in each dimension of observer "obs".

##### obs.QQuantiles(n)

For each dimension of observer "obs", a vector of values corresponding to the n-th q-quantile. This vector has n+1 entries, with the minimum in the first entry, and the maximum in the last entry.

##### obs.Histogram(center, resolution, numBins)

For each dimension of observer "obs", a vector of histogram frequencies with the specified properties.

## Parameters

### ChannelSets

Rows represent channel sets, defined as space-separated lists of channel names. Use row labels to name channel sets. Channel names may include * and ? wildcards, and character ranges enclosed in [], optionally negated by an exclamation mark. Ranges of channels may be specified using : or - to separate begin from end.

### Observers

Rows represent observers. In the "observe what" column, you may specify channel sets, views, or expressions. In the "observe when" column, provide a boolean condition, or "true" in order to observe always. For "observe over", give a typical lifetime of an observation. Specify multiple entries in the first column in order to do multivariate statistics. An observer is reset to its initial state whenever the expression in the "reset when" column evaluates to true. By default, observers keep their information in between runs.

### Views

Rows represent views. Views are defined by arithmetic expressions that may contain statistical functions, and properties of observers. Data from multiple columns will be concatenated in the output.

### OutputView

Specify a view to be copied into the filter's output signal. When empty, the filter's input will be used as an output.

### VisualizeViews

Wildcard expressions matching views to be displayed. Enter a * to display all defined views.

### EventScripts

Specify statements to be executed during StartRun, StopRun, Initialize, or Process. This is intended for creating and initializing variables that are later used within Observer or View expressions.

## States

"Observe when", "observe what", and view expressions may contain any state variable present in the system.