MVPA Toolbox - Experience Based Cognition (EBC) Competition Extension

A free add-on to the MVPA toolbox to recreate the EBC competition results of Chigirev & Stephens from the Princeton EBC Team.

Overview

The EBC Competition

The Experience Based Cognition (EBC) Competition is a DARPA-funded event hosted by the University of Pittsburgh in which contestants apply machine learning and statistical algorithms to predict cognitive variables from fMRI images. For more information on the competition, see the competition homepage. The Princeton EBC Team submitted a number of entries to the competition; of these, a submission by Denis Chigirev and Greg Stephens finished in 2nd place. (The final rankings have been announced at OHBM 2006). Their submission used ridge regression and a number of averaging techniques, all of which are included in this extension; their methods paper can be found here.

The MVPA EBC Extension

The EBC Extension to the MVPA toolbox is a set of scripts that extends the functionality of the toolbox to allow for the analysis of regression experiments, as well as an example dataset taken from the data used in the EBC competition. The extension includes two tutorials to guide users through an example analysis that applies ridge regression to voxel data in an attempt to predict continously-scale cognitive measurements produced by three experimental subjects.

Most importantly, however, is that the EBC extension (like the toolbox itself) is highly modular, and we really hope that others will write modules to support their own analyses within the MVPA framework. We encourage anyone who is interested in contributing in any way to the toolbox to please email us; we welcome contributors and collaborators, and we hope to make it easier for future competitors to benefit from and extend current methods.

Please Note: With the release of MVPA 0.8, the EBC version 0.3 functionality has been included as part of the standard MVPA toolbox. Please see the online documentation for more information.

Installation

Please see the MVPA Wiki Setup Page for instructions on how to install the MVPA toolbox.

Documentation

Please see the MVPA Wiki Documentation for the new documentation for the EBC toolbox.

Release Notes (deprecated)

0.3 release - download

• Major milestone release.
• All performance optimizations have now been implemented. The tutorials cover spatial, temporal, and across-subject averaging, and combining these with optimized parameters will result in an official EBC score (movie #1 to movie #2) of 0.476. This is consistent with the test scores achieved by Greg and Denis's submission to the competition.
• A new, general purpose preprocessor has been introduced: apply_to_runs, which allows for the easy inclusion of filters, denoising, zscoring, etc., without the need to know MVPA code. See the commented script for details.

0.2 release - download

• Two major performance optimizations have been implemented: the data can be time filtered, and the parameters are no longer random. You can expect offical average scores (movie #1 to movie #2) up to about 0.3, instead of 0.15, using the advanced tutorial.
• statmap_xcorr has been rewritten in C to provide massive speed improvements in calculations.
• Both tutorials have been updated. The advanced tutorial has been completely overhauled, and is now modular and more powerful. The documentation is also included in the archive.
• The advanced tutorial now includes calculating a score using the offical EBC scoring program.
• The major optimizations remaining are spatial filtering, better voxel selection, and parameter searching. A function to perform spatial filtering has been written and included (create_spatial_avg_pat.m), but it is unoptimized and untested, so use at your own risk.

0.1 release - download

• Initial release.
• A fancy HTML version is only available for the introductory tutorial.
• This release is entirely unoptimized, so please bear with performance issues and abysmal generalization performance. Eventually, we will have all of the optimizations from Chigirev & Stephens's submission. These include spatial filtering, temporal filtering, better voxel selection, and parameter optimizations. Until then, the random parameters used in this early release cannot be expected to generate decent results; in the meantime, why not try your own optimizations and see if you can improve performance?

Troubleshooting and Contact

This is an early version of the software and documentation, so please be patient with teething issues. If you find or encounter a bug, a typo, confusing documentation, or problematic behavior, please email David Weiss (primary author of the EBC extension) or mvpa_support@princeton.edu and we will attempt to respond to the issue and post a solution here.

PLEASE NOTE: This extension is, like the MVPA toolbox, beta software. It is available free and unsupported to those who might find it useful. We do not take any responsibility whatsoever for any problems that you may have related to the use of the MVPA toolbox.