BVQXtools

From BVwiki

Jump to: navigation, search

BVQXtools is a MATLAB toolbox maintained by Brain Innovation. As most important part of the toolbox, one central function (class) can be used to read, create, alter, and write most of BrainVoyager QX's file formats within MATLAB.

Although the toolbox is still being developed, file input and output works reliably for most of the major formats (at least in their versions used by BrainVoyager QX v1.86 and onwards).

Contents

Release notes

v0.7f

In those files added after February 2008, the relation of the author wrongly said "SCAN lab" instead of "SCAN Unit", which has been corrected (affecting so many files that this is no "snapshot" release). Additionally, a fast up-to-4D interpolation routine (MEX) has been added. See flexinterpn and flexinterpn_method for more info.

v0.7e

For the most part, functions have been cleaned and sped up again. Additionally, this version comes with an installer. After downloading and unzipping, you will find two files, one of which is an M-file that, together with the accompanying Mat-file, will install the toolbox on your harddrive.

v0.7d

BrainVoyager QX v1.10 has enhanced some file formats (statistical maps) and also added the SDM file format (also requiring some additional fields in the GLM file format). This release mainly copes with these changes.

Additionally, some functions have been enhanced in performance and some minor functionality has been added as well.

v0.7c

Due to platform specificity of MEX (compiled C-based) functions, it is required to run one batch M-file after installation, bvqxtools_postinstall. This will also make sure that the classes of the toolbox can be run. Additionally, a list of supported file formats will be printed in the MATLAB console window.

Please note that binary files (required for some of the functions) have been moved from the main toolbox directory (e.g. BVQXtools_v07c) to their own directory (BVQXtools_v07c_files), which will be automatically re-used by toolbox snapshots. This makes sure that the snapshots (versions between major releases) only needs to contain the M-files of the toolbox, not the binary files again...

For SPM2/5 users, a function has been added to resample (MNI) result maps (importvmpfromspms). To visualize results, please use the file colin.vmr in the files folder. With the surfaces (and transformation files) found in the colin folder, you can easily create rendered screenshots of your maps.

Be aware that some features have been removed from the toolbox because Brain Innovation has decided to split the toolbox into a basic toolbox, which will be made available to the public, and an add-on toolbox with additional processing routines, which are distributed only to licensed customers upon request.

Note: spreading of BVQXtools may only occur in one of its states available for download on this website. In case you receive added functions, do not share code that is not publicly available.

v0.7b

As of this version, the memory management of objects has changed, so that now objects are, sort of, passed "by reference" into functions. That means that if you write a function making use of a BVQXfile object, any alterations the functions does to the object will be done on the central object storage, so they will be present in the calling function as well! This is NOT the Matlab standard!

Consequently, as a side-effect, one must manually clear the storage space for objects loaded/created (although in the current release a "garbage collection" scheme is implemented, so that objects created outside of the Base Workspace are automatically removed, once the creating function is left). Unfortunately, Matlab, at this time, does not offer destructors as part of their class/object implementation, which would make this superfluous. To clear an object, use the "object.ClearObject;" method.

To prevent any unwanted object "destruction", another function has been introduced to the @BVQXfile class: bless. By calling this function on an object (as in "bless(object);"), you assure that the object will not be removed from the storage after a function is left.

Please inspect some of the new functions which demonstrate the usage of this new memory scheme (e.g. createvmr.m).

Getting started

Downloading the latest edition

Please download the latest publicly available edition, which is BVQXtools v0.7f. If you find bugs, please report them directly to the current maintainer, Jochen Weber, at jw2661 (at) columbia (dot) edu.

In case you have requested a feature or want to experiment with interim/daily updates of the toolbox, feel free to download the BVQXtools v0.7f daily snapshot.

Installation

Simply unzip the contents to a path of your choice, change into this directory in Matlab and run the M-file from the ZIP archive. This will start the installation, during which the C-based functions will be compiled and the file formats specifications cache is created.

Make sure to save the altered path in pathtool or to add the selected installation directory at later sessions again to make use of the toolbox.

You're now ready to use the BVQXfile function (class) to read and write BrainVoyager QX file formats.

Usage Overview

Creating files

With most file types, this feature is available by using the following syntax to create a new file in memory, e.g.: 

vmr = BVQXfile('new:vmr');

Many files (e.g. VMRs) are created with standard dimension, while others (e.g. VTCs) will be completely empty for not to allocate memory uselessly.

Reading files

To read files into MATLAB, a simple syntax can be used: 

objvar = BVQXfile(filename);

The class itself uses so-called binary format files and text format files stored in the folder BVQXtools/@BVQXfile/formats as information about the file formats. These specification files are human readable, so they can also be used to code a specific reader in virtually any other programming language. The same files are also used to write the files, which makes this a much more flexible and easier to maintain library of parsing information than using distinct reading and writing functions. Excluded from this logic are DICOM and FIF files, which use their own IO function, mostly for speed issues.

Due to the fact that MATLAB requires all objects of a class (in fact, objects are specifically treated structures) to contain the same, common fields, the content variable internally only stores some basic fields that allow the class methods to lookup the actually stored variables.

However, due to the overloading of the struct notation, access to object data is virtually easy. So, let's say you want to read and access the contents of a GLM file: 

glm = BVQXfile('C:\FMRIDATA\MSP_STUDY\P03\P03_RUN2.glm');

Then the following fields will be available in the variable glm: 

ans = 

                      FileVersion: 3
                      ProjectType: 1
                   ProjectTypeRFX: 0
                   NrOfTimePoints: 250
                   NrOfPredictors: 4
                      NrOfStudies: 1
               SeparatePredictors: 0
               TransformationType: 0
                       Resolution: 3
                SerialCorrelation: 0
                       MeanAR1Pre: 0
                      MeanAR1Post: 0
                           XStart: 57
                             XEnd: 231
                           YStart: 52
                             YEnd: 172
                           ZStart: 59
                             ZEnd: 197
            CortexBasedStatistics: 0
      NrOfVoxelsForBonfCorrection: 54127
    CortexBasedStatisticsMaskFile: ''
                            Study: [1x1 struct]
                        Predictor: [1x4 struct]
                     DesignMatrix: [250x4 double]
                              iXX: [4x4 double]
                          GLMData: [1x1 struct]

With the sub fields having the following contents: 

glm.Study

ans = 

        NrOfTimePoints: 250
    NameOfAnalyzedFile: 'C:/FMRIDATA/MSP_STUDY/P03/P03_3DMC_SCSAI_SD3DSS4.00mm_LTR_THP3c_TAL.vtc'
         NameOfRTCFile: 'Interactive'

Altering objects

For the VMR example above, it would be easy to create a random VMR: 

vmr.VMRData = uint8(225 * rand([256, 256, 256]));

As a convenience, many fields (such as number of volumes for VTCs) are checked/replaced immediately before writing the file to disk, so as to make sure that the file content is valid.

Writing files

To write an object back to file, the common Save and SaveAs methods can be used: 

glm.Save;    % save under the currently related filename
glm.SaveAs('file.glm');  % use a new filename

So, in our case above, it might be desirable to mask all maps with a pre-created mask: 

% iterate over beta maps
for bmc = 1:size(glm.GLMData.BetaMaps, 4)

    % multiply with mask (0 or 1)
    glm.GLMData.BetaMaps(:, :, :, bmc) = ...
        glm.GLMData.BetaMaps(:, :, :, bmc) .* msk; ...
end

glm.SaveAs('C:\FMRIDATA\MSP_STUDY\P03_MASKED.glm');


Documentation

Please keep in mind the toolbox is still under (ongoing) development, and the author has (unfortunately...) put more emphasis on the implementation of features than writing a truly comprehensive BVQXtools Documentation.


Transparent file access

Memory problems and solutions

Since performing a (voxel or vertex based) group statistics may require a huge amount of available memory (say 3GB for 30 VTCs with 100MB each), it is often necessary to perform an analysis bit by bit.

One way to solve this is to split an analysis up into temporally independent calculations (e.g. GLM estimation for separate studies). While this will work in some cases, there are situations where the time course data of all studies must be present in working memory to perform a calculus. However, it will (hopefully) only rarely occur that both all spatial and temporal information must be loaded in memory at the same time.

To simplify the access of large numerical arrays within files in chunks, the @transio class has been introduced into the toolbox.

Working mechanism

For each file type (e.g. VTCs) a threshold can be set (which by default is set to positive infinity). If for any given member variable of a file (in the case of a VTC the VTCData member) this threshold is exceeded, instead of reading the array into memory, a transio-object reference is created instead and the according number of bytes is skipped during the read.

Later, during access to that array, all read and write operations are passed to two functions that perform these operations directly on disk, efficiently blocking even complex indexing schemes into as few as possible read operations on disk.

Enabling transio access

To enable transio access for VTCs, you would give the BVQXfile class the new threshold: 

BVQXfile(0, 'transiosize', 'vtc', 65536);

With this command, every VTC that is opened afterwards will contain transio objects instead of large data arrays for fields that surpass the 64kB threshold.

To set this threshold for all (binary) files, leave out the filetype: 

BVQXfile(0, 'transiosize', 5e5);

This would set the threshold to roughly 500kb for all filetypes

Advantages

There are quite a few advantages that might be useful in several different cases:

  • opening time of large files reduces dramatically (e.g. to check whether a VTC file has the correct PRT file linked)
  • highly efficient file access, allowing even complex index operations in a reasonable time
  • highly efficient memory consumption by only storing required information

Syntax

The general syntax is quite straight forward. Indexing follows MATLAB rules with one major exception: indices might not be reused! Thus, if you want to access the same position within an array (or file) more than once, you must (at least currently) do so by first accessing unique positions and then indexing into the resulting array.

Here is just one example: 

 % enable transio for VTCs
BVQXfile(0, 'transiosize', 'vtc', 65536);

% open VTC, but do not load the VTCData...
vtc = BVQXfile('S1_R1.vtc');

% show memory consumtion
whos

This might then give... 

  Name      Size                    Bytes  Class

  vtc       1x1                     26527  BVQXfile object

Grand total is 2488 elements using 26527 bytes

A useful way to employ transio objects could then be: 

% create a mask in memory
mask = false(vdsize(2:4));

% get VTC size (overloaded correctly)
vdsize = size(vtc.VTCData);

% iterate over BV Z (TAL X) axis
for sc = 1:vdsize(4)

    % and find voxels where mean is > 80
   mask(:, :, sc) = squeeze(mean(vtc.VTCData(:, :, :, sc))) > 80;
end

% now extract only voxels of interest
maskedtc = vtc.VTCData(:, mask);

To yet receive the entire array, one can of course use the full-colon notation, but if the size is unknown (e.g. for a GLM map, which in the case of surfaces is VxP, vertices-by-predictors), another function, resolve(tiobj);, which has been introduced in v0.7b.

Limitations

WARNING: Also, for the current implementation, write operations are directly performed within the file that was opened. So, use any write operations into variables with extreme care!

For obvious reasons, it is impossible to grow or shrink a transio object within a file. Although it would be possible to implement this (by first copying the array into a temporary file and then doing the operations there), this would certainly require a chunk of memory at least by the size of the entire array to efficiently perform these operations.

Hence, for the time being, call such as 

% growing a VTC in time dim
vtc.VTCData(end+1:end+50, :, :, :) = 0;

% or removing BV system Z slices 21:end
vtc.VTCData(:, :, :, 21:end) = [];

will not work!!

Other functions

For your convenience, a number of additional function are part of the toolbox, of which a few shall be explained here:

  • acpc2tal - converts BV system coordinates from ACPC space into TAL space (with the respective TAL information), or inverse
  • acsvread - reads in tabular data, can be used for virtually all tabular text files, including header skipping and column naming
  • any2ascii - converts virtually any datatype into an ASCII representation that is usable for a call to eval
  • applybvtrf - applies a BV transformation matrix to a coordinate list (or inverse)
  • applyfdr - gives the threshold of an applied FDR to either t,F,r or p statistic (in the input statistic type)
  • averagevmrs - averages a list of VMRs (supports unmatching dimensions, either in a minimum or maximum containing box)
  • bv2tal - converts the BV internal system coordinates into TAL compliant ones
  • bvcoordconv - aplly the transformation chain: TAL - BVSys - BVInt - matrix subscripts - matrix indices (or reverse)
  • bvtrf - compile a BV-compliant TRF matrix from translation and rotation parameters
  • clearbvqxobjects - clears a list of objects from the global storage
  • clustercoords - returns the list of coordinates of clustered sub-volumes
  • correlpvalue - converts a correlation value to p (correltstat and correlinvtstat can be used to convert r to t and back)
  • convones - fast convolutes the given vector with a all-1's vector of given length
  • createvmr - similar to BrainVoyager project creation, a VMR object will be created from DICOM, PAR/REC or Analyze files
  • custom_tpdf, etc. - custom versions of distribution functions often needed for fMRI-based statistics (F, t, beta distributions); copies from free-of-charge libraries
  • dispslicemovie - internally uses dispslice to visualize a 3D dataset as a series of slices that are scanned through, one by one
  • emptysrf - create an empty surface file
  • fdr_thresholds - finds FDR thresholds for given set of p values
  • fileswapendian - swaps (part of) a file in endian type (with configuration wordsize)
  • findfiles - Linux/Unix like function for the GNU find command
  • fisherr2z - convert Fisher's correlation coefficient to z-value (or inverse)
  • gluetostring - takes a cell array of strings and returns a 1xN char array made up from the single strings put together by some glue
  • grep - Linux/Unix like function for the GNU grep command
  • hrf - returns the (configurable) canonical HRF shape
  • hxsingle / hxdouble - convert a single/double numeric into an exact ASCII representation and back
  • importvtcfromanalyze - import an MNI space Analyze time series as VTC
  • initialalignment - compute BV's initial alignment matrix from VMR/FMR headers
  • interpn_linnonull - (Tri-) Linear interpolation of data where 0 is treated as missing sample
  • makelabel - create a usable label from a given string
  • md5sum - compute MD5 checksum over file or char array
  • minarray - remove all null slices of an array
  • mkadir - absolute version of mkdir command/function
  • mni2tal - Matthew Brett's function to convert from MNI space to TAL
  • mstrrep - multiple replacements within strings
  • newnatresvmp - create a new, standard space, natural resolution VMP object, as the default is still old, 1x1x1mm VMP for BVQXfile('new:vmp');
  • orthimage - return an image (scaled 0 to 1) for the design's pairwise orthogonality (as in SPM)
  • orthvec / orthvecs - regressor orthogonalization
  • psctrans - percent signal change transformation of time course data
  • readpar - reader for Philips PAR/REC files
  • samplefmrspace - sample BV's FMR space (differing from VMR system coordinates!)
  • splittocell - split a 1xN char array to a list of strings
  • spmmat2prt - convert an SPM2/5 design matrix into a BVQX PRT file
  • spmmat2rtc - convert an SPM2/5 design matrix into a BVQX RTC file
  • tdclient - perform a request to a (local) Talairach Daemon database (local version implemented in tdlocal or tdlocal2)
  • tfmatrix - create a transformation matrix (quaternion) from translation, rotation, scaling and shearing parameters
  • vtc_concat - concatenate VTCs (in time), uses transio and allows large files
  • ztrans - z-transformation of time course data

History

  • 11/5/2008, v0.7f
    • corrected the current affiliation of the main author in newer files
    • added functions: flexinterpn (MEX), gluetostringc (MEX), splittocellc (MEX)
    • added method: GLM::RFX_rMap to generate OLS and robust regression correlation maps
    • some minor corrections with some functions and added help for MEX functions
  • 10/18/2008, v0.7e
    • cleaned a lot of functions and speeding up things with profiler
    • added file format: SPSS (allows to write SAV files)
    • added functions: alphasim, bvqxfilehead, clustercoordsc (MEX), clustervol, smoothdata3, splitclustercoords
    • added methods: GLM::JoinRFX, GLM::PlotBetas, SPSS::ImportData
  • 6/18/2008, v0.7d, release with BrainVoyager QX v1.10 file formats
    • updated/added file formats: ATD, CMP, GAL, GLM, SDM, SMP, VMP
    • updated/added functions: acsvread, createsrf, cutnpastemovie, depcorrt, scaleimage, smoothest3d, spherecoords, spherecoordsinv
    • updated/added methods: AFT::BoundingBox, AFT::DeleteHandle, AFT::Handles, AFT::SetHandle, SRF::CreateVMVFromMTC, SRF::CreateVMVFromSMP, VMP::ClusterTable
  • 10/3/2007, v0.7c, release with split binary files folder
    • binary files moved to their own folder (simplifying snapsnot handling)
    • some functions removed (to protect Brain Innovation's business interest)
    • new functions: importvmpsfromspms, spherevmr
    • new methods: SRF::DensityMap, VMP::ImportSPMMaps, VTC::CrossCorrelate
    • added files: colin.vmr (used to overlay resampled SPM results in BV), colin_RH.srf, colin_LH.srf + transformation files (used to create surface results of SPM maps)
  • 9/7/2007, v0.7b, release with new memory management scheme
    • new file formats: ATD, BSA
    • updated / tested formats: DMR, DWI, FMR, HDR, MAP, SFH
    • new classes: @BVQXfigure (general figure/ui class), @BVQXinifile (config file handling), @BVQXprogress (simple progress bar), @BVQXtool (GUI)
    • new methods: AFT::BoundingBox, AFT::GetVolume, MAP::CreateVMP, VMP::ClusterTable, VMP::ConvertToNatRes, VMP::MakeHiResVMP, VMR::CleanVMR, VMR::CreateVTC, VMR::OverlayVMP, VMR::SetConvention, VOI::ApplyTrf
    • new functions: acpc2tal, applybvtrf, applyfdr, averagevmrs, bvcoordconv, clustercoords, correlinvtstat, correlpvalue, correltstat, dilate3d, erode3d, fileswapendian, fisherr2z, floodfill3, importvtcfromanalyze, initialalignment, orthimage, samplefmrspace, spmmat2rtc, transio::resolve
    • improvements: memory management centralized to allow call-by-reference, including garbage collection of subfunctions
  • 7/20/2007, v0.7a, re-release
    • service patch to get transio access working again from within BVQXfile
  • 7/17/2007, v0.7a, release for BVQX v1.9 (new file formats)
    • new file formats: CPS, DDT, FIF, GRB, SFH
    • updated / tested formats: DMR, MAP, VMP, VMR
    • new methods: FIF::*, POI::Area, PRT::ImportStimChannels, SRF::RecalcNormals, VMP::Update
    • new functions: fdr_thresholds, fifio, ima2v16, renamefile, spmmat2prt
    • improvements: more efficient cache of FileFormat information in BVQXfile class, transio class is now fopen save (Matlab R14SP2)
  • 6/19/2007, v0.6f, again minor amendments and additions
    • new file formats: DMR, DWI, EIG, VGD
    • new methods: GLM::RFX_RemovePredictors, MTC::SmoothIterativeNN, VTC::Resample
    • new functions: orthrtc, readbesa
    • minor bug and format fixes in a few formats, functions, and methods
  • 3/2/2007, v0.6e, rather only minor amendments and additions
    • transio read support for Matlab v6.5
    • added function: ltriasc (get ASCII output of lower triangular matrix for Mx)
    • added method: GLM::RFX_tMap (currently only for MTC-based RFX GLMs)
    • some minor format/bug fixes
  • 2/7/2007, v0.6d, UFF file format, file selection and spec caching feature for BVQXfile and new methods, functions + bug fixes
    • UFF file format added
    • added file selection to BVQXfile: v = BVQXfile('*.vmr');
    • specifications are cached for faster BVQXfile access
    • new methods: AVG::Average, AVG::Plot, GLM::SingleStudy_tMap, SRF::FindSpikes, VOI::BVCoords, VTC::CreateGLM, VTC::VOITimeCourse
    • new functions: orthvecs, glmtstat, mx, mxgrep, parrecvmr, writestcfiles
    • bug fixes in several formats and functions
  • 1/12/2007, v0.6c, several smaller bug fixes, new file formats and methods
    • bug fixes in formats: CMP, SRF, SSM, VMV
    • most file formats now support creation from scratch
    • new methods: Any::Help, CMP::AddComponent, DTSF::SaveAsFBR, PRT::ConvertToMS, PRT::ConvertToVol, PRT::CreateRTC, VMP::SaveAsVMR, VVD::CalcROIBetas
    • extension ICA is now supported (as CMP)
    • new functions: convones, gammapdf, hrf, mx, orthvec
  • 12/20/2006, v0.6b, two new file formats: OLT, POI; fixed some bugs, etc.
    • new file formats: OLT (lookup table for colors) and POI (patch of interest)
    • bug fix: file formats that transposed a matrix would do so on save (rendering the contents useless)
    • enhanced SRF::Show method, now colors are displayed as in SRF, also support for SMP overlay
  • 12/13/2006, v0.6a, added @transio class
    • new class: transio, adds transparent access into binary files (VTC, MTC, GLM) for less memory consumption
  • 12/4/2006, v0.5c, added own DICOM reader, some new formats and methods
    • new functions: dcmio, psctrans, tdlocal2 (based on v2.0), ztrans
    • new formats: DCM, VVD, VWP
    • new methods: GLM::JoinFFX, GLM::PSCMaps, PRT::CleanUp, SRF::Show, VMP::SaveSubset, VTC::MeanStd
  • 11/21/2006, v0.5b, some fixed and speedups
    • new function: tal2bv
    • improved functions: bv2tal, tdlocal (speedup)
  • 11/19/2006, v0.5a, many fixes and additions
    • new/improved file formats: CTR, FBR, FDP, GRAD, HDR, MDM, ROID, TDMC, TMV, TVL, VMR
    • new functions: tdclient, tdlocal
    • improved functions: splittocell (much more efficient)
  • 11/3/2006, v0.4a, create new files, full class interface
    • full class interface (overloaded methods to remove BVQXCONT, etc.)
    • implemented consistency check feature
    • added creation of emtpy files feature
    • fixed AVG.tff (Event-related averaging files, thanks to Erik Chang)
  • 10/4/2006, v0.3a, corrected some of the BFF files
    • fixed GLM.bff (DesignMatrix was read with columns and rows swapped)
    • fixed MSK.bff (older MATLAB version < 7 would give a warning)
  • 9/13/2006, v0.2a, first public release
Retrieved from "http://wiki.brainvoyager.com/BVQXtools"
Personal tools