libsvmTL

• MultiClassSVMOneVsOne
• MultiClassSVMOneVsRest
  
  
• TwoClassSVMc
• TwoClassSVMnu
  
  
• Kernel_LINEAR,
• Kernel_RBF,
• Kernel_POLY,
• Kernel_SIGMOID
• Kernel_MATRIX (which is a wrapper for the other kernel classes)
  
  
• BasicFV (just a linear vector like std::vector)
• SparseFV (a sparse vector class like in original libsvm)
  
  
• OneClass-Algorithm will follow soon

You may use any combination of the above classes, e.g.

MultiClassSVMOneVsOne< TwoClassSVMc< Kernel_LINEAR> > svm;

Or if you like to calculate all kernel evaluations only once (and you have enough memory to store the whole kernel matrix), use:

MultiClassSVMOneVsOne< TwoClassSVMc< Kernel_MATRIX< Kernel_LINEAR> > > svm;

General way for setting / getting parameters from each class

Each class (Kernels, Two-class-SVM's, Multi-class-SVM's, Models, etc.) can read and write its private variables from/to a templated "structured data class", e.g. the loadParameters() method of Kernel_POLY looks like this:

class Kernel_POLY
{
    //...

    template<typename STDATA>
    void loadParameters( STDATA& stData)
          {
            stData.getValue( "gamma", p_gamma);
            stData.getValue( "coef0", p_coef0);
            stData.getValue( "degree", p_degree);
          }
};

• StDataASCII (uses a map<string,string> internally)
• StDataASCIIFile (inherits from StDataASCII and adds load/save methods)
• StDataCmdLine (inherits from StDataASCII, but parses its values from commandline (argc, argv))
• StDataNetCDF (reads and writes all values directly from a NetCDF file)

• StDataMatlab (interface to Matlab's Structures, to get full access to libsvmTL from Matlab via a mex-interface)

Customizable Progress Reporter Class allows time out

Each class reports its progress during training to a given ProgressReporter object. This allows nicer display of progress, even in GUI applications and to set a time-out, e.g. when doing large grid searches over weekend.

Runtime-Creation of any template class combinations via template meta programming

SVMFactory allows to create any combination of the above classes at runtime (instead of compile time). Implemented with template meta programming. So if you write your own Kernel-classes just add it to the kernel-list and you are done. Sample-Code:

/*----------------------------------------------------------
 *  specify available multi-class Algorithms
 *---------------------------------------------------------*/
typedef TTLIST_2( svt::MultiClassSVMOneVsOne, 
                  svt::MultiClassSVMOneVsRest) MyMultiClassList;

/*--------------------------------------------------------
 *  specify available two-class Algorithms
 *---------------------------------------------------------*/
typedef TTLIST_2( svt::TwoClassSVMc, 
                  svt::TwoClassSVMnu) MyTwoClassList;

/*--------------------------------------------------------
 *  specify available kernel functions
 *--------------------------------------------------------*/
typedef TLIST_6( svt::Kernel_LINEAR, 
                 svt::Kernel_RBF, 
                 svt::Kernel_POLY, 
                 svt::Kernel_SIGMOID,
                 svt::Kernel_MATRIX<svt::Kernel_LINEAR>,
                 svt::Kernel_MATRIX<svt::Kernel_RBF> ) MyKernelList;

/*--------------------------------------------------------
 *  Create approprate combination at runtime
 *--------------------------------------------------------*/
std::string mcName = "one_vs_rest"; 
std::string tcName = "c_svc";
std::string kfName = "rbf";

SVMAdapter* svm = svt::BasicSVMFactory< BasicFV,
                                        StDataASCIIFile,
                                        MyMultiClassList,
                                        MyTwoClassList,
                                        MyKernelList
                                      >::create( mcName, tcName, kfName);

Only one shell application with subcommands

Instead of having different shell applications like 'svm-train' and 'svm-predict', the libsvmTL has only one application that uses subcommands. The shell calls are:

     svmtl train [options] trainfile
     svmtl classify [options] testfile
     svmtl crossval [options] trainfile
     svmtl gridsearch [options] trainfile

Ready-to-use shell application as template

the class SVMApplication template class contains all methods for a full shell application, so the simplest way to write a fully functional svmtl Application which uses you self-written kernel, is

/*--------------------------------------------------------
 *  My svmtl application using MySuperKernel
 *--------------------------------------------------------*/
#include <SVMApplicationWithDefaults.hh>
#include <MySuperKernel.hh>

/*--------------------------------------------------------
 *  specify available kernel functions
 *--------------------------------------------------------*/
typedef TLIST_3( MySuperKernel,
                 svt::Kernel_MATRIX<MySuperKernel>,
		 svt::DefaultKernelList) MyKernelList;

int main( int argc, const char** argv)
{
  svt::SVMApplication< svt::BasicFV,
                       svt::StDataASCIIFile,
                       svt::DefaultMultiClassList,
                       svt::DefaultTwoClassList,
                       svt::DefaultOneClassList,
                       MyKernelList>        sappl;
  return sappl.main( argc, argv, std::cout);
}

If you have to implemented the methods name(), description(), loadParameters(), saveParameters(), getParamHelp() correctly in your MySuperKernel class, you will be able to set all your kernel-parameters from commandline, you'll get help for the parameters if you call the application e.g. with "mysvmtl train --help" and they will also be saved and restored correctly from the model file.

Overall speed improvements

Using non-sparse Feature-Vectors (class BasicFV) speeds up kernel evaluations. If you really need sparse feature vecotrs, just use the SparseFV class.

The libsvmTL implements a uniqueID() method for each feature vector, that allows to cache kernel evaluations from one training to another using the Kernel_MATRIX wrapper. This significantly speeds up cross-validation and grid-search

Optimized Cross-Validation algorithm that first trains all two-class SVM's for the whole dataset and then retrains only those two-class-SVM's that are affected from left-out vectors (significantly speed up for high number of subsets in cross validation and leave-one-out validation)

Optimized Grid only recalculates Kernel evaluations is kernel parameters change (e.g. when changing C ind C-SVM you don't need to reevaluate the kernel) (Planned, not implemented yet)

here is a comparison table for the runtime performance:

• Data set: unsmoothed USPS training set (first 2000 letters). Gray values taken as raw features (= 256 Features), normalized to -1..+1
• Computer: Athlon MP 1.2 GHz; 1GB RAM
• OS: Debian Linux, Kernel 2.4.25
• Compiler: gcc 3.3.3 (Debian 20040314), Optimization -O3
• time measurement: Using 'time' shell command and take the "user" time. As time does only change in fractions of seconds, when repeting an experiment, the tests are only done once.

... to be continued for optimzed grid search