246 lines
9.8 KiB
Plaintext
246 lines
9.8 KiB
Plaintext
|
-----------------------------------------
|
||
|
|
--- MATLAB/OCTAVE interface of LIBSVM ---
|
||
|
|
-----------------------------------------
|
||
|
|
|
||
|
|
Table of Contents
|
||
|
|
=================
|
||
|
|
|
||
|
|
- Introduction
|
||
|
|
- Installation
|
||
|
|
- Usage
|
||
|
|
- Returned Model Structure
|
||
|
|
- Other Utilities
|
||
|
|
- Examples
|
||
|
|
- Additional Information
|
||
|
|
|
||
|
|
|
||
|
|
Introduction
|
||
|
|
============
|
||
|
|
|
||
|
|
This tool provides a simple interface to LIBSVM, a library for support vector
|
||
|
|
machines (http://www.csie.ntu.edu.tw/~cjlin/libsvm). It is very easy to use as
|
||
|
|
the usage and the way of specifying parameters are the same as that of LIBSVM.
|
||
|
|
|
||
|
|
Installation
|
||
|
|
============
|
||
|
|
|
||
|
|
On Windows systems, pre-built binary files are already in the
|
||
|
|
directory '..\windows', so no need to conduct installation. Now we
|
||
|
|
provide binary files only for 64bit MATLAB on Windows. If you would
|
||
|
|
like to re-build the package, please rely on the following steps.
|
||
|
|
|
||
|
|
We recommend using make.m on both MATLAB and OCTAVE. Just type 'make'
|
||
|
|
to build 'libsvmread.mex', 'libsvmwrite.mex', 'svmtrain.mex', and
|
||
|
|
'svmpredict.mex'.
|
||
|
|
|
||
|
|
On MATLAB or Octave:
|
||
|
|
|
||
|
|
>> make
|
||
|
|
|
||
|
|
If make.m does not work on MATLAB (especially for Windows), try 'mex
|
||
|
|
-setup' to choose a suitable compiler for mex. Make sure your compiler
|
||
|
|
is accessible and workable. Then type 'make' to start the
|
||
|
|
installation.
|
||
|
|
|
||
|
|
Example:
|
||
|
|
|
||
|
|
matlab>> mex -setup
|
||
|
|
(ps: MATLAB will show the following messages to setup default compiler.)
|
||
|
|
Please choose your compiler for building external interface (MEX) files:
|
||
|
|
Would you like mex to locate installed compilers [y]/n? y
|
||
|
|
Select a compiler:
|
||
|
|
[1] Microsoft Visual C/C++ version 7.1 in C:\Program Files\Microsoft Visual Studio
|
||
|
|
[0] None
|
||
|
|
Compiler: 1
|
||
|
|
Please verify your choices:
|
||
|
|
Compiler: Microsoft Visual C/C++ 7.1
|
||
|
|
Location: C:\Program Files\Microsoft Visual Studio
|
||
|
|
Are these correct?([y]/n): y
|
||
|
|
|
||
|
|
matlab>> make
|
||
|
|
|
||
|
|
On Unix systems, if neither make.m nor 'mex -setup' works, please use
|
||
|
|
Makefile and type 'make' in a command window. Note that we assume
|
||
|
|
your MATLAB is installed in '/usr/local/matlab'. If not, please change
|
||
|
|
MATLABDIR in Makefile.
|
||
|
|
|
||
|
|
Example:
|
||
|
|
linux> make
|
||
|
|
|
||
|
|
To use octave, type 'make octave':
|
||
|
|
|
||
|
|
Example:
|
||
|
|
linux> make octave
|
||
|
|
|
||
|
|
For a list of supported/compatible compilers for MATLAB, please check
|
||
|
|
the following page:
|
||
|
|
|
||
|
|
http://www.mathworks.com/support/compilers/current_release/
|
||
|
|
|
||
|
|
Usage
|
||
|
|
=====
|
||
|
|
|
||
|
|
matlab> model = svmtrain(training_label_vector, training_instance_matrix [, 'libsvm_options']);
|
||
|
|
|
||
|
|
-training_label_vector:
|
||
|
|
An m by 1 vector of training labels (type must be double).
|
||
|
|
-training_instance_matrix:
|
||
|
|
An m by n matrix of m training instances with n features.
|
||
|
|
It can be dense or sparse (type must be double).
|
||
|
|
-libsvm_options:
|
||
|
|
A string of training options in the same format as that of LIBSVM.
|
||
|
|
|
||
|
|
matlab> [predicted_label, accuracy, decision_values/prob_estimates] = svmpredict(testing_label_vector, testing_instance_matrix, model [, 'libsvm_options']);
|
||
|
|
matlab> [predicted_label] = svmpredict(testing_label_vector, testing_instance_matrix, model [, 'libsvm_options']);
|
||
|
|
|
||
|
|
-testing_label_vector:
|
||
|
|
An m by 1 vector of prediction labels. If labels of test
|
||
|
|
data are unknown, simply use any random values. (type must be double)
|
||
|
|
-testing_instance_matrix:
|
||
|
|
An m by n matrix of m testing instances with n features.
|
||
|
|
It can be dense or sparse. (type must be double)
|
||
|
|
-model:
|
||
|
|
The output of svmtrain.
|
||
|
|
-libsvm_options:
|
||
|
|
A string of testing options in the same format as that of LIBSVM.
|
||
|
|
|
||
|
|
Returned Model Structure
|
||
|
|
========================
|
||
|
|
|
||
|
|
The 'svmtrain' function returns a model which can be used for future
|
||
|
|
prediction. It is a structure and is organized as [Parameters, nr_class,
|
||
|
|
totalSV, rho, Label, ProbA, ProbB, nSV, sv_coef, SVs]:
|
||
|
|
|
||
|
|
-Parameters: parameters
|
||
|
|
-nr_class: number of classes; = 2 for regression/one-class svm
|
||
|
|
-totalSV: total #SV
|
||
|
|
-rho: -b of the decision function(s) wx+b
|
||
|
|
-Label: label of each class; empty for regression/one-class SVM
|
||
|
|
-sv_indices: values in [1,...,num_traning_data] to indicate SVs in the training set
|
||
|
|
-ProbA: pairwise probability information; empty if -b 0 or in one-class SVM
|
||
|
|
-ProbB: pairwise probability information; empty if -b 0 or in one-class SVM
|
||
|
|
-nSV: number of SVs for each class; empty for regression/one-class SVM
|
||
|
|
-sv_coef: coefficients for SVs in decision functions
|
||
|
|
-SVs: support vectors
|
||
|
|
|
||
|
|
If you do not use the option '-b 1', ProbA and ProbB are empty
|
||
|
|
matrices. If the '-v' option is specified, cross validation is
|
||
|
|
conducted and the returned model is just a scalar: cross-validation
|
||
|
|
accuracy for classification and mean-squared error for regression.
|
||
|
|
|
||
|
|
More details about this model can be found in LIBSVM FAQ
|
||
|
|
(http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html) and LIBSVM
|
||
|
|
implementation document
|
||
|
|
(http://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.pdf).
|
||
|
|
|
||
|
|
Result of Prediction
|
||
|
|
====================
|
||
|
|
|
||
|
|
The function 'svmpredict' has three outputs. The first one,
|
||
|
|
predictd_label, is a vector of predicted labels. The second output,
|
||
|
|
accuracy, is a vector including accuracy (for classification), mean
|
||
|
|
squared error, and squared correlation coefficient (for regression).
|
||
|
|
The third is a matrix containing decision values or probability
|
||
|
|
estimates (if '-b 1' is specified). If k is the number of classes
|
||
|
|
in training data, for decision values, each row includes results of
|
||
|
|
predicting k(k-1)/2 binary-class SVMs. For classification, k = 1 is a
|
||
|
|
special case. Decision value +1 is returned for each testing instance,
|
||
|
|
instead of an empty vector. For probabilities, each row contains k values
|
||
|
|
indicating the probability that the testing instance is in each class.
|
||
|
|
Note that the order of classes here is the same as 'Label' field
|
||
|
|
in the model structure.
|
||
|
|
|
||
|
|
Other Utilities
|
||
|
|
===============
|
||
|
|
|
||
|
|
A matlab function libsvmread reads files in LIBSVM format:
|
||
|
|
|
||
|
|
[label_vector, instance_matrix] = libsvmread('data.txt');
|
||
|
|
|
||
|
|
Two outputs are labels and instances, which can then be used as inputs
|
||
|
|
of svmtrain or svmpredict.
|
||
|
|
|
||
|
|
A matlab function libsvmwrite writes Matlab matrix to a file in LIBSVM format:
|
||
|
|
|
||
|
|
libsvmwrite('data.txt', label_vector, instance_matrix)
|
||
|
|
|
||
|
|
The instance_matrix must be a sparse matrix. (type must be double)
|
||
|
|
For 32bit and 64bit MATLAB on Windows, pre-built binary files are ready
|
||
|
|
in the directory `..\windows', but in future releases, we will only
|
||
|
|
include 64bit MATLAB binary files.
|
||
|
|
|
||
|
|
These codes are prepared by Rong-En Fan and Kai-Wei Chang from National
|
||
|
|
Taiwan University.
|
||
|
|
|
||
|
|
Examples
|
||
|
|
========
|
||
|
|
|
||
|
|
Train and test on the provided data heart_scale:
|
||
|
|
|
||
|
|
matlab> [heart_scale_label, heart_scale_inst] = libsvmread('../heart_scale');
|
||
|
|
matlab> model = svmtrain(heart_scale_label, heart_scale_inst, '-c 1 -g 0.07');
|
||
|
|
matlab> [predict_label, accuracy, dec_values] = svmpredict(heart_scale_label, heart_scale_inst, model); % test the training data
|
||
|
|
|
||
|
|
For probability estimates, you need '-b 1' for training and testing:
|
||
|
|
|
||
|
|
matlab> [heart_scale_label, heart_scale_inst] = libsvmread('../heart_scale');
|
||
|
|
matlab> model = svmtrain(heart_scale_label, heart_scale_inst, '-c 1 -g 0.07 -b 1');
|
||
|
|
matlab> [heart_scale_label, heart_scale_inst] = libsvmread('../heart_scale');
|
||
|
|
matlab> [predict_label, accuracy, prob_estimates] = svmpredict(heart_scale_label, heart_scale_inst, model, '-b 1');
|
||
|
|
|
||
|
|
To use precomputed kernel, you must include sample serial number as
|
||
|
|
the first column of the training and testing data (assume your kernel
|
||
|
|
matrix is K, # of instances is n):
|
||
|
|
|
||
|
|
matlab> K1 = [(1:n)', K]; % include sample serial number as first column
|
||
|
|
matlab> model = svmtrain(label_vector, K1, '-t 4');
|
||
|
|
matlab> [predict_label, accuracy, dec_values] = svmpredict(label_vector, K1, model); % test the training data
|
||
|
|
|
||
|
|
We give the following detailed example by splitting heart_scale into
|
||
|
|
150 training and 120 testing data. Constructing a linear kernel
|
||
|
|
matrix and then using the precomputed kernel gives exactly the same
|
||
|
|
testing error as using the LIBSVM built-in linear kernel.
|
||
|
|
|
||
|
|
matlab> [heart_scale_label, heart_scale_inst] = libsvmread('../heart_scale');
|
||
|
|
matlab>
|
||
|
|
matlab> % Split Data
|
||
|
|
matlab> train_data = heart_scale_inst(1:150,:);
|
||
|
|
matlab> train_label = heart_scale_label(1:150,:);
|
||
|
|
matlab> test_data = heart_scale_inst(151:270,:);
|
||
|
|
matlab> test_label = heart_scale_label(151:270,:);
|
||
|
|
matlab>
|
||
|
|
matlab> % Linear Kernel
|
||
|
|
matlab> model_linear = svmtrain(train_label, train_data, '-t 0');
|
||
|
|
matlab> [predict_label_L, accuracy_L, dec_values_L] = svmpredict(test_label, test_data, model_linear);
|
||
|
|
matlab>
|
||
|
|
matlab> % Precomputed Kernel
|
||
|
|
matlab> model_precomputed = svmtrain(train_label, [(1:150)', train_data*train_data'], '-t 4');
|
||
|
|
matlab> [predict_label_P, accuracy_P, dec_values_P] = svmpredict(test_label, [(1:120)', test_data*train_data'], model_precomputed);
|
||
|
|
matlab>
|
||
|
|
matlab> accuracy_L % Display the accuracy using linear kernel
|
||
|
|
matlab> accuracy_P % Display the accuracy using precomputed kernel
|
||
|
|
|
||
|
|
Note that for testing, you can put anything in the
|
||
|
|
testing_label_vector. For more details of precomputed kernels, please
|
||
|
|
read the section ``Precomputed Kernels'' in the README of the LIBSVM
|
||
|
|
package.
|
||
|
|
|
||
|
|
Additional Information
|
||
|
|
======================
|
||
|
|
|
||
|
|
This interface was initially written by Jun-Cheng Chen, Kuan-Jen Peng,
|
||
|
|
Chih-Yuan Yang and Chih-Huai Cheng from Department of Computer
|
||
|
|
Science, National Taiwan University. The current version was prepared
|
||
|
|
by Rong-En Fan and Ting-Fan Wu. If you find this tool useful, please
|
||
|
|
cite LIBSVM as follows
|
||
|
|
|
||
|
|
Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support
|
||
|
|
vector machines. ACM Transactions on Intelligent Systems and
|
||
|
|
Technology, 2:27:1--27:27, 2011. Software available at
|
||
|
|
http://www.csie.ntu.edu.tw/~cjlin/libsvm
|
||
|
|
|
||
|
|
For any question, please contact Chih-Jen Lin <cjlin@csie.ntu.edu.tw>,
|
||
|
|
or check the FAQ page:
|
||
|
|
|
||
|
|
http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html#/Q10:_MATLAB_interface
|