The Library
Help/Info
Current Release









Last Modified:
Jan 03, 2011

Machine Learning



This page documents all the machine learning algorithms present in the library. In particular, there are algorithms for performing classification, regression, clustering, anomaly detection, and feature ranking, as well as algorithms for doing more specialized computations.

A good tutorial and introduction to the general concepts used by most of the objects in this part of the library can be found in the svm example program. After reading this example another good one to consult would be the model selection example program. Finally, if you came here looking for a binary classification or regression tool then I would try the krr_trainer first as it is generally the easiest method to use.

The major design goal of this portion of the library is to provide a highly modular and simple architecture for dealing with kernel algorithms. Towards this end, dlib takes a generic programming approach using C++ templates. In particular, each algorithm is parameterized to allow a user to supply either one of the predefined dlib kernels (e.g. RBF operating on column vectors), or a new user defined kernel. Moreover, the implementations of the algorithms are totally separated from the data on which they operate. This makes the dlib implementation generic enough to operate on any kind of data, be it column vectors, images, or some other form of structured data. All that is necessary is an appropriate kernel.


Paper Describing dlib Machine Learning

Davis E. King. Dlib-ml: A Machine Learning Toolkit. 
   Journal of Machine Learning Research 10, pp. 1755-1758, 2009

@Article{dlib09,
  author = {Davis E. King},
  title = {Dlib-ml: A Machine Learning Toolkit},
  journal = {Journal of Machine Learning Research},
  year = {2009},
  volume = {10},
  pages = {1755-1758},
}
         

Primary Algorithms

Binary Classification
Multiclass Classification
Regression
Unsupervised
Semi-Supervised
Feature Selection

Other Tools

Validation
Trainer Adapters
Kernels
Function Objects
Data IO
Miscellaneous
[top]

batch



This is a convenience function for creating batch_trainer objects.

Specification: dlib/svm/pegasos_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

batch_cached



This is a convenience function for creating batch_trainer objects that are setup to use a kernel matrix cache.

Specification: dlib/svm/pegasos_abstract.h
File to include: dlib/svm.h

[top]

batch_trainer



This is a batch trainer object that is meant to wrap online trainer objects that create decision_functions. It turns an online learning algorithm such as svm_pegasos into a batch learning object. This allows you to use objects like svm_pegasos with functions (e.g. cross_validate_trainer) that expect batch mode training objects.

Specification: dlib/svm/pegasos_abstract.h
File to include: dlib/svm.h

[top]

compute_mean_squared_distance



This is a function that simply finds the average squared distance between all pairs of a set of data samples. It is often convenient to use the reciprocal of this value as the estimate of the gamma parameter of the radial_basis_kernel.

Specification: dlib/svm/feature_ranking_abstract.h
File to include: dlib/svm.h

[top]

cross_validate_multiclass_trainer



Performs k-fold cross validation on a user supplied multiclass classification trainer object such as the one_vs_one_trainer. The result is described by a confusion matrix.

Specification: dlib/svm/cross_validate_multiclass_trainer_abstract.h
File to include: dlib/svm.h
Code Examples: 1, 2

[top]

cross_validate_regression_trainer



Performs k-fold cross validation on a user supplied regression trainer object such as the svr_trainer and returns the mean squared error.

Specification: dlib/svm/cross_validate_regression_trainer_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

cross_validate_trainer



Performs k-fold cross validation on a user supplied binary classification trainer object such as the svm_nu_trainer or rbf_network_trainer.

Specification: dlib/svm/svm_abstract.h
File to include: dlib/svm.h
Code Examples: 1, 2

[top]

cross_validate_trainer_threaded



Performs k-fold cross validation on a user supplied binary classification trainer object such as the svm_nu_trainer or rbf_network_trainer. This function does the same thing as cross_validate_trainer except this function also allows you to specify how many threads of execution to use. So you can use this function to take advantage of a multi-core system to perform cross validation faster.

Specification: dlib/svm/svm_threaded_abstract.h
File to include: dlib/svm_threaded.h

[top]

decision_function



This object represents a classification or regression function that was learned by a kernel based learning algorithm. Therefore, it is a function object that takes a sample object and returns a scalar value.

Specification: dlib/svm/function_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

discriminant_pca



This object implements the Discriminant PCA technique described in the paper:
A New Discriminant Principal Component Analysis Method with Partial Supervision (2009) by Dan Sun and Daoqiang Zhang
This algorithm is basically a straightforward generalization of the classical PCA technique to handle partially labeled data. It is useful if you want to learn a linear dimensionality reduction rule using a bunch of data that is partially labeled.

Specification: dlib/statistics/dpca_abstract.h
File to include: dlib/statistics.h

[top]

distance_function



This object represents a point in kernel induced feature space. You may use this object to find the distance from the point it represents to points in input space as well as other points represented by distance_functions.

Specification: dlib/svm/function_abstract.h
File to include: dlib/svm.h

[top]

empirical_kernel_map



This object represents a map from objects of sample_type (the kind of object a kernel function operates on) to finite dimensional column vectors which represent points in the kernel feature space defined by whatever kernel is used with this object.

To use the empirical_kernel_map you supply it with a particular kernel and a set of basis samples. After that you can present it with new samples and it will project them into the part of kernel feature space spanned by your basis samples.

This means the empirical_kernel_map is a tool you can use to very easily kernelize any algorithm that operates on column vectors. All you have to do is select a set of basis samples and then use the empirical_kernel_map to project all your data points into the part of kernel feature space spanned by those basis samples. Then just run your normal algorithm on the output vectors and it will be effectively kernelized.

Regarding methods to select a set of basis samples, if you are working with only a few thousand samples then you can just use all of them as basis samples. Alternatively, the linearly_independent_subset_finder often works well for selecting a basis set. I also find that picking a random subset typically works well.



Specification: dlib/svm/empirical_kernel_map_abstract.h
File to include: dlib/svm.h
Code Examples: 1, 2

[top]

fill_lisf



This is a simple function for filling a linearly_independent_subset_finder with data points by using random sampling.

Specification: dlib/svm/linearly_independent_subset_finder_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

find_approximate_k_nearest_neighbors



This function is a simple approximate form of find_k_nearest_neighbors. Instead of checking all possible edges it randomly samples a large number of them and then performs exact k-nearest-neighbors on that randomly selected subset.

Specification: dlib/manifold_regularization/graph_creation_abstract.h
File to include: dlib/manifold_regularization.h

[top]

find_clusters_using_kmeans



This is just a simple linear kmeans clustering implementation.

Specification: dlib/svm/kkmeans_abstract.h
File to include: dlib/svm.h

[top]

find_gamma_with_big_centroid_gap



This is a function that tries to pick a reasonable default value for the gamma parameter of the radial_basis_kernel. It picks the parameter that gives the largest separation between the centroids, in kernel feature space, of two classes of data.

Specification: dlib/svm/feature_ranking_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

find_k_nearest_neighbors



This is a function which finds all the k nearest neighbors of a set of points and outputs the result as a vector of sample_pair objects. It takes O(n^2) where n is the number of data samples. A faster approximate version is provided by find_approximate_k_nearest_neighbors.

Specification: dlib/manifold_regularization/graph_creation_abstract.h
File to include: dlib/manifold_regularization.h

[top]

find_percent_shortest_edges_randomly



This function is a simple approximate form of find_k_nearest_neighbors. Instead of checking all possible edges it randomly samples a large number of them and then returns the best ones.

Specification: dlib/manifold_regularization/graph_creation_abstract.h
File to include: dlib/manifold_regularization.h
Code Examples: 1

[top]

is_binary_classification_problem



This function simply takes two vectors, the first containing feature vectors and the second containing labels, and reports back if the two could possibly contain data for a well formed classification problem.

Specification: dlib/svm/svm_abstract.h
File to include: dlib/svm.h

[top]

is_learning_problem



This function simply takes two vectors, the first containing feature vectors and the second containing labels, and reports back if the two could possibly contain data for a well formed learning problem. In this case it just means that the two vectors have the same length and aren't empty.

Specification: dlib/svm/svm_abstract.h
File to include: dlib/svm.h

[top]

kcentroid



This object represents a weighted sum of sample points in a kernel induced feature space. It can be used to kernelize any algorithm that requires only the ability to perform vector addition, subtraction, scalar multiplication, and inner products.

An example use of this object is as an online algorithm for recursively estimating the centroid of a sequence of training points. This object then allows you to compute the distance between the centroid and any test points. So you can use this object to predict how similar a test point is to the data this object has been trained on (larger distances from the centroid indicate dissimilarity/anomalous points).

The object internally keeps a set of "dictionary vectors" that are used to represent the centroid. It manages these vectors using the sparsification technique described in the paper The Kernel Recursive Least Squares Algorithm by Yaakov Engel. This technique allows us to keep the number of dictionary vectors down to a minimum. In fact, the object has a user selectable tolerance parameter that controls the trade off between accuracy and number of stored dictionary vectors.



Specification: dlib/svm/kcentroid_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

kernel_matrix



This is a simple set of functions that makes it easy to turn a kernel object and a set of samples into a kernel matrix. It takes these two things and returns a matrix expression that represents the kernel matrix.

Specification: dlib/svm/kernel_matrix_abstract.h
File to include: dlib/svm.h

[top]

kkmeans



This is an implementation of a kernelized k-means clustering algorithm. It performs k-means clustering by using the kcentroid object.

Specification: dlib/svm/kkmeans_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

krls



This is an implementation of the kernel recursive least squares algorithm described in the paper The Kernel Recursive Least Squares Algorithm by Yaakov Engel.

The long and short of this algorithm is that it is an online kernel based regression algorithm. You give it samples (x,y) and it learns the function f(x) == y. For a detailed description of the algorithm read the above paper.



Specification: dlib/svm/krls_abstract.h
File to include: dlib/svm.h
Code Examples: 1, 2

[top]

krr_trainer



Performs kernel ridge regression and outputs a decision_function that represents the learned function.

The implementation is done using the empirical_kernel_map and linearly_independent_subset_finder and thus allows you to run the algorithm on large datasets and obtain sparse outputs. It is also capable of automatically estimating its regularization parameter using leave-one-out cross-validation.

Specification: dlib/svm/krr_trainer_abstract.h
File to include: dlib/svm.h
Code Examples: 1, 2

[top]

linearly_independent_subset_finder



This is an implementation of an online algorithm for recursively finding a set (aka dictionary) of linearly independent vectors in a kernel induced feature space. To use it you decide how large you would like the dictionary to be and then you feed it sample points.

The implementation uses the Approximately Linearly Dependent metric described in the paper The Kernel Recursive Least Squares Algorithm by Yaakov Engel to decide which points are more linearly independent than others. The metric is simply the squared distance between a test point and the subspace spanned by the set of dictionary vectors.

Each time you present this object with a new sample point it calculates the projection distance and if it is sufficiently large then this new point is included into the dictionary. Note that this object can be configured to have a maximum size. Once the max dictionary size is reached each new point kicks out a previous point. This is done by removing the dictionary vector that has the smallest projection distance onto the others. That is, the "least linearly independent" vector is removed to make room for the new one.



Specification: dlib/svm/linearly_independent_subset_finder_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

linear_kernel



This object represents a linear function kernel for use with kernel learning machines.

Specification: dlib/svm/kernel_abstract.h
File to include: dlib/svm.h

[top]

linear_manifold_regularizer



Many learning algorithms attempt to minimize a function that, at a high level, looks like this:

   f(w) == complexity + training_set_error

The idea is to find the set of parameters, w, that gives low error on your training data but also is not "complex" according to some particular measure of complexity. This strategy of penalizing complexity is usually called regularization.

In the above setting, all the training data consists of labeled samples. However, it would be nice to be able to benefit from unlabeled data. The idea of manifold regularization is to extract useful information from unlabeled data by first defining which data samples are "close" to each other (perhaps by using their 3 nearest neighbors) and then adding a term to the above function that penalizes any decision rule which produces different outputs on data samples which we have designated as being close.

It turns out that it is possible to transform these manifold regularized learning problems into the normal form shown above by applying a certain kind of preprocessing to all our data samples. Once this is done we can use a normal learning algorithm, such as the svm_c_linear_trainer, on just the labeled data samples and obtain the same output as the manifold regularized learner would have produced.

The linear_manifold_regularizer is a tool for creating this preprocessing transformation. In particular, the transformation is linear. That is, it is just a matrix you multiply with all your samples. For a more detailed discussion of this topic you should consult the following paper. In particular, see section 4.2. This object computes the inverse T matrix described in that section.

Linear Manifold Regularization for Large Scale Semi-supervised Learning by Vikas Sindhwani, Partha Niyogi, and Mikhail Belkin



Specification: dlib/manifold_regularization/linear_manifold_regularizer_abstract.h
File to include: dlib/manifold_regularization.h
Code Examples: 1

[top]

load_libsvm_formatted_data



This is a function that loads the data from a file that uses the LIBSVM format. It loads the data into a std::vector of sparse vectors. If you want to load data into dense vectors (i.e. dlib::matrix objects) then you can use the sparse_to_dense function to perform the conversion.

Specification: dlib/data_io/libsvm_io_abstract.h
File to include: dlib/data_io.h

[top]

mlp



This object represents a multilayer layer perceptron network that is trained using the back propagation algorithm. The training algorithm also incorporates the momentum method. That is, each round of back propagation training also adds a fraction of the previous update. This fraction is controlled by the momentum term set in the constructor.

It is worth noting that a MLP is, in general, very inferior to modern kernel algorithms such as the support vector machine. So if you haven't tried any other techniques with your data you really should.



Specification: dlib/mlp/mlp_kernel_abstract.h
File to include: dlib/mlp.h
Code Examples: 1

Implementations:
mlp_kernel_1:
This is implemented in the obvious way.
kernel_1a
is a typedef for mlp_kernel_1
kernel_1a_c
is a typedef for kernel_1a that checks its preconditions.
[top]

normalized_function



This object represents a container for another function object and an instance of the vector_normalizer object. It automatically normalizes all inputs before passing them off to the contained function object.

Specification: dlib/svm/function_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

null_trainer



This is a convenience function for creating null_trainer_type objects.

Specification: dlib/svm/null_trainer_abstract.h
File to include: dlib/svm.h

[top]

null_trainer_type



This object is a simple tool for turning a decision_function (or any object with an interface compatible with decision_function) into a trainer object that always returns the original decision function when you try to train with it.

dlib contains a few "training post processing" algorithms (e.g. reduced and reduced2). These tools take in a trainer object, tell it to perform training, and then they take the output decision function and do some kind of post processing to it. The null_trainer_type object is useful because you can use it to run an already learned decision function through the training post processing algorithms by turning a decision function into a null_trainer_type and then giving it to a post processor.



Specification: dlib/svm/null_trainer_abstract.h
File to include: dlib/svm.h

[top]

offset_kernel



This object represents a kernel with a fixed value offset added to it.

Specification: dlib/svm/kernel_abstract.h
File to include: dlib/svm.h

[top]

one_vs_all_decision_function



This object represents a multiclass classifier built out of a set of binary classifiers. Each binary classifier is used to vote for the correct multiclass label using a one vs. all strategy. Therefore, if you have N classes then there will be N binary classifiers inside this object.

Specification: dlib/svm/one_vs_all_decision_function_abstract.h
File to include: dlib/svm.h

[top]

one_vs_all_trainer



This object is a tool for turning a bunch of binary classifiers into a multiclass classifier. It does this by training the binary classifiers in a one vs. all fashion. That is, if you have N possible classes then it trains N binary classifiers which are then used to vote on the identity of a test sample.

Specification: dlib/svm/one_vs_all_trainer_abstract.h
File to include: dlib/svm.h

[top]

one_vs_one_decision_function



This object represents a multiclass classifier built out of a set of binary classifiers. Each binary classifier is used to vote for the correct multiclass label using a one vs. one strategy. Therefore, if you have N classes then there will be N*(N-1)/2 binary classifiers inside this object.

Specification: dlib/svm/one_vs_one_decision_function_abstract.h
File to include: dlib/svm.h
Code Examples: 1, 2

[top]

one_vs_one_trainer



This object is a tool for turning a bunch of binary classifiers into a multiclass classifier. It does this by training the binary classifiers in a one vs. one fashion. That is, if you have N possible classes then it trains N*(N-1)/2 binary classifiers which are then used to vote on the identity of a test sample.

Specification: dlib/svm/one_vs_one_trainer_abstract.h
File to include: dlib/svm.h
Code Examples: 1, 2

[top]

pick_initial_centers



This is a function that you can use to seed data clustering algorithms like the kkmeans clustering method. What it does is pick reasonable starting points for clustering by basically trying to find a set of points that are all far away from each other.

Specification: dlib/svm/kkmeans_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

polynomial_kernel



This object represents a polynomial kernel for use with kernel learning machines.

Specification: dlib/svm/kernel_abstract.h
File to include: dlib/svm.h

[top]

probabilistic



This is a trainer adapter which simply runs the trainer it is given though the train_probabilistic_decision_function function.

Specification: dlib/svm/svm_abstract.h
File to include: dlib/svm.h

[top]

probabilistic_decision_function



This object represents a binary decision function for use with kernel-based learning-machines. It returns an estimate of the probability that a given sample is in the +1 class.

Specification: dlib/svm/function_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

probabilistic_function



This object represents a binary decision function for use with any kind of binary classifier. It returns an estimate of the probability that a given sample is in the +1 class.

Specification: dlib/svm/function_abstract.h
File to include: dlib/svm.h

[top]

projection_function



This object represents a function that takes a data sample and projects it into kernel feature space. The result is a real valued column vector that represents a point in a kernel feature space. Instances of this object are created using the empirical_kernel_map.

Specification: dlib/svm/function_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

radial_basis_kernel



This object represents a radial basis function kernel for use with kernel learning machines.

Specification: dlib/svm/kernel_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

randomize_samples



Randomizes the order of samples in a column vector containing sample data.

Specification: dlib/svm/svm_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

rank_features



Finds a ranking of the top N (a user supplied parameter) features in a set of data from a two class classification problem. It does this by computing the distance between the centroids of both classes in kernel defined feature space. Good features are then ones that result in the biggest separation between the two centroids.

Specification: dlib/svm/feature_ranking_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

rbf_network_trainer



Trains a radial basis function network and outputs a decision_function. This object can be used for either regression or binary classification problems. It's worth pointing out that this object is essentially an unregularized version of kernel ridge regression. This means you should really prefer to use kernel ridge regression instead.

Specification: dlib/svm/rbf_network_abstract.h
File to include: dlib/svm.h

[top]

reduced



This is a convenience function for creating reduced_decision_function_trainer objects.

Specification: dlib/svm/reduced_abstract.h
File to include: dlib/svm.h

[top]

reduced2



This is a convenience function for creating reduced_decision_function_trainer2 objects.

Specification: dlib/svm/reduced_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

reduced_decision_function_trainer



This is a batch trainer object that is meant to wrap other batch trainer objects that create decision_function objects. It performs post processing on the output decision_function objects with the intent of representing the decision_function with fewer basis vectors.

Specification: dlib/svm/reduced_abstract.h
File to include: dlib/svm.h

[top]

reduced_decision_function_trainer2



This is a batch trainer object that is meant to wrap other batch trainer objects that create decision_function objects. It performs post processing on the output decision_function objects with the intent of representing the decision_function with fewer basis vectors.

It begins by performing the same post processing as the reduced_decision_function_trainer object but it also performs a global gradient based optimization to further improve the results.



Specification: dlib/svm/reduced_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

remove_long_edges



This is a simple function for removing edges with a large distance value from a vector of sample_pairs.

Specification: dlib/manifold_regularization/graph_creation_abstract.h
File to include: dlib/manifold_regularization.h

[top]

remove_percent_longest_edges



This is a simple function for removing edges with a large distance value from a vector of sample_pairs.

Specification: dlib/manifold_regularization/graph_creation_abstract.h
File to include: dlib/manifold_regularization.h

[top]

remove_percent_shortest_edges



This is a simple function for removing edges with a small distance value from a vector of sample_pairs.

Specification: dlib/manifold_regularization/graph_creation_abstract.h
File to include: dlib/manifold_regularization.h

[top]

remove_short_edges



This is a simple function for removing edges with a small distance value from a vector of sample_pairs.

Specification: dlib/manifold_regularization/graph_creation_abstract.h
File to include: dlib/manifold_regularization.h

[top]

roc_c1_trainer



This is a convenience function for creating roc_trainer_type objects that are setup to pick a point on the ROC curve with respect to the +1 class.

Specification: dlib/svm/roc_trainer_abstract.h
File to include: dlib/svm.h

[top]

roc_c2_trainer



This is a convenience function for creating roc_trainer_type objects that are setup to pick a point on the ROC curve with respect to the -1 class.

Specification: dlib/svm/roc_trainer_abstract.h
File to include: dlib/svm.h

[top]

roc_trainer_type



This object is a simple trainer post processor that allows you to easily adjust the bias term in a trained decision_function object. That is, this object lets you pick a point on the ROC curve and it will adjust the bias term appropriately.

So for example, suppose you wanted to set the bias term so that the accuracy of your decision function on +1 labeled samples was 99%. To do this you would use an instance of this object declared as follows: roc_trainer_type<trainer_type>(your_trainer, 0.99, +1);



Specification: dlib/svm/roc_trainer_abstract.h
File to include: dlib/svm.h

[top]

rvm_regression_trainer



Trains a relevance vector machine for solving regression problems. Outputs a decision_function that represents the learned regression function.

The implementation of the RVM training algorithm used by this library is based on the following paper:
Tipping, M. E. and A. C. Faul (2003). Fast marginal likelihood maximisation for sparse Bayesian models. In C. M. Bishop and B. J. Frey (Eds.), Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, Key West, FL, Jan 3-6.


Specification: dlib/svm/rvm_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

rvm_trainer



Trains a relevance vector machine for solving binary classification problems. Outputs a decision_function that represents the learned classifier.

The implementation of the RVM training algorithm used by this library is based on the following paper:
Tipping, M. E. and A. C. Faul (2003). Fast marginal likelihood maximisation for sparse Bayesian models. In C. M. Bishop and B. J. Frey (Eds.), Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, Key West, FL, Jan 3-6.


Specification: dlib/svm/rvm_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

sample_pair



This object is intended to represent an edge in an undirected graph which has data samples at its vertices.

Specification: dlib/manifold_regularization/sample_pair_abstract.h
File to include: dlib/manifold_regularization.h
Code Examples: 1

[top]

save_libsvm_formatted_data



This is actually a pair of overloaded functions. Between the two of them they let you save sparse or dense data vectors to file using the LIBSVM format.

Specification: dlib/data_io/libsvm_io_abstract.h
File to include: dlib/data_io.h

[top]

select_all_distinct_labels



This is a function which determines all distinct values present in a std::vector and returns the result.

Specification: dlib/svm/multiclass_tools_abstract.h
File to include: dlib/svm.h

[top]

sigmoid_kernel



This object represents a sigmoid kernel for use with kernel learning machines.

Specification: dlib/svm/kernel_abstract.h
File to include: dlib/svm.h

[top]

simplify_linear_decision_function



This is a set of functions that takes various forms of linear decision functions and collapses them down so that they only compute a single dot product when invoked.

Specification: dlib/svm/simplify_linear_decision_function_abstract.h
File to include: dlib/svm.h

[top]

sort_basis_vectors



A kernel based learning method ultimately needs to select a set of basis functions represented by a particular choice of kernel and a set of basis vectors. sort_basis_vectors() is a function which attempts to perform supervised basis set selection. In particular, you give it a candidate set of basis vectors and it sorts them according to how useful they are for solving a particular decision problem.

Specification: dlib/svm/sort_basis_vectors_abstract.h
File to include: dlib/svm.h

[top]

sparse_linear_kernel



This object represents a linear function kernel for use with kernel learning machines that operate on sparse vectors.

Specification: dlib/svm/sparse_kernel_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

sparse_polynomial_kernel



This object represents a polynomial kernel for use with kernel learning machines that operate sparse vectors.

Specification: dlib/svm/sparse_kernel_abstract.h
File to include: dlib/svm.h

[top]

sparse_radial_basis_kernel



This object represents a radial basis function kernel for use with kernel learning machines that operate sparse vectors.

Specification: dlib/svm/sparse_kernel_abstract.h
File to include: dlib/svm.h

[top]

sparse_sigmoid_kernel



This object represents a sigmoid kernel for use with kernel learning machines that operate on sparse vectors.

Specification: dlib/svm/sparse_kernel_abstract.h
File to include: dlib/svm.h

[top]

sparse_to_dense



This is a simple function that takes a std::vector of sparse vectors and returns to you the equivalent std::vector of dense vectors.

Specification: dlib/data_io/libsvm_io_abstract.h
File to include: dlib/data_io.h

[top]

squared_euclidean_distance



This is a simple function object that computes squared euclidean distance between two matrix objects.

Specification: dlib/manifold_regularization/function_objects_abstract.h
File to include: dlib/manifold_regularization.h
Code Examples: 1

[top]

svm_c_ekm_trainer



This object represents a tool for training the C formulation of a support vector machine for solving binary classification problems. It is implemented using the empirical_kernel_map to kernelize the svm_c_linear_trainer. This makes it a very fast algorithm capable of learning from very large datasets.

Specification: dlib/svm/svm_c_ekm_trainer_abstract.h
File to include: dlib/svm.h

[top]

svm_c_linear_trainer



This object represents a tool for training the C formulation of a support vector machine to solve binary classification problems. It is optimized for the case where linear kernels are used and is implemented using the oca optimizer and uses the exact line search described in the following paper:
Optimized Cutting Plane Algorithm for Large-Scale Risk Minimization by Vojtech Franc, Soren Sonnenburg; Journal of Machine Learning Research, 10(Oct):2157--2192, 2009.


Specification: dlib/svm/svm_c_linear_trainer_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

svm_c_trainer



Trains a C support vector machine for solving binary classification problems and outputs a decision_function. It is implemented using the SMO algorithm.

The implementation of the C-SVM training algorithm used by this library is based on the following paper:

Specification: dlib/svm/svm_c_trainer_abstract.h
File to include: dlib/svm.h

[top]

svm_nu_trainer



Trains a nu support vector machine for solving binary classification problems and outputs a decision_function. It is implemented using the SMO algorithm.

The implementation of the nu-svm training algorithm used by this library is based on the following excellent papers:

Specification: dlib/svm/svm_nu_trainer_abstract.h
File to include: dlib/svm.h
Code Examples: 1, 2

[top]

svm_one_class_trainer



Trains a one-class support vector classifier and outputs a decision_function. It is implemented using the SMO algorithm.

The implementation of the one-class training algorithm used by this library is based on the following paper:

Specification: dlib/svm/svm_one_class_trainer_abstract.h
File to include: dlib/svm.h

[top]

svm_pegasos



This object implements an online algorithm for training a support vector machine for solving binary classification problems.

The implementation of the Pegasos algorithm used by this object is based on the following excellent paper:

Pegasos: Primal estimated sub-gradient solver for SVM (2007) by Shai Shalev-Shwartz, Yoram Singer, Nathan Srebro In ICML

This SVM training algorithm has two interesting properties. First, the pegasos algorithm itself converges to the solution in an amount of time unrelated to the size of the training set (in addition to being quite fast to begin with). This makes it an appropriate algorithm for learning from very large datasets. Second, this object uses the kcentroid object to maintain a sparse approximation of the learned decision function. This means that the number of support vectors in the resulting decision function is also unrelated to the size of the dataset (in normal SVM training algorithms, the number of support vectors grows approximately linearly with the size of the training set).



Specification: dlib/svm/pegasos_abstract.h
File to include: dlib/svm.h
Code Examples: 1, 2

[top]

svr_trainer



This object implements a trainer for performing epsilon-insensitive support vector regression. It is implemented using the SMO algorithm.

The implementation of the eps-SVR training algorithm used by this object is based on the following paper:

Specification: dlib/svm/svr_trainer_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

test_binary_decision_function



Tests a decision_function that represents a binary decision function and returns the test accuracy.

Specification: dlib/svm/svm_abstract.h
File to include: dlib/svm.h

[top]

test_multiclass_decision_function



Tests a multiclass decision function (e.g. one_vs_one_decision_function) and returns a confusion matrix describing the results.

Specification: dlib/svm/cross_validate_multiclass_trainer_abstract.h
File to include: dlib/svm.h
Code Examples: 1, 2

[top]

test_regression_function



Tests a regression function (e.g. decision_function) and returns the mean squared error.

Specification: dlib/svm/cross_validate_regression_trainer_abstract.h
File to include: dlib/svm.h

[top]

train_probabilistic_decision_function



Trains a probabilistic_function using some sort of binary classification trainer object such as the svm_nu_trainer or krr_trainer.

The probability model is created by using the technique described in the following papers:
Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods by John C. Platt. March 26, 1999
A Note on Platt's Probabilistic Outputs for Support Vector Machines by Hsuan-Tien Lin, Chih-Jen Lin, and Ruby C. Weng


Specification: dlib/svm/svm_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

use_gaussian_weights



This is a simple function object that takes a single argument which should be an object similar to sample_pair.

Specification: dlib/manifold_regularization/function_objects_abstract.h
File to include: dlib/manifold_regularization.h
Code Examples: 1

[top]

use_weights_of_one



This is a simple function object that takes a single argument and always returns 1

Specification: dlib/manifold_regularization/function_objects_abstract.h
File to include: dlib/manifold_regularization.h

[top]

vector_normalizer



This object represents something that can learn to normalize a set of column vectors. In particular, normalized column vectors should have zero mean and a variance of one.

Specification: dlib/statistics/statistics_abstract.h
File to include: dlib/statistics.h
Code Examples: 1

[top]

vector_normalizer_pca



This object represents something that can learn to normalize a set of column vectors. In particular, normalized column vectors should have zero mean and a variance of one. This object also uses principal component analysis for the purposes of reducing the number of elements in a vector.

Specification: dlib/statistics/statistics_abstract.h
File to include: dlib/statistics.h

[top]

verbose_batch



This is a convenience function for creating batch_trainer objects. This function generates a batch_trainer that will print status messages to standard output so that you can observe the progress of a training algorithm.

Specification: dlib/svm/pegasos_abstract.h
File to include: dlib/svm.h
Code Examples: 1

[top]

verbose_batch_cached



This is a convenience function for creating batch_trainer objects. This function generates a batch_trainer that will print status messages to standard output so that you can observe the progress of a training algorithm. It will also be configured to use a kernel matrix cache.

Specification: dlib/svm/pegasos_abstract.h
File to include: dlib/svm.h