Sequential Minimal Optimization - Algorithm

Algorithm

SMO is an iterative algorithm for solving the optimization problem described above. SMO breaks this problem into a series of smallest possible sub-problems, which are then solved analytically. Because of the linear equality constraint involving the Lagrange multipliers, the smallest possible problem involves two such multipliers. Then, for any two multipliers and, the constraints are reduced to:

and this reduced problem can be solved analytically.

The algorithm proceeds as follows:

  1. Find a Lagrange multiplier that violates the Karush–Kuhn–Tucker (KKT) conditions for the optimization problem.
  2. Pick a second multiplier and optimize the pair .
  3. Repeat steps 1 and 2 until convergence.

When all the Lagrange multipliers satisfy the KKT conditions (within a user-defined tolerance), the problem has been solved. Although this algorithm is guaranteed to converge, heuristics are used to choose the pair of multipliers so as to accelerate the rate of convergence.

Read more about this topic:  Sequential Minimal Optimization

Other articles related to "algorithm":

Timeline Of Algorithms - 1960s
... Hoare 1962 - Ford–Fulkerson algorithm developed by L ... Fulkerson 1962 - Bresenham's line algorithm developed by Jack E ... Fedorenko 1965 - Cooley–Tukey algorithm rediscovered by James Cooley and John Tukey 1965 - Levenshtein distance developed by Vladimir Levenshtein ...
Tomasulo Algorithm
... The Tomasulo algorithm is a hardware algorithm developed in 1967 by Robert Tomasulo from IBM ... This algorithm differs from scoreboarding in that it utilizes register renaming ... The Tomasulo algorithm also uses a common data bus (CDB) on which computed values are broadcast to all the reservation stations that may need it ...
Barcode Reader - New Algorithms For Barcode Decoding - Symbology Decoding Algorithm
... The Symbology Decoding Algorithm for barcode scanners is the first symbology-based algorithm for decoding ... from the entire image to detect transitions in the signal, whereas the traditional algorithm relies on the maxima and minima ... The Symbology Decoding Algorithm for Bar Code Scanners exhibited high resilience to blur and noise when tested on 1D Universal Product Codes ...
Algorithm - History: Development of The Notion of "algorithm" - History After 1950
... of efforts have been directed toward further refinement of the definition of "algorithm", and activity is on-going because of issues surrounding, in ... For more, see Algorithm characterizations ...
Markov Chain Monte Carlo - Random Walk Algorithms
... Here are some random walk MCMC methods Metropolis–Hastings algorithm Generates a random walk using a proposal density and a method for rejecting proposed moves ... Multiple-try Metropolis A variation of the Metropolis–Hastings algorithm that allows multiple trials at each point ... This allows the algorithm to generally take larger steps at each iteration, which helps combat problems intrinsic to large dimensional problems ...