Random Forest - Learning Algorithm

Learning Algorithm

Each tree is constructed using the following algorithm:

  1. Let the number of training cases be N, and the number of variables in the classifier be M.
  2. We are told the number m of input variables to be used to determine the decision at a node of the tree; m should be much less than M.
  3. Choose a training set for this tree by choosing n times with replacement from all N available training cases (i.e. take a bootstrap sample). Use the rest of the cases to estimate the error of the tree, by predicting their classes.
  4. For each node of the tree, randomly choose m variables on which to base the decision at that node. Calculate the best split based on these m variables in the training set.
  5. Each tree is fully grown and not pruned (as may be done in constructing a normal tree classifier).

For prediction a new sample is pushed down the tree. It is assigned the label of the training sample in the terminal node it ends up in. This procedure is iterated over all trees in the ensemble, and the mode vote of all trees is reported as random forest prediction.

Read more about this topic:  Random Forest

Other articles related to "learning algorithm, algorithm, learning":

How Supervised Learning Algorithms Work - Empirical Risk Minimization
... In empirical risk minimization, the supervised learning algorithm seeks the function that minimizes ... Hence, a supervised learning algorithm can be constructed by applying an optimization algorithm to find ... The learning algorithm is able to memorize the training examples without generalizing well ...
Meta Learning (computer Science) - Different Views On Meta Learning
... These are some of the views on (and approaches to) meta learning, please note that there exist many variations on these general approaches Discovering meta-knowledge works by inducing knowledge (e.g ... rules) that expresses how each learning method will perform on different learning problems ... ) in the learning problem, and characteristics of the learning algorithm (type, parameter settings, performance measures...) ...
Overfitting - Machine Learning
... The concept of overfitting is important in machine learning ... Usually a learning algorithm is trained using some set of training examples, i.e ... However, especially in cases where learning was performed too long or where training examples are rare, the learner may adjust to very specific random features of the training data ...
Supervised Learning - Overview
... In order to solve a given problem of supervised learning, one has to perform the following steps Determine the type of training examples ... Determine the structure of the learned function and corresponding learning algorithm ... Run the learning algorithm on the gathered training set ...
ADALINE - Learning Algorithm
... Let us assume is the learning rate (some constant) d is the desired output o is the actual output then the weights are updated as follows ...

Famous quotes containing the word learning:

    Some, for renown, on scraps of learning dote,
    And think they grow immortal as they quote.
    Edward Young (1683–1765)