Slides: https://eo4geocourses.github.io/UT-ITC_Satellite_Data_Classification_Random_Forests
Created by:Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente
#Abstract: Random Forests (RF) is one of the most popular machine learning classifiers due to its capability to obtain good classification results with a relatively reduced number of training samples and because it relies on a reduced number of user-defined parameters. In addition, RF can easily address the Hughes phenomenon due to the two randomness dimensions: first, it randomly selects samples to create a user-defined number of decision trees and, second, a randomly user-defined number of variables are used for splitting the decision tree nodes. In this lecture, we will be introducing the concept of ensemble classifier by emphasizing the main methods used to build an ensemble: bootstrapping and bagging. We will explain how RF classifier is built and how the final decision is taken. The concept of out-of-the-bag will be explained in detail especially in connection with its role for internal evaluation of the RF accuracy. We will also explain how RF calculates the variable importance, a measure that can be used to discard the less relevant input variables. In the end, special attention is given to the proximity measure used to identify either outliers or subclasses in the training samples.
Table of contents: (1) Concept of ensemble classifiers; (2) Random Forests; (3) Random Forest hyper-parameters; (4) Fundamental differences between tree growth on decision trees and Random Forest; (5) Accuracy assessments in Random Forests
Learning outcomes: LO1: Define the concept of ‘ensemble classifier’ LO2: Explain the concepts of bagging and bootstrapping LO3: Describe how variable importance is calculated in Random Forest classifier LO4: Explain the role of proximity measure for removing outliers from the training sample set LO5: List the main advantages and disadvantages of the RF classifier