Supervised Learning
What is supervised learning? (IBM)
- Supervised Machine Learning: Technique using labeled data sets for training ML model to identify underlying patterns/relationships. Datasets consist of labeled sample data points & the correct outputs. The model uses input data to train and adjust the model parameters until it fits the task. The labeled training data gives the ground truth, teaching the model to identify features and data labels relationships.
What is ground truth data?
- Ground truth data: data that is verified against the actual outcomes expected, created usually through annotation/measurement by hand; used for training, validating, and testing models. By definition, this data is confirmed to be true and accurate data, it is the basis the model will use, a gold standard.
How supervised learning works
Through labeled training data the relationships between the inputs and the outputs of the data are understood. Then the model can be trained and applied to real world situations to find correct outputs to unseen data.
- Requires a large dataset to train the model and find correlations.
- Test data is used after training to evaluate model efficacy.
- Cross-validation: process of testing the model using another portion of the dataset.
- gradient descent: family of algorithms, including stochastic gradient descent (SGD), which are used to optimize the model's algorithms. This is done by assessing accuracy with a loss function.
- Loss function: An equation used to measure discrepancies between a model's predictions and the actual values the model is trying to predict.
- Dimensionality reduction: Reduce the number of features in the data to the most crucial ones for predicting data labels, increasing efficiency and maintaining accuracy.
- Features: individual measurable properties/characteristics of a data point; the specific attributes of the data used to describe the phenomenon being observed. Chosen through feature selection The loss function produces a gradient that indicates the direction the model should be adjusted towards, to reduce errors. The optimization is applied throughout training in order to optimize the model.
An example of supervised learning in action
- Captcha is a crowd sourced example of supervised learning, where the model data was annotated by the users of captcha, based on labels prepared by the data scientists, then annotated for each image based on the crowd's classifications.
Types of supervised learning
- Classification: Uses an algorithm to sort data into categories, recognizing entities within a dataset and tries to determine labels/definitions for those entities. This type includes logistic regression. This model is commonly used for neural networks.
- Regression: Used for understanding the relationship between dependent and independent variables. Outputs are continuous values, and the model attempts to predict the target output. E.g. projections for sales revenue, financial planning, etc. Includes linear regression, and other regression methods.
- Ensemble Learning: meta-approach which entails using multiple modles on the same classification/regression task. Then all results are pooled together to find the best overall approach. Each model used is called a weak learner/base model. This approach could be good for mitigating ==bias-variance tradeoff==
Supervised learning algorithms
Gradient descent and other optimization algorithms can be used to train a large number of ML algorithms suitable for supervised learning:
- Naive Bayes: classification algorithm using Bayes' theorem that doesn't use the presence of one feature to predict another feature probability, assuming all predictors have equal impact on the result.
- Linear Regression: Identifies relationship between continuous dependent variables and one or more independent variables. Useful for predicting future outcomes. Represents the relationship as a straight line.
- Nonlinear Regression: Used when outputs cannot be produced from linear inputs (linear regression), and thus represents the values using a nonlinear, curved, line. Useful for complex relationships with large number of parameters.
- Logistic Regression: Used for categorical dependent variables, where there are binary outputs (true, false, or positive/negative, etc). Useful for binary identification problems (such as presence of an atom in spectrum).
- Polynomial Regression: Similar to linear regression, representing relationship modelling on a graph; though the functions express the relationship using polynomial degrees (raising the input features to powers) and thus a linear model can fit a non linear pattern.
- Support vector machine (SVM): Useful for data classification and regression, separating the classes of data points with a decision boundary or hyperplane; The SVM plots the hyperplane that can maximize the distance between the data point groupings.
- K-nearest neighbor: nonparametric algorithm to classify data points based on proximity to other available data. Assumes similar data points can be found near each other when plotted mathematically.
- Random forest: Forest references a collection of uncorrelated decision trees that are merged to increase accuracy and cut back on variance/deviation.
Supervised learning versus other learning methods
Other types of learning include: 1) unsupervised learning 2) semi-supervised learning 3) self supervised learning 4) reinforcement learning
Supervised versus unsupervised learning
Unsupervised learning uses data that hasn't been labeled and has no "ground truth". The model discovers patterns and relationships on its own within the data set. Generative AI models are trained in this fashion.
Pros of unsupervised learning
- exploratory analysis
- no data labeling
- flexible
- scalable
Cons of unsupervised learning
- imprecise
- sensitive
- requires good data
Supervised versus semi-supervised learning
Semi-supervised uses labeled data for a small part of the learning and unlabeled for the rest. Good where labeled data is costly.
Pros of semi-supervised learning
- less reliant on labeling
- hidden pattern discovery
- more flexible
Cons of semi-supervised learning
- sensitive
- bias
- complexity
Supervised vs self-supervised learning
Self supervised use unlabeled data and generate their own supervisory signals (its own labels basically). And ground truth is an emergent property of this process. Loss function uses the emergent labels for assessing performance.
Pros of self-supervised learning
- efficient
- scalable
- low need for labels
- versatile
Cons of self-supervised learning
- intensive
- complex
- unreliable
Supervised versus reinforcement learning
Useful for training autonomous agents such as cars. Does not use labeled data, but teaches the model through trial and error, rather than finding patterns from a dataset.
pros of reinforcement
- applies to complex tasks
- no labels needed
- self correcting
- adaptable
cons of reinforcement learning
- inconsistent results
- needs data from environment ( a lot)
- rewards hacking - models exploit loopholes
- task-specific
Challenges of supervised learning
- Personnel limitations: requires expertise in domain use
- human involvement: Similar to personnel, requires supervision
- time requirements: data must be manually labelled
- inflexibility: struggles labeling data outside of training dataset
- bias: more likely for bias to emerge
- overfit: can become too closely tailored to training dataset.
Children