Read the linear algebra of machine learning in one article (10 cases)

Linear algebra is a sub-discipline of mathematics involving vectors, matrices, and linear transformations.

It is an important foundation of machine learning. From the symbol describing the operation of the algorithm to the implementation of the algorithm in the code, it belongs to the research scope of the discipline.

Although linear algebra is an indispensable part of the machine learning field, the close relationship between the two is often unexplained or can only be explained by abstract concepts such as vector space or specific matrix operations.

After reading this article, you will learn:

How to use linear algebra structures such as tabular datasets and images when working with data.

Linear algebra concepts used in data preparation, such as one-hot encoding and dimensionality reduction.

In-depth use of linear algebraic symbols and methods in sub-fields such as deep learning, natural language processing, and recommendation systems.

let's start.

The 10 machine learning cases are:

Dataset and Data Files datasets and data files

Images and Photographs Images and Photos

One-Hot Encoding one-hot encoding

Linear Regression Linear Regression

RegularizaTIon regularization

Principal Component Analysis Principal Component Analysis

Singular-Value DecomposiTIon singular value decomposition

Latent SemanTIc Analysis Latent Semantic Analysis

Recommender Systems Recommended System

Deep Learning Deep Learning

1. Data sets and data files

In machine learning, you can fit a model on a dataset.

This is a set of numbers in a tabular format where each row represents a set of observations and each column represents a feature of the observation.

For example, the following set of data is part of the Iris flower dataset.

5.1, 3.5, 1.4, 0.2, Iris-setosa

4.9, 3.0, 1.4, 0.2, Iris-setosa

4.7, 3.2, 1.3, 0.2, Iris-setosa

4.6, 3.1, 1.5, 0.2, Iris-setosa

5.0, 3.6, 1.4, 0.2, Iris-setosa

These data are actually a matrix: a key data structure in linear algebra.

Next, the data is decomposed into input data and output data to fit a supervised machine learning model (such as measured values ​​and flower varieties) to obtain matrix (X) and vector (y). Vectors are another key data structure in linear algebra.

Each line has the same length, that is, the number of data per line is the same, so we can say that the data is vectorized. These row data can be provided to the model at one time or in batches, and the model can be pre-configured to obtain fixed-width row data.

2. Images and photos

Maybe you are more accustomed to working with images or photos in computer vision applications.

Each image you use is itself a table structure of fixed width and height, with each cell having 1 pixel value for representing a black and white image or 3 pixel values ​​representing a color image.

Photographs are also a type of linear algebraic matrix.

Image-related operations, such as cropping, scaling, cropping, etc., are described using the symbols and operations of linear algebra.

3. one-hot coding

Sometimes classified data is used in machine learning.

It may be a category label used to resolve a classification problem, or it may be a classification input variable.

It is common to code categorical variables to make them easier to use and learn by some techniques. One-hot encoding is a common categorical variable encoding.

One-hot coding can be understood as: creating a table, using columns to represent each category, and rows representing each instance in the data set. Add a check or "1" value to the categorical value of the given row in the column and add the "0" value to all other columns.

For example, a total of 3 rows of color variables:

Red

Green

Blue

. . .

These variables may be encoded as:

Red, green, blue 1, 0, 0 0, 1, 0 0, 0, 1 . . .

Each line is encoded as a binary vector, a vector that is assigned a value of "0" or "1". This is an example of sparse characterization, a complete subdomain of linear algebra.

4. Linear regression

Linear regression is a statistically traditional method used to describe the relationship between variables.

This method is often used in machine learning to predict the value of a simpler regression problem.

There are many ways to describe and solve linear regression problems by finding a set of coefficients, multiplying them with each input variable and adding the results to get the best output variable prediction.

If you've used machine learning tools or machine learning libraries, the most common way to solve linear regression problems is through least squares optimization, which is solved using linear regression matrix decomposition methods (such as LU decomposition or singular value decomposition).

Even the usual summary methods for linear regression equations use linear algebraic notation:

y = A. b

Where y is the output variable, A is the data set, and b is the model coefficient.

5. Regularization

When applying machine learning, we often seek the simplest and most feasible model to solve the problem.

Simpler models are generally better at generalizing from concrete examples to unseen data.

In many methods involving coefficients, such as regression methods and artificial neural networks, simpler models typically have smaller coefficient values.

One technique commonly used to minimize the coefficient values ​​of a model during data fitting is called regularization. Common implementations include regularized L2 and L1 forms.

These two regularization forms are actually a measure of the size or length of the coefficient vector, which is directly derived from a linear algebra method called vector norm.

6. Principal component analysis

Typically, a dataset has many columns, and the number of columns can be tens, hundreds, thousands, or more.

Modeling data with many characteristics is challenging. Moreover, models built from data containing irrelevant features are generally not as good as models trained with the most relevant data.

It is difficult to know which features of the data are relevant and which are not.

The method of automatically reducing the number of columns in a data set is called dimensionality reduction, and perhaps the most popular method is principal component analysis (PCA).

This method creates a projection of high dimensional data for visualization and models in machine learning.

The core of the PCA method is the matrix decomposition method of linear algebra, which may use feature decomposition. The more general implementation can use singular value decomposition (SVD).

7. Singular value decomposition

Another popular dimension reduction method is the singular value decomposition method, referred to as SVD.

As mentioned above, as the method name suggests, it is a matrix decomposition method derived from the field of linear algebra.

This method has a wide range of applications in linear algebra and can be directly applied to feature selection, visualization, and noise reduction.

In machine learning we will see the following two cases of using SVD.

8. Latent semantic analysis

In the machine learning sub-domain (called natural language processing) for processing text data, a document is usually represented as a large matrix in which words appear.

For example, a column of a matrix can be a known word in a vocabulary, a line can be a sentence, a paragraph, a page, or a document of a text, and a cell in the matrix is ​​labeled as the number or frequency of occurrences of the word.

This is a sparse matrix representation of the text. Matrix decomposition methods (such as singular value decomposition) can be applied to this sparse matrix, which can extract the most relevant part of the matrix representation. Documents processed in this way are easier to compare, query, and serve as the basis for overseeing machine learning models.

This form of data preparation is called Latent Semantic Analysis (LSA), also known as Latent Semantic Indexing (LSI).

9. Recommendation system

The predictive modeling problem involving product recommendations is called the recommendation system, which is a sub-area of ​​machine learning.

For example, recommend books to you based on your purchase history on Amazon and a customer's purchase history similar to yours, or recommend movies or TV shows based on your viewing history on Netflix.

The development of the recommendation system mainly involves linear algebra methods. A simple example would be to use a distance metric such as Euclidean distance or dot product to calculate the similarity between sparse customer behavior vectors.

Matrix decomposition methods such as singular value decomposition are widely used in recommendation systems to extract useful parts of project and user data for query, retrieval, and comparison.

10. Deep learning

Artificial neural network is a nonlinear machine learning algorithm inspired by information processing elements in the brain. Its effectiveness has been verified in a series of problems, the most important of which is predictive modeling.

Deep learning is a recent renaissance of artificial neural networks using the latest methods and faster hardware, which makes it possible to develop and train larger and deeper (more layers) networks on very large data sets. Deep learning methods often achieve the latest results in a series of challenging areas such as machine translation, photo captioning, and speech recognition.

The execution of neural networks involves the multiplication and addition of linear algebraic data structures. If extended to multiple dimensions, the deep learning method can handle vectors, matrices, and even tensors of inputs and coefficients, where the tensor is a two-dimensional matrix.

Linear algebra is the core of the deep learning method. It implements deep learning methods through matrix representation. For example, Google's TensorFlow Python library contains the word "tensor" in its name.

LED Underwater Light Series

LED Underwater light,Full colour underwater luminaires,Waterproof lighting underwater lamp

Kindwin Technology (H.K.) Limited , https://www.ktlleds.com