Personal tools

Linear Algebra

(University of Michigan at Ann Arbor)


- Overview

Linear algebra is the branch of mathematics concerning linear equations. Linear algebra is central to almost all areas of mathematics. Linear algebra is the study of linear sets of equations and their transformation properties. Linear algebra allows the analysis of rotations in space, least squares fitting, solution of coupled differential equations, determination of a circle passing through three given points, as well as many other problems in mathematics, physics, and engineering. Confusingly, linear algebra is not actually an algebra in the technical sense of the word "algebra" (i.e., a vector space  over a field , and so on).

Linear algebra, mathematical discipline that deals with vectors and matrices and, more generally, with vector spaces and linear transformations. Unlike other parts of mathematics that are frequently invigorated by new ideas and unsolved problems, linear algebra is very well understood. Its value lies in its many applications, from mathematical physics to modern algebra and coding theory. 

Linear algebra is a field of mathematics that is universally agreed to be a prerequisite to a deeper understanding of machine learning. Although linear algebra is a large field with many esoteric theories and findings, the nuts and bolts tools and notations taken from the field are practical for machine learning practitioners. Linear algebra is the mathematics of data. It has had a marked impact on the field of statistics. Linear algebra underlies many practical mathematical tools, such as Fourier series and computer graphics.


- Linear Algebra for Machine Learning

Linear algebra (LA) is a pillar of machine learning. You cannot develop a deep understanding and application of machine learning without it. It is very important to have sufficient knowledge of a few linear algebra concepts if you are looking to understand the underlying concepts behind machine learning. If you don’t know the math behind these advanced machine learning algorithms, you can’t wish to develop a mastery over them.

Areas of mathematics such as statistics and calculus require prior knowledge of linear algebra, which will help you understand ML in depth. Many ML experts may be of the opinion that linear algebra  helps to some extent, but it definitely improves one’s math skills and intuition in ML. 

Linear Algebra is a branch of mathematics that deals with linear equations and linear functions which are represented through matrices and vectors. In simpler words, LA helps you understand geometric terms such as planes, in higher dimensions, and perform mathematical operations on them. By definition, algebra deals primarily with scalars (one-dimensional entities), but LA has vectors and matrices (entities which possess two or more dimensional components) to deal with linear equations and functions. LA can also be called as the extended version of algebra.

Here are a few concepts of linear algebra that you need to learn about for knowing how machine learning works. They are: Vectors and Matrix, Symmetric Matrix, Eigenvalues and Eigenvector, Principal Component Analysis (PCA), One-Hot Encoding, Linear Regression, Regularization, Singular-Value Decomposition, Latent Semantic Analysis, Recommender Systems. Deep Learning.


[Hallstatt, Austria - Civil Engineering Discoveries]

- Linear Algebra Topics

The linear algebra prerequisite should include the following topics:

  • Mathematical operations with matrices (addition, multiplication)
  • Matrix inverses and determinants
  • Solving systems of equations with matrices
  • Euclidean vector spaces
  • Eigenvalues and eigenvectors
  • Orthogonal matrices
  • Positive definite matrices
  • Linear transformations
  • Projections
  • Linear dependence and independence
  • Singular value decomposition


- Linear Algebra for Data Science and Machine Learning

Linear algebra is the bedrock of data science. Knowing linear algebra boosts your ability to understand data science algorithms.

Linear Algebra is a continuous form of mathematics and is applied throughout science and engineering because it allows you to model natural phenomena and to compute them efficiently. Because it is a form of continuous and not discrete mathematics, a lot of computer scientists don’t have a lot of experience with it. Linear Algebra is also central to almost all areas of mathematics like geometry and functional analysis. Its concepts are a crucial prerequisite for understanding the theory behind Machine Learning, especially if you are working with Deep Learning Algorithms. You don’t need to understand Linear Algebra before getting started with Machine Learning, but at some point, you may want to gain a better understanding of how the different Machine Learning algorithms really work under the hood. This will help you to make better decisions during a Machine Learning system’s development. So if you really want to be a professional in this field, you will have to master the parts of Linear Algebra that are important for Machine Learning. 

In Linear Algebra, data is represented by linear equations, which are presented in the form of matrices and vectors. Therefore, you are mostly dealing with matrices and vectors rather than with scalars. When you have the right libraries, like Numpy, at your disposal, you can compute complex matrix multiplication very easily with just a few lines of code.  


- Examples of Linear Algebra in Machine Learning

Here are 10 examples of linear algebra in machine learning:  

  • Dataset and Data Files
  • Images and Photographs
  • One-Hot Encoding
  • Linear Regression
  • Regularization
  • Principal Component Analysis
  • Singular-Value Decomposition
  • Latent Semantic Analysis
  • Recommender Systems
  • Deep Learning



[More to come ...]

Document Actions