Linear algebra, translated

How to study this

Picture first, numbers second, symbols last.

Linear algebra is rarely hard because of the arithmetic. It feels hard because textbooks start with the symbols, and symbols mean nothing until you have a picture to attach them to. These notes give you the picture first.

• Start in 2D. Anything true in ℝ³ or ℝⁿ can first be drawn on paper in ℝ². Once it makes sense with two numbers, three or more is "the same thing, longer list".
• Say it in words. After any definition, explain it out loud as if to a friend who never took math. If you can't, you've memorised it rather than understood it.
• Predict before computing. "Will this dot product be positive or negative?" "Will this matrix have an inverse?" Guessing first builds intuition quickly.
• Check what kind of answer is wanted. A number? A vector? A matrix? Yes or no? Many lost marks come from giving the wrong type of answer.
• Check your work. Almost every computation in this course can be checked in under a minute. See Self-check habits at the end.

Martian to English

Keep this handy. Most confusion is vocabulary.

Vectors

An arrow, or a list of numbers. It's the same thing.

The idea

A vector like (3, 1) is a set of directions: "go 3 right and 1 up". It doesn't care where you start, only how far and which way. You can also read it as a list of data: 3 apples, 1 banana. Both readings are correct, and switching between them is half of this course.

• Adding vectors means doing one move, then the other. Put them tip to tail.
• Scaling (multiplying by a number) stretches or shrinks the arrow. A negative number flips it around.
• Length is just Pythagoras: ‖(3, 4)‖ = √(9 + 16) = 5. More in Norm.
• ℝⁿ just means "lists of n numbers". (2, 0, −1, 5) lives in ℝ⁴. You can't draw it, but every rule works exactly the same: entry by entry.

u + v = (3+1, 1+2) = (4, 3)
2u    = (6, 2)                same direction, twice as long
u − v = (3−1, 1−2) = (2, −1)
‖u‖   = √(3² + 1²) = √10 ≈ 3.16

Linear combinations

A recipe: some of this vector, plus some of that one.

The idea

A linear combination of v₁ and v₂ is anything of the form c₁v₁ + c₂v₂, where c₁ and c₂ are ordinary numbers (the "weights"). You scale each ingredient, then add.

Practical hook: a smoothie. If a banana gives (100 cal, 1 g protein) and a scoop of yogurt gives (60 cal, 5 g protein), then 2 bananas and 3 scoops give

2·(100, 1) + 3·(60, 5) = (200 + 180, 2 + 15) = (380 cal, 17 g protein)

The interesting question runs backwards: "I want exactly this nutrition. How much of each ingredient do I need?" That's asking for the weights, and it's where systems of equations come from.

a(1, 1) + b(2, 1) = (7, 4)

Match each slot:
  a + 2b = 7
  a +  b = 4
Subtract: b = 3, then a = 1

Check: 1·(1, 1) + 3·(2, 1) = (1 + 6, 1 + 3) = (7, 4)   ✓

The norm (length)

Pythagoras, extended to as many slots as you like.

The idea

‖v‖ = √(v₁² + v₂² + … + vₙ²)

• In 2D it's the hypotenuse. In 3D it's the diagonal of a box. In ℝⁿ it's the same formula with more terms.
• Distance between two points p and q is ‖p − q‖. Subtract, then take the length.
• Unit vector: v / ‖v‖. It keeps the direction and sets the length to 1. Think "just the compass heading, no distance".
• Practical hook: comparing two customers' shopping lists, or two songs' features, by the distance between their vectors. Small distance means similar. Recommendation apps do exactly this in ℝ¹⁰⁰⁰.

Rules worth knowing

v = (1, 2, 2)
‖v‖ = √(1 + 4 + 4) = √9 = 3
unit vector = (1/3, 2/3, 2/3)
  check: √(1/9 + 4/9 + 4/9) = √1 = 1   ✓

Distance from (1, 2, 3) to (4, 6, 3):
  difference = (3, 4, 0)
  ‖(3, 4, 0)‖ = √(9 + 16 + 0) = 5

In ℝ⁴: ‖(1, 1, 1, 1)‖ = √4 = 2

The dot product

One number that says how much two arrows agree.

The idea

Multiply matching entries and add them up: (a, b)·(c, d) = ac + bd. The answer is a plain number, not a vector.

Practical hook: you already use dot products at the till. If you buy 2 coffees and 3 muffins at $4 and $3:

(2, 3)·(4, 3) = 2×4 + 3×3 = 17     your bill is $17

Geometric meaning: u·v = ‖u‖‖v‖ cos θ, where θ is the angle between them. So the sign tells you the direction relationship at a glance:

Physics example you may know: work = force · displacement. Only the part of your push that goes along the direction of motion counts. That "part along a direction" idea becomes projections.

It works the same in ℝⁿ: (1, 0, 2, 3)·(2, 5, 1, −1) = 2 + 0 + 2 − 3 = 1.

(1, 2)·(4, −2) = 4 − 4 = 0          → perpendicular

Angle between (1, 1) and (1, 0):
  dot = 1,  ‖(1,1)‖ = √2,  ‖(1,0)‖ = 1
  cos θ = 1/√2   →   θ = 45°        ✓ matches the picture

Vector projections

The shadow one arrow casts on another.

The idea

Shine a light straight down onto the line of v. The shadow u casts on it is the projection of u onto v. It answers: "how much of u points in v's direction?"

Every vector splits into two pieces: the part along v (the projection) and the part perpendicular to v (what's left over).

Practical hook: a box on a ramp. Gravity pulls straight down, but you split it into the part along the ramp (makes the box slide) and the part into the ramp (pressed against the surface). That split is exactly a projection.

Formulas

vector projection:   proj_v(u) = (u·v / v·v) · v      ← a vector
scalar projection:   comp_v(u) = u·v / ‖v‖             ← a number: signed shadow length
perpendicular part:  u − proj_v(u)

Reading the vector formula: u·v / v·v is just a number, "how many v's long is the shadow". Then you multiply v by it.

u = (3, 4),  v = (2, 1)

u·v = 6 + 4 = 10
v·v = 4 + 1 = 5
proj_v(u) = (10/5)·(2, 1) = (4, 2)
perp part = (3, 4) − (4, 2) = (−1, 2)

Check: perp part · v = −2 + 2 = 0   ✓ it really is perpendicular
Check: (4, 2) + (−1, 2) = (3, 4)   ✓ pieces add back to u

The cross product

Give it two arrows in 3D, get back an arrow perpendicular to both.

The idea

• u × v is a vector that sticks straight out of the flat surface containing u and v.
• Its length equals the area of the parallelogram u and v make.
• Its direction follows the right-hand rule: fingers along u, curl toward v, thumb points to u × v.
• Real-world: torque. Push a wrench handle, and the bolt turns along the axis perpendicular to both the handle and your push.

The formula, written so it's memorisable:

(a₁, a₂, a₃) × (b₁, b₂, b₃) = ( a₂b₃ − a₃b₂ ,  a₃b₁ − a₁b₃ ,  a₁b₂ − a₂b₁ )
Pattern: for each slot, cover that slot and do a "criss-cross" on the other two.
The middle one is the one people get backwards.

(1, 2, 3) × (4, 5, 6)
  x: 2·6 − 3·5 = −3
  y: 3·4 − 1·6 =  6
  z: 1·5 − 2·4 = −3
  answer: (−3, 6, −3)

Check by dotting with both originals:
  (−3, 6, −3)·(1, 2, 3) = −3 + 12 − 9  = 0   ✓
  (−3, 6, −3)·(4, 5, 6) = −12 + 30 − 18 = 0  ✓

Lines and planes in ℝⁿ

A line is "start here, walk one way". A plane is "start here, walk two ways".

Lines

Think GPS: a starting point p, a direction d, and time t. Where are you at time t?

x = p + t·d
e.g. start at (1, 0, 2), walk in direction (2, 1, 3):
(x, y, z) = (1, 0, 2) + t(2, 1, 3)
t = 0 → (1, 0, 2),  t = 1 → (3, 1, 5),  t = −1 → (−1, −1, −1)

This formula doesn't care about dimension. A line in ℝ⁴ is just (1, 0, 2, −1) + t(1, 1, 0, 3): one starting point, one direction, one parameter.

Planes, version 1: parametric (works in any ℝⁿ)

A plane needs a starting point and two directions that don't point the same way. Two parameters, because a plane has two degrees of freedom.

x = p + s·u + t·v
Every point on the plane is "start at p, go s steps along u and t steps along v".
That's p plus a linear combination of u and v.

Planes, version 2: one equation (ℝ³ only)

In 3D a plane can also be pinned down by one point on it and one normal vector n that sticks straight out of it, like a pencil standing upright on a table. Every arrow lying in the table is perpendicular to the pencil, so its dot product with n is zero.

n · (x − p) = 0      →      ax + by + cz = d
The coefficients (a, b, c) ARE the normal vector. Read it straight off.

2(x − 1) − 1(y − 2) + 1(z − 3) = 0
2x − 2 − y + 2 + z − 3 = 0
2x − y + z = 3

Check the point: 2(1) − 2 + 3 = 3   ✓

normal = u × v = (1·1 − 0·1,  0·0 − 1·1,  1·1 − 1·0) = (1, −1, 1)
1(x − 1) − 1(y − 0) + 1(z − 0) = 0
x − y + z = 1

Check p:     1 − 0 + 0 = 1   ✓
Check p + u: (2, 1, 0) → 2 − 1 + 0 = 1   ✓

x = 1 + 2t,  y = t,  z = 2 + 3t
2(1 + 2t) − t + (2 + 3t) = 3
4 + 6t = 3   →   t = −1/6

point = (1 − 1/3, −1/6, 2 − 1/2) = (2/3, −1/6, 3/2)
Check: 4/3 + 1/6 + 3/2 = 8/6 + 1/6 + 9/6 = 3   ✓

Distance from a point to a plane

This is a projection onto the normal in disguise:

distance = |a·x₀ + b·y₀ + c·z₀ − d| / ‖(a, b, c)‖
e.g. origin to 2x − y + z = 3:  |0 − 3| / √6 = 3/√6 ≈ 1.22

Systems of linear equations

Several conditions that all have to be true at once.

The idea

Each equation in two unknowns is a line. Solving the system means finding where all the lines meet. In 3 unknowns each equation is a plane. There are only three possibilities, and it's worth being able to draw all three:

"Linear" means each variable appears only multiplied by a number and added: no x², no xy, no sin x, no 1/x. That's what keeps the pictures flat (lines and planes, never curves).

a = adults, c = children
   a +  c = 10      ← count of tickets
 12a + 8c = 100     ← money

From the first: c = 10 − a
12a + 8(10 − a) = 100  →  4a = 20  →  a = 5, c = 5

Check: 5 + 5 = 10 ✓   60 + 40 = 100   ✓

Augmented matrices, REF and RREF

High-school elimination, but with tidy bookkeeping.

Augmented matrix: the letters stripped out

The variable names never change during elimination, so stop writing them. Keep only the numbers, one row per equation, one column per variable, and a bar where the = sign was:

 x + 2y −  z = 3          [ 1  2  −1 | 3 ]
2x + 4y +  z = 9    →     [ 2  4   1 | 9 ]
A missing variable gets a 0 in its column. Don't skip it.

The three legal moves

Only ever do these, because none of them change the answer:

1. Swap two rows. (Listing equations in a different order.)
2. Multiply a row by a non-zero number. (Doubling both sides.)
3. Add a multiple of one row to another. (Classic elimination.)

REF vs RREF, in plain words

The same system, at both stages:
REF                      RREF
[ 1  1  1 | 9 ]          [ 1  0  0 | 4 ]
[ 0  2  1 | 7 ]          [ 0  1  0 | 2 ]
[ 0  0  3 | 9 ]          [ 0  0  1 | 3 ]   → x = 4, y = 2, z = 3

Strategy: go left to right, making zeros below each pivot (that gets REF). Then go right to left, scaling pivots to 1 and making zeros above them (that gets RREF).

 x + y + z = 6
2x + y − z = 1
 x − y + z = 2

[ 1   1   1 |  6 ]
[ 2   1  −1 |  1 ]
[ 1  −1   1 |  2 ]

R2 → R2 − 2R1,  R3 → R3 − R1
[ 1   1   1 |   6 ]
[ 0  −1  −3 | −11 ]
[ 0  −2   0 |  −4 ]

Row 3 says −2y = −4, so y = 2
Row 2: −2 − 3z = −11, so z = 3
Row 1: x + 2 + 3 = 6, so x = 1

Check in eq. 2: 2(1) + 2 − 3 = 1   ✓

[ 1  2  −1 | 3 ]
[ 2  4   1 | 9 ]

R2 → R2 − 2R1          [ 1  2  −1 | 3 ]
                       [ 0  0   3 | 3 ]     ← REF. Pivots in columns x and z.
R2 → R2 / 3            [ 1  2  −1 | 3 ]
                       [ 0  0   1 | 1 ]
R1 → R1 + R2           [ 1  2   0 | 4 ]
                       [ 0  0   1 | 1 ]     ← RREF

Column y has no pivot, so y is free. Let y = t.
  z = 1
  x = 4 − 2t
Solution: (x, y, z) = (4, 0, 1) + t(−2, 1, 0)

Check t = 0: 4 + 0 − 1 = 3 ✓   8 + 0 + 1 = 9   ✓

Reading the final grid

Matrix–vector and matrix–matrix multiplication

A matrix is a machine: vector goes in, vector comes out.

The most useful idea: the column view

Multiplying a matrix by a vector means "take this much of column 1, plus this much of column 2". A bakery example makes it concrete:

              cake  cookie
eggs        [  3      1   ]   [ 2 ]  ← 2 cakes
flour (cups)[  2      3   ] × [ 5 ]  ← 5 batches of cookies

= 2 × (3, 2) + 5 × (1, 3) = (6 + 5, 4 + 15) = (11 eggs, 19 cups)

Each column is a recipe. The vector is how many of each you're making. The answer is your shopping list. That's a linear combination of the columns.

Two ways to compute Ax, same answer

A = [  1  0  2 ]     x = (2, 1, 1)
    [ −1  3  1 ]

Row view (dot each row with x):          ← fastest by hand
  row 1: 1·2 + 0·1 + 2·1 = 4
  row 2: −1·2 + 3·1 + 1·1 = 2

Column view (weights × columns):         ← gives the meaning
  2·(1, −1) + 1·(0, 3) + 1·(2, 1) = (4, 2)   ✓ same

Size check: A is 2×3, x has 3 entries, the answer has 2 entries. The number of columns of A must equal the length of x.

[ 1  1 ] [ a ]   [ 10  ]          a +  c = 10
[ 12 8 ] [ c ] = [ 100 ]   means  12a + 8c = 100

Matrix × matrix

• Size rule: (m×n)(n×p) = (m×p). The inner numbers must match; the outer numbers give the answer's size.
• Entry in row i, column j of AB = (row i of A) · (column j of B).
• Order matters. AB is usually not BA. Socks-then-shoes is not shoes-then-socks.

Matrix inverse

The undo button.

The idea

If A is a machine, A⁻¹ is the machine that reverses it: A⁻¹A = I. You can't divide by a matrix, so to solve Ax = b you multiply both sides by the undo button: x = A⁻¹b.

Some matrices have no inverse. If A squashes the whole plane onto a line, lots of different inputs land on the same output, and there's no way to know which one you started from. You can't unscramble an egg.

The 2×2 shortcut

[ a  b ]⁻¹      1     [  d  −b ]
[ c  d ]    = ─────── [ −c   a ]
               ad − bc
Swap a and d, flip the signs of b and c, divide by ad − bc.
If ad − bc = 0, there is no inverse.

For bigger matrices: write [ A | I ], row reduce until the left side is I, and the right side becomes A⁻¹.

A = [ 2  1 ]      ad − bc = 6 − 5 = 1
    [ 5  3 ]

A⁻¹ = [  3  −1 ]
      [ −5   2 ]

Check A·A⁻¹:
  row 1: 2·3 + 1·(−5) = 1,   2·(−1) + 1·2 = 0
  row 2: 5·3 + 3·(−5) = 0,   5·(−1) + 3·2 = 1   ✓ identity

2x + y = 4          A = [ 2  1 ],  b = (4, 11)
5x + 3y = 11            [ 5  3 ]

x = A⁻¹b = [  3  −1 ] [ 4  ]  =  (12 − 11, −20 + 22) = (1, 2)
           [ −5   2 ] [ 11 ]

Check: 2 + 2 = 4 ✓   5 + 6 = 11   ✓

A = [ 1  2 ]
    [ 3  7 ]

[ 1  2 | 1  0 ]
[ 3  7 | 0  1 ]

R2 → R2 − 3R1     [ 1  2 |  1  0 ]
                  [ 0  1 | −3  1 ]
R1 → R1 − 2R2     [ 1  0 |  7 −2 ]
                  [ 0  1 | −3  1 ]

A⁻¹ = [  7  −2 ]
      [ −3   1 ]      ✓ matches the 2×2 shortcut (det = 7 − 6 = 1)

Rules worth knowing

• (A⁻¹)⁻¹ = A. Undoing the undo gets you back where you started.
• (AB)⁻¹ = B⁻¹A⁻¹. Order reverses.
• (Aᵀ)⁻¹ = (A⁻¹)ᵀ.
• Only square matrices can have inverses.

Determinants

How much the matrix stretches area (or volume).

The idea

Take the 1×1 unit square and push it through the matrix. It becomes a parallelogram. The determinant is that parallelogram's area.

• det = 6: areas get 6 times bigger.
• det negative: space got flipped over, like a mirror image.
• det = 0: everything got squashed flat. That's exactly why there's no inverse.

How to compute

• 2×2: ad − bc.
• 3×3: cofactor expansion. Tip: expand along whichever row or column has the most zeros; it saves work.
• Or row reduce to a triangle and multiply the diagonal, tracking what the row moves did:

A = [ 2  0  1 ]
    [ 1  3  2 ]
    [ 1  1  1 ]

Expand along row 1 (it has a zero):
det = 2·(3·1 − 2·1) − 0·(…) + 1·(1·1 − 3·1)
    = 2·1 + 1·(−2)
    = 0

Cramer's rule

Solve for one variable at a time using determinants.

The recipe

For a square system Ax = b with det A ≠ 0:

            det(Aᵢ)
  xᵢ  =  ───────────
            det(A)

Aᵢ = A with column i replaced by b.

In words: to get a variable, swap the answer column into that variable's slot, take the determinant, divide by the original determinant.

2x + y = 4          A = [ 2  1 ],  b = (4, 11)
5x + 3y = 11            [ 5  3 ]          det A = 6 − 5 = 1

x:  swap b into column 1   [ 4   1 ]   det = 12 − 11 = 1   →  x = 1/1 = 1
                           [ 11  3 ]
y:  swap b into column 2   [ 2   4 ]   det = 22 − 20 = 2   →  y = 2/1 = 2
                           [ 5  11 ]

✓ Same answer as the inverse method in section 11.

 x + y + z = 6        A = [ 1   1   1 ]    b = (6, 1, 2)
2x + y − z = 1            [ 2   1  −1 ]
 x − y + z = 2            [ 1  −1   1 ]

det A  = 1(1 − 1) − 1(2 + 1) + 1(−2 − 1) = 0 − 3 − 3 = −6

A₁ = [ 6  1  1 ; 1  1 −1 ; 2 −1  1 ]    det = −6    →  x = −6/−6  = 1
A₂ = [ 1  6  1 ; 2  1 −1 ; 1  2  1 ]    det = −12   →  y = −12/−6 = 2
A₃ = [ 1  1  6 ; 2  1  1 ; 1 −1  2 ]    det = −18   →  z = −18/−6 = 3

✓ Same answer as the row reduction in section 9.

When to use it

Linear transformations

Moves that keep grid lines straight, parallel and evenly spaced, and keep the origin still.

The single most useful trick

The columns of the matrix are where the basic arrows land. Column 1 is where (1, 0) goes; column 2 is where (0, 1) goes. To build any transformation matrix, just ask "where do those two arrows end up?"

Real-world: every rotation, zoom and flip in a video game or photo editor is a matrix doing exactly this.

Is it linear?

Officially: T(u + v) = T(u) + T(v) and T(cu) = cT(u). Quick screening tests:

• Does 0 go to 0? If not, it's not linear. So "shift everything right by 1" is out.
• Any squares, products of variables, sin(x), constants added? Not linear.
• If each output is just "numbers × variables, added up", it's linear, and you can read off the matrix.

Span, independence, basis and subspaces

Often the hardest part of the course. Go slowly and picture everything in 2D and 3D.

Span: where can you get to?

Imagine you can only move along certain arrows, as far as you like, forwards or backwards. The span is every place you can reach.

• One non-zero arrow in 2D: you can reach a line.
• Two arrows pointing different ways in 2D: you can reach the whole plane.
• Two arrows pointing the same way (one is a multiple of the other): still only a line. The second one added nothing.

Independence: is anything redundant?

A set of vectors is independent if none of them can be built from the others. Paint analogy: red, blue and yellow are independent. Add purple and it's redundant, because you could already mix it from red and blue.

How to test: put the vectors in as the columns of a matrix and row reduce. Pivot in every column: independent. Any column without a pivot: dependent. For n vectors in ℝⁿ you can also just check det ≠ 0.

Basis and dimension

A basis is a set of vectors that is independent (no waste) and spans the whole space (reaches everywhere). It's a coordinate system. The number of vectors in any basis is the dimension. The usual one for ℝ² is (1, 0), (0, 1), but (1, 0), (1, 1) works just as well.

Subspaces

A subspace is a flat thing through the origin: a line through 0, a plane through 0, just {0}, or the whole space. Three-part test:

1. Does it contain the zero vector? (Fastest way to rule things out.)
2. Add two things in it. Is the result still in it?
3. Scale something in it. Is the result still in it?

Are (1, 2, 3), (4, 5, 6), (7, 8, 9) independent?

Row reduce with them as columns and you'll get only 2 pivots. Dependent.
Specifically: (7, 8, 9) = 2·(4, 5, 6) − (1, 2, 3)
             = (8 − 1, 10 − 2, 12 − 3)  ✓

So they span only a plane in 3D, not all of 3D space.

Eigenvalues and eigenvectors

The directions a matrix doesn't turn, only stretches.

The idea

Most vectors get knocked off their line when a matrix acts on them. A few special ones stay on their own line and just get longer, shorter or flipped. Those are eigenvectors, and the stretch factor is the eigenvalue λ:

A v = λ v        "A does to v what a plain number would"

Picture spinning a globe: every point moves except those on the axis. The axis is an eigenvector with eigenvalue 1. Real uses: Google's original PageRank, vibration modes of bridges, population models.

Recipe

1. Solve det(A − λI) = 0 for λ. (Why: we need (A − λI)v = 0 to have a non-zero solution, so A − λI must squash something, so its det is 0.)
2. For each λ, solve (A − λI)v = 0 by row reduction to find v.

A = [ 2  1 ]
    [ 1  2 ]

det(A − λI) = (2 − λ)² − 1 = 0
2 − λ = ±1   →   λ = 1 or λ = 3

λ = 3:  A − 3I = [ −1  1 ; 1  −1 ]  →  v = (1, 1)
        check: A(1, 1) = (3, 3) = 3·(1, 1)     ✓
λ = 1:  A − I  = [ 1  1 ; 1  1 ]    →  v = (1, −1)
        check: A(1, −1) = (1, −1) = 1·(1, −1)  ✓

How it all connects

Most of the course is one question asked in different ways.

For a square matrix A, every statement below is true together, or they are all false together. Try picking one example matrix and checking a few of them to see they agree.

Self-check habits

Thirty seconds of checking is worth more marks than any trick.