Vectors

The fundamental building block for linear algebra is the vector, so let's talk about why it's so important.

A vector in a coordinate grid

There are three primary interpretations of vectors:

The physics student perspective
The mathematician's perspective
The computer science perspective

Obviously we're only really concerned with the computer science perspective, but we'll briefly touch on the other two so you won't get caught off guard in a technical discussion.

Interpretations of Vectors

Physics Perspective

The physics student perspective is that vectors are arrows pointing in space. A given vector is defined by its length and the direction it's pointing, but as long as those two facts remain the same, you can move it around and it's still the same vector.

Three perspectives on vectors: physics, CS, and math

The physics perspective: vectors are arrows in space

Below is a visualization showing how 2D vectors differ from 3D vectors:

2D vectors live in a plane; 3D vectors live in space

CS Perspective

The computer science perspective is that vectors are ordered lists of numbers. For example, if you're comparing LLMs and the only features you care about are price (per million tokens) and context window size, you might model each LLM as a pair of numbers:

\begin{bmatrix} \$3.00/\text{M tokens} \\ 128{,}000\;\text{ctx} \end{bmatrix}

Look familiar? That's because "vector" is really just a fancy word for "list." Since there are two entries, it's a two-dimensional vector. One last thing to remember is that order does matter here.

Mathematician's Perspective

The mathematician's perspective is that a vector is anything you can add and scale in a well-behaved way. This matters to us because it's exactly what lets you perform arithmetic operations on lists and have the results still make sense.

Abstract mathematician's view of vectors with addition and scaling

The mathematician's view: vectors are anything you can add and scale

Thinking About Coordinate Systems

While many of you are already familiar with coordinate systems, it's worth walking through them again to prepare for what's to come. In two dimensions, you have a horizontal line called the x-axis and a vertical line called the y-axis.

The x-axis and y-axis

The place where they intersect is the origin, which you should think of as the center of space and the root of all vectors.

The origin: center of space

After choosing an arbitrary distance to represent a length of $1$, you make tick marks on each axis spaced out by this distance.

Tick marks define the unit length

When we want to convey the idea of 2D space as a whole, we extend these tick marks to make grid lines:

The full coordinate grid

Let's establish a specific thought to have in mind when we say the word "vector." Given the geometric focus here, whenever we introduce a new topic involving vectors, think about an arrow inside a coordinate system, with its tail sitting at the origin.

A vector as an arrow rooted at the origin

The coordinates of a vector are a pair of numbers that give instructions for how to get from the tail of the vector (at the origin) to its tip. The first number and second number correspond to the $x$-axis and $y$-axis respectively.

\mathbf{v}=\begin{bmatrix} x \\ y \end{bmatrix} \quad\Longleftrightarrow\quad \text{walk } x \text{ units along } x\text{, then } y \text{ units along } y.

To distinguish vectors from points, the convention is to write this pair of numbers vertically with square brackets around them. Every pair of numbers gives you one and only one vector, and every vector is associated with one and only one pair of numbers.

Vector coordinates showing the walk from origin to tip

Coordinates as walking instructions

In three dimensions, you add a third axis called the $z$-axis, which is perpendicular to both the $x$ and $y$ axes. Each vector is then associated with a list of three numbers:

\mathbf{v}=\begin{bmatrix} x \\ y \\ z \end{bmatrix}

3D coordinate system with a vector triplet

3D vectors use three coordinates

Vector Operations

Every topic in linear algebra centers around two operations: vector addition and scalar multiplication. Luckily, both are relatively straightforward.

Addition

Let's say we have two vectors, one pointing up and a little to the right, and another pointing to the right and a little bit down.

Two vectors ready to be added

To add these two vectors, move the second vector so that its tail sits on the tip of the first one. Then draw a new vector from the tail of the first to where the tip of the second now sits. That new vector is their sum.

Vector addition: tip-to-tail

This works because each vector represents a movement or step with a certain distance and direction. The example below shows two steps of $2$ and $5$ that add up to the total distance traveled of $7$.

Addition on a number line: same idea, one dimension

In general, vector addition means matching up terms and adding them together:

\begin{bmatrix} x_1 \\ y_1 \end{bmatrix} + \begin{bmatrix} x_2 \\ y_2 \end{bmatrix} = \begin{bmatrix} x_1+x_2 \\ y_1+y_2 \end{bmatrix}

Scaling

The other fundamental vector operation is multiplication by a number. If you take the number $2$ and multiply it by a given vector, you stretch that vector so that it's twice as long as it was before.

Scaling by $2$: the vector doubles in length

If you multiply a vector by $\frac{1}{3}$, you squish it down so that it's one-third of its original length. If you multiply by a negative number like $-1.5$, the vector gets flipped around, then stretched out by a factor of $1.5$.

Scaling by $\frac{1}{3}$: the vector shrinks

This process of stretching, squishing, and sometimes reversing direction is called "scaling." The number you are multiplying the vector by is called the "scalar."

Numerically speaking, scaling multiplies each component:

c\,\mathbf{v} = c\begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} cx \\ cy \end{bmatrix}

\frac{1}{3}\begin{bmatrix} 12 \\ 9 \end{bmatrix} = \begin{bmatrix} 4 \\ 3 \end{bmatrix}

This will come up later in more detail when we discuss dot products.

Conclusion

Whether you think of vectors as arrows or lists of numbers, the main thing to remember is that they give us a structured way to represent and transform data. The two main ways we transform this data are vector addition and scalar multiplication.