Interactive Linear Algebra

1. Why Vectors are the Language of AI

In advanced AI and data science, you must understand the "why." So, why vectors? Because they are the ultimate translators, converting complex, real-world concepts into a structured format that computers can process. This is vectorization.

By treating data as points in a high-dimensional space, we can measure distances, find relationships, and make intelligent predictions. Mastering vectors is the first step to mastering machine learning.

2. Deconstructing a Vector: The Role of Basis

A vector like $$\vec{v} = \begin{bmatrix} 3 \\ 4 \end{bmatrix}$$ is more than a list of numbers. It's a recipe. It's a linear combination of the simplest possible vectors: the standard basis vectors.

$$\hat{\imath} = \begin{bmatrix} 1 \\ 0 \end{bmatrix} \text{ (one unit step along the x-axis)}$$

$$\hat{\jmath} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} \text{ (one unit step along the y-axis)}$$

So, our vector $$\vec{v}$$ is simply: "Take 3 steps in the $$\hat{\imath}$$ direction and 4 steps in the $$\hat{\jmath}$$ direction."

$$\vec{v} = 3\hat{\imath} + 4\hat{\jmath} = 3\begin{bmatrix} 1 \\ 0 \end{bmatrix} + 4\begin{bmatrix} 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 3 \\ 0 \end{bmatrix} + \begin{bmatrix} 0 \\ 4 \end{bmatrix} = \begin{bmatrix} 3 \\ 4 \end{bmatrix}$$

The Core Idea: The components of a vector are just scalars that stretch the basis vectors. This is the key to understanding matrices, which are machines for transforming this entire coordinate system.

3. Unit Vectors: Capturing Pure Direction

Often in AI, we only care about the direction of a vector, not its length. For this, we use a unit vector—a vector with a magnitude of exactly 1. The basis vectors $$\hat{\imath}$$ and $$\hat{\jmath}$$ are unit vectors!

To create a unit vector from any vector $$\vec{v}$$, you divide it by its own magnitude. This is called normalization.

$$\hat{v} = \frac{\vec{v}}{||\vec{v}||}$$

For our vector $$\vec{v} = \begin{bmatrix} 3 \\ 4 \end{bmatrix}$$, we know $$||\vec{v}|| = 5$$. So the unit vector is:

$$\hat{v} = \frac{1}{5}\begin{bmatrix} 3 \\ 4 \end{bmatrix} = \begin{bmatrix} 3/5 \\ 4/5 \end{bmatrix} = \begin{bmatrix} 0.6 \\ 0.8 \end{bmatrix}$$

This new vector $$\hat{v}$$ points in the exact same direction as $$\vec{v}$$, but its length is guaranteed to be 1. This is crucial for comparing directions without being biased by magnitude.

4. The Dot Product: The Geometry of Similarity

The dot product is one of the most powerful tools in your arsenal. It takes two vectors and returns a single number (a scalar) that tells us how "aligned" they are.

The Calculation

$$\vec{a} \cdot \vec{b} = \begin{bmatrix} a_1 \\ a_2 \end{bmatrix} \cdot \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} = a_1b_1 + a_2b_2$$

The Geometric Intuition

The dot product is also defined as: $$\vec{a} \cdot \vec{b} = ||\vec{a}|| \cdot ||\vec{b}|| \cdot \cos(\theta)$$, where $$\theta$$ is the angle between them. This reveals its true meaning:

• If $$\vec{a} \cdot \vec{b} > 0$$ ($$\theta < 90^\circ$$): They point in a similar direction.
• If $$\vec{a} \cdot \vec{b} = 0$$ ($$\theta = 90^\circ$$): They are orthogonal (perpendicular). Their similarity is zero.
• If $$\vec{a} \cdot \vec{b} < 0$$ ($$\theta > 90^\circ$$): They point in opposing directions.

In AI, "cosine similarity" (the dot product of two unit vectors) is a standard way to measure how similar two pieces of data (like documents or user profiles) are.

5. Test Your Conceptual Understanding

These problems test your conceptual understanding. Strive to explain the "why" in your answers.