Dot Products

Often we want to know how close two vectors are by calculating the angle $\theta$ between them. For this purpose, we define the dot product:

$$ a \cdot b = \left| a \right| \left| b \right| \cos \theta $$

But this is unhelpful to work with, so we can define it differently, and this also helps us use it to retrieve $\theta$ too:

$$ a \cdot b = a_1b_1 + a_2b_2 + \dots + a_nb_n $$

The equivalence of the definitions allows us to manipulate it. Since $\cos \theta \leq 1$:

$$ a_1b_1 + a_2b_2 + \dots + a_nb_n = \left| a \right| \left| b \right| \cos \theta \leq \left| a \right| \left| b \right| $$

Squaring both sides yields:

$$ (a_1b_1 + a_2b_2 + \dots + a_nb_n)^2 \leq (a_1^2 + \dots + a_n^2)(b_1^2 + \dots + b_n^2) $$

This is called the Cauchy-Schwarz Inequality and is considered one of the most important inequalities in all of mathematics.

Inner Products

Dot products work very well for real-number vectors, but for general vectors in vector spaces, we need a more generalised product (what would a dot product of polynomials even look like, anyway?). We call this generalised product the inner product of a real vector space.

Inner products are denoted $\langle u, v \rangle$ and output a scalar. They have the following properties, which are motivated by the dot product:

• Distributivity
  • $\langle u+v, w \rangle = \langle u, w \rangle + \langle v, w \rangle$
• Linearity
  • $\alpha\langle u, v \rangle = \langle \alpha u, v \rangle$
• Commutativity
  • $\langle u, v \rangle = \langle v, u \rangle$
• Positive-definiteness
  • $\langle v, v \rangle \geq 0$, and $\langle v, v \rangle = 0$ if and only if $v = 0$

Orthogonality

Vectors are orthogonal if their dot product is $0$. For real vectors, this would mean that they are at right angles to one another (as $a \cdot b = \left| a \right| \left| b \right| \cos \theta = 0$, which can only hold if either of the vectors is $0$ or $\cos \theta$ is $0$).