Definition

Sometimes vectors in the spanning set are “redundant”, meaning that the set would still span the vector space if it was removed. For example:

$$ \{\begin{pmatrix}1\\4\\7\end{pmatrix}, \begin{pmatrix}0\\2\\5\end{pmatrix}, \begin{pmatrix}1\\6\\12\end{pmatrix}\} $$

Note that the third vector in the span is in fact equal to the first vector plus the second, meaning that even if it were not there the vector space spanned would still be equivalent.

The formal definition of this “redundancy” is that a set $\{v_1,v_2,\dots,v_n\}$ is linearly dependent if there exists constants $c_1,\dots,c_n$ that are not all zero such that

$$ c_1v_1 + c_2v_2 + \dots + c_nv_n = 0 $$

If no such constants exist, i.e. $c_1 = \dots = c_n = 0$, the vectors are linearly independent.

Alternatively, we can say that $v_1, \dots, v_n$ are linearly independent if and only if each vector in $span(v_1, \dots, v_n)$ has only one representation as a linear combination.

Degrees of Freedom

The intuitive way of looking at this concept is as the “number of variables that are free to change”. For example:

$$ a+b+c=2 $$

In this equation, two of the variables are free to change, while the last is determined by the others. That is two degrees of freedom. If we add in a second equation:

$$ a-2b+3c = 5 $$

Only one variable is free to change, and the other two are modified to fit the equations. It therefore makes sense that the number of degrees of freedom is the number of variables in the system, minus the number of equations.

However, let’s pretend we get another equation into the mix.

$$ 2a - b + 4c = 7 $$

This equation is the previous two added together. This does not limit the degrees of freedom in any way, as no new relation is being expressed.

Using the notion of linear independence, we can say that degrees of freedom are really the number of variables in the system minus one per linearly independent equation.