Sunday 11 March 2018

Visualization Challenge on Simple Linear Algebra Problem

A $n \times n$ matrix multiplies with $n$ dimensional column vector to produce another $n$ dimensional column vector.  Here, we do the visualization of multiplication of matrix and vector to produce another vector and then see how many combinations of either vector or matrix are possible.

Dimensionally, it is given as:
$$[n\times n]\times [n \times 1]=[n \times 1]$$

The task of visualization is:
Suppose you multiply $n\times n$ matrix with $n$ dimensional column vector to produce another column vector which obviously has $n$ dimensions.
If your two vectors are known, you would have infinite number of choices for the matrix.  But if the matrix and second vector were given, you would have just one choice for the first vector if any.
Try to visualize it and convince to yourself geometrically.

Now, I would show how I did the visualization and convinced myself.*

Multiplication of $n \times n$ matrix with $n$ dimensional column vector is finding linear combination of column vectors of the matrix where the linear coefficients are given by components of the left hand side column vector.  The linear combination of those matrix column vectors would give the column vector of the right hand side.

If two vectors are given, we would have fixed linear coefficients and fixed second column vector.  That means we should choose the matrix column vectors in such a way that their linear combination, given the linear coefficients, would produce the right hand side column vector.  If we take two dimensional space and then try to visualize, it can be easily intuited.  Given a vector(of RHS), we can produce it from infinite number of combinations of two other vectors (with linear coefficients given) if we can see it by taking parallelogram law of vector addition.  Given two points as opposite edges of parallelogram, there are infinite number of ways the other two opposite points can be located to form a parallelogram.

If the matrix and second vector are given to us, let's see how we can visualize to convince ourselves that there is only one vector choice if any.  The problem translates to finding the linear coefficients such that the linear combination of the given matrix vectors would give right hand side column vector.  The linear combination of given $n$ dimensional column vectors giving given $n$ dimensional column vector has only one set of linear coefficients.  This is because each set of weights when used in doing linear combination of matrix column vectors, would give each unique vector.  So, there can be only one combination if any of the linear coefficients.  So, the vector can be only one.  It  can be 0 if the matrix column vectors are not independent.

*Also liked to add that the way people do visualization and have intuition differs from individual to individual.  This is my attempt to explain how I did the visualization.  Convincing to yourself about why something is true matters the most than how you convinced in my opinion.  Have fun!

No comments :

Post a Comment