Matrix Multiplication

DPV Ch 2.5
Let $X$ and $Y$ be 2 integral matrices, each of dimension $n\times n$. The matrix product $XY$ is defined to be the matrix $Z$ of dimension $n\times n$, where \[ z_{ij} = \sum_{k=1}^n x_{ik} y_{kj} \] for all $1\le i\le n$ and $1\le j\le n$. Computing $XY$ using this definition takes $\Theta(n^3)$ integer operations (multiplications & additions).
Strassen's method of matrix multiplication beats this bound.

Strassen's method

Strassen's method relies on {block multiplication}. Assuming $n$ is a perfect power of 2 and writing \[ X = \begin{bmatrix} A & B \\ C & D \end{bmatrix} \] and \[ Y = \begin{bmatrix} E & F \\ G & H \end{bmatrix} \] where each of the submatrices $A$, $B$, $C$, $D$, $E$, $F$, $G$, and $H$ are of dimension $n/2$ by $n/2$, we see that \[ XY = \begin{bmatrix} A & B \\ C & D \end{bmatrix} \begin{bmatrix} E & F \\ G & H \end{bmatrix} = \begin{bmatrix} AE+BG & AF+BH \\ CE+DG & CF+DH \end{bmatrix}. \] We now define the following 7 matrix products \begin{eqnarray*} P_1 = A(F-H) &\quad& P_5 = (A+D)(E+H) \\ P_2 = (A+B)H &\quad& P_6 = (B-D)(G+H) \\ P_3 = (C+D)E &\quad& P_7 = (A-C)(E+F) \\ P_4 = D(G-E) && \end{eqnarray*} and we see that \[ XY = \begin{bmatrix} A & B \\ C & D \end{bmatrix} \begin{bmatrix} E & F \\ G & H \end{bmatrix} = \begin{bmatrix} P_5+P_4-P_2+P_6 & P_1+P_2 \\ P_3+P_4 & P_1+P_5-P_3-P_7 \end{bmatrix}. \] The number of integer multiplicatins $T(n)$ using the above block multiplication satisfies the recurrence \[ T(n) = \left\{ \begin{array}{ll} 1 & \mbox{if $n = 1$} \\ n^2 + 7T(n/2) & \mbox{if $n > 1$, $n$ a power of $2$.} \end{array} \right. \] By the Master Theorem, $T(n) = O(n^{\log_2 7}) = O(n^{2.81})$, which is $o(n^3)$.

We assume the size $n$ is a perfect power of 2. If it is not, we can simply augment the input matrices by padding it with enough 0 entries.
Strassen's algorithm is useful in practice only when the input matrices are large.
An even faster matrix multiplication algorithm by Coppersmith and Winograd exists. It runs in time $O(n^{2.376})$. However, it is not useful in practice due to the large constant hidden in the big-O notation.