Master Theorem

DPV Ch 2.2
The “divide-and-conquer” algorithm design paradigm works as follows.
1. Divide step divides the problems into a number of smaller subproblems, each of the same size.
2. Solve step recursively solves each subproblem.
3. Combine step combines the solutions to all the subproblems into a solution to the original problem.
Usually, the combine step is trivial or takes about as much time as the divide step.

Divide-and-conquer recurrences

Suppose a divide-and-conquer algorithm divides the given problem into equal-sized subproblems, say $a$ subproblems, each of size $n/b$. Its worst-case running time $T(n)$ then satisfies the recurrence \[ T(n) = \left\{ \begin{array}{ll} 1 & \mbox{if $n = 1$} \\ aT(n/b)+D(n) & \mbox{if $n > 1$, $n$ a power of $b$.} \end{array} \right. \] where $D(n)$ is the time spent in the divide and combine steps.
Assume that $a$ and $b$ are real numbers, with $a>0$ and $b>1$. Usually $a$ is integral but $b$ may be fractional. For example, we may have $T(n) = 3T(2n/3)+1$. Here $T$ is defined on the set of rational numbers $(3/2)^i$. The related function on integers, $T(n)=3T(\lceil 2n/3\rceil)+1$, turns out to have the same asymptotic bound.

Let $n=b^k$ so that $k=\log_b n$ ($n$ is not necessarily an integer but it is a power of $b$). Iterating the recurrence yields \begin{eqnarray*} T(n) = T(b^k) &=& D(b^k) + aT(b^{k-1}) \\ &=& D(b^k) + aD(b^{k-1}) + a^2T(b^{k-2}) \\ &=& \sum_{i=0}^{k-1} a^iD(b^{k-i}) + a^kT(1). \end{eqnarray*}
The second term $a^kT(1)$ is the solution when $D(\cdot)=0$, called the homogeneous solution (h.s.) $a^kT(1) = a^{\log_b n} = n^{\log_b a}$.
Let $h=\log_b a$ so h.s. = $n^h$. Usually $h\ge0$ since $a\ge 1$.

A common driving function is $D(n)=n^d$ where $d\ge0$. The sum becomes $n^d\sum_{i=0}^{k-1} (a/b^d)^i$, a geometric progression.
Sum of a geometric progression is \[ \sum_{i=0}^{k-1} r^i = \left\{ \begin{array}{ll} \frac{r^k-1}{r-1} & \mbox{if $r \ne 1$} \\ k & \mbox{if $r = 1$.} \end{array} \right. \]
Therefore, asymptotically \[ \sum_{i=0}^{k-1} r^i \mbox{ is } \left\{ \begin{array}{ll} \Theta(1) & \mbox{if $0<r<1$} \\ \Theta(k) & \mbox{if $r = 1$} \\ \Theta(r^k) & \mbox{if $r > 1$.} \end{array} \right. \]

Master Theorem. For $D(n)=n^d$, \[ T(n) \mbox{ is } \left\{ \begin{array}{ll} \Theta(n^d) & \mbox{if $a<b^d$, that is, $h < d$} \\ \Theta(n^h\log n) & \mbox{if $a=b^d$, that is, $h = d$} \\ \Theta(n^h) & \mbox{if $a>b^d$, that is, $h > d$.} \end{array} \right. \] Informally, “$T(n) = \max\{ \mbox{homogeneous solution, driver} \}$”