Logarithms

DPV p50, Ch 1.2.2
A \(\log n\) factor occurs in time bounds of several algorithms, e.g., binary search, binary search tree, efficient sorting, etc. It helps to understand these algorithms by thinking base 2.

Base 2 notation

Fact. The binary numeral of any positive integer \(n\) has \(\Theta(\log n)\) bits.
For any integer \(k\ge0\), the number \(2^k\) is represented by the bit string \(1\overbrace{0\ldots 0}^{k \textrm{ 0's}}\). In general, if \(2^k\le n < 2^{k+1}\), then the base-2 numeral for \(n\) has \(k+1\) bits. Thus, the base-2 numeral for \(n\) has exactly \(\lfloor\log n\rfloor + 1\) bits.
Many algorithms get a \(\log n\) factor in their running time because they process numbers bit-by-bit.
Here are two examples.

We do a binary search on an original array of size 10, and so we processed arrays of sizes 10, 5, 2, and 1, in order. In binary notation the sizes are 1010, 101, 10, 1.
In general, an iteration starting with a size \(s\) array derives an array of size \(\le\lfloor s/2\rfloor\), e.g. size 10 can give size 5 or 4.
In binary notaion we obtain \(\lfloor s/2\rfloor\) by right-shifting \(s\) one position.
We can right-shift a number \(\lfloor \log n\rfloor\) times before it reaches 1.
Therefore, binary search does \(O(\log n)\) probes, and it runs in time \(O(\log n)\).

The method of “repeated squaring” computes \(a^n\) in \(\Theta(\log n)\) arithmetic operations.
The idea is to compute the sequence
\(a\),
\(a^2\),
\((a^2)^2 = a^4\),
\((a^4)^4 = a^8\),
\((a^8)^8 = a^{16}\),
\(\ldots\)
The binary numeral of \(n\) tells us which powers of \(a\) to use. For instance, \(a^{19} = a^{16}a^2a^1\), or in binary natation: \(a^{10011} = a^{10000}a^{10}a^1\).

Here is the low-level algorithm for fast exponentiation of \(a^n\) using repeated squaring:

x ← 1
while n > 0 {
    if n is odd {
        x ← x*a
    }
    a ← a*a
    n ← ⌊n/2⌋
}
return x

Number-theoretic computation is important in cryptography. An important algorithm in modern cryptography is modulo-exponentiation. In this problem we are given \(a\), \(n\), and \(m\), and we want to find \(a^n \bmod m\). We can modify the above repeated squaring algorithm to compute modulo-exponentiation.
We can use repeated squaring to efficiently compute matrix exponentiation \(A^n\). For example, one can compute the \(n\)th Fibonacci number using \(\Theta(\log n)\), instead of \(\Theta(n)\), scalar arithmetic operations. The algorithm relies on the fact that for all natural number \(n\), if \(F=\begin{bmatrix} 0&1\\ 1&1\end{bmatrix}\) then \(F^n=\begin{bmatrix} f_{n-1}&f_n\\ f_n&f_{n+1}\end{bmatrix}\), where \(f_i\) is the \(i\)th Fibonacci number.