A2: Pseudocode & Asymptotics
San Skulrattanakulchai
September 7, 2018
Pseudocode
- Liberal use of global variables.
- Indentation indicates block structures.
- The pair of symbols { and } is used to delimit a block.
- A statement has no termination symbol unless ambiguous, i.e., a statement may span several lines, and a line may contain multiple statements, in which case semicolons are used as statement separators.
I use
for ... do ...
while ... do ...
repeat ... until
loop constructs. The test condition of the for and while loops is executed one more time than its body; the test condition of the repeat loop is executed as many times as its body.
Pseudocode (continued)
- Both /* */ style comment and // style comment are used.
- Objects have attributes which are accessed using the dot notation.
- An array is an object having a `length’ attribute. A[a..b] is a subarray indexed from a to b. A[i] accesses the ith element of A. Array indexing starts from 1 almost always. However, we’ll index starting from 0 when convenient.
- The notation A[i,j] denotes the element at row i and column j of a 2-dimensional array A.
- The logical operators (and, or) are short-circuiting.
- Object parameters are passed by reference; non-object parameters are passed by value.
Asymptotics
- A real-valued function is said to be defined on virtually all of the non-negative integers \(\mathbb{N}\) if it is not defined on only a finite number of them. For example, \(f(n)=\lg \lg n\) is such a function since it is defined for all integers \(n\ge2\).
- A function \(f\) that’s defined on virtually all of \(\mathbb{N}\) is said to be eventually positive if there exists \(n_0\in\mathbb{N}\) such that \(f(n) > 0\) for all \(n\ge n_0\). An example of such a function is \(f(n)=n^2-4\); a non-example is the function \(f(n)=-2n^2+100n+50\).
- The functions of resource (time & space) usage of algorithms are always defined on virtually all of \(\mathbb{N}\), and are eventually positive.
- Let \(f,g\) be such functions. We write \(f\) is \(O(g)\) (reads \(f\) is of order at most \(g\) or \(f\) is big-Oh of \(g\)) to mean there exist positive number \(c\), and non-negative integer \(n_0\) such that \(f(n)\le cg(n)\) for all \(n\ge n_0\).
Examples
- \(1000n-2000\) is \(O(n)\).
- \(n^3+100n^2\) is \(O(n^3)\).
- \(n+\sqrt n\) is \(O(n)\); \(5\sqrt n + 500 n^{1/3}\) is \(O(n^{1/2})\).
- \(3 + \sin n\) is \(O(1)\).
- \(\log_a n\) is \(O(\log_b n)\) for any constants \(a,b>1\). (So, bases may be dropped.)
- \(20\log n\) is \(O(\log^2 n)\).
- \(2^{n+1}\) is \(O(2^n)\); \(2^{2n}\) is not \(O(2^n)\).
Helpful rules
- Let functions \(f, F, g, G\) be given such that \(f\) is \(O(F)\), and \(g\) is \(O(G)\).
- Rule of Products: \(fg\) is \(O(FG)\).
E.g., \((1000n)(0.5n^2+50n^{3/2})\) is \(O(n^3)\).
- Rule of Exponents: \(f^k\) is \(O(F^k)\) for any constant \(k>0\).
E.g., \((n+10)^2\) is \(O(n^2)\); \(\sqrt{n^2+100n}\) is \(O(n)\).
- Rule of Sums: \(f+g\) is \(O(\max \{F, G\})\). E.g., \(10n^4+1000n^3\) is \(O(n^4)\);
\((n^3/10 + 100n^2)(\log^3 n + 50\log^2 n+\log n)^4\) is \(O(n^3\log^{12} n)\).
Other asymptotic notions
- We express upper bounds with \(O(\cdot)\). Similarly, we can express lower bounds with \(\Omega(\cdot)\) and equality with \(\Theta(\cdot)\).
- \(f\) is \(\Omega(g)\) means there exist positive number \(c\), and non-negative integer \(n_0\) such that \(f(n)\ge cg(n)\) for all \(n\ge n_0\).
Examples: \(n\sqrt{n}\) is \(\Omega(n)\); \(n/100\) is \(\Omega(n)\).
- \(f\) is \(\Theta(g)\) means there exist positive numbers \(c_1, c_2\), and non-negative integer \(n_0\) such that \(c_1g(n)\le f(n)\le c_2g(n)\) for all \(n\ge n_0\).
Examples:
\(100n\) is \(\Theta(n)\);
\(23n^3+1000n^2-10\) is \(\Theta(n^3)\).
- The following two notations \(o(\cdot)\) and \(\omega(\cdot)\) behave similarly to \(\prec\) and \(\succ\), respectively.
- \(f\) is \(o(g)\) means \(\lim_{n\to\infty} f(n)/g(n) = 0\).
- \(f\) is \(\omega(g)\) means \(\lim_{n\to\infty} g(n)/f(n) = 0\).
Exercises
- Show that \(f\) is \(\Theta(g)\) if and only if \(f\) is \(O(g)\) and \(f\) is \(\Omega(g)\).
- Show that \(\Omega\) and \(\Theta\) satisfy the same rules as for \(O\) given previously.
- Show that \(O\), \(\Omega\), and \(\Theta\) behave similarly to \(\preceq\), \(\succeq\), and \(\equiv\) respectively (i.e., partial order, partial order, and equivalence relation respectively).
- Show that \(o\) and \(\omega\) behave similarly to \(\prec\) and \(\succ\) respectively, i.e., they are strict partial order relations.
- (Polynomials) Fix a positive integer \(d\), positive number \(a_d\) and real numbers \(a_0, a_1, \ldots, a_{d-1}\). Show that \(\sum_{i=0}^d a_in^i\) is \(\Theta(n^d)\). For example, \(n^{10}-100n^9 + 1010n^4 - 200\) is \(\Theta(n^{10})\).
- \(\sum_{i=1}^n i^k\) is \(\Theta(n^{k+1})\) whenever \(k\) is any fixed, positive constant.
- For positive \(a\), we have \(\sum_{i=0}^n a^i\) is \(\Theta(1)\) if \(a<1\), is \(\Theta(n)\) if \(a=1\), and is \(\Theta(a^{n+1})\) if \(a>1\).
- \(\log n!\) is \(\Theta(n \log n)\).
Notes
- Definition of \(O, \Omega, \Theta, o, \omega\) can readily be generalized to functions of \(> 1\) variable, and/or functions whose domain is the set of reals.
- Splitting technique is useful for proving asymptotics.
- People also write \(f=O(g)\), \(f=\Theta(g)\), etc. In such cases, \(O(g)\) and \(\Theta(g)\) should be interpreted as sets, and \(=\) should be interpreted as \(\in\). When asymptotics appear in a formula, they need to be properly intrepreted correctly as well. Moreover, their meanings depend on whether they appear on the left or right side of the equality sign!