Growth of functions

Table from Gary & Johnson’s Computer and Intractability A Guide to the Theory of NP-Completeness

Tractable VS Intractable Problems

• A polynomially-bounded algorithm is considered good while an alorithm that requires an exponential amount of time is considered bad.
• A problem admitting a polynomial-time algorithm is called tractable (easy); one whose any algorithm that solves it requires an exponential amount of time is called intractable (hard).
• Using polynomial functions to separate the easy from the hard problems is arbitrary, but it is motivated by these facts.
  • All reasonable deterministic computational models are polynomially equivalent, i.e., any model can simulate any other model with at most a polynomial increase in running time.
  • Real-life easy problems seem to be bounded by a low-degree polynomial.

The Class P

• Definition. \(\mathcal{P}\) is the class of languages decidable in polynomial time on a single-tape DTM, i.e., \(\mathcal{P} = \bigcup_k TIME(n^k)\).
• Encoding of input must be reasonable. Specifically, input should not be so encoded as to make the algorithm efficient even though it’s not. Numbers must not be encoded in unary base; they must be encoded in base \(k\) for some \(k\ge 2\).
• Consider two encoding schemes of the same input; one encoding is \(n_1\) characters long, and another encoding is \(n_2\) characters long. The two encodings are polynomially related if there exist polynomials \(p_1\) and \(p_2\) such that \(n_1\) is \(O(p_2(n_2))\) and \(n_2\) is \(O(p_1(n_1))\).
• We will not fix input encoding but it must be polynomially related to some “reasonable encoding”. E.g., a graph may be encoded as a list of edges, an adjacency matrix, or adjacency lists.

PATH is in P

• \(PATH = \{ \langle G, s, t\rangle : G \textrm{ is a digraph with an } s,t\textrm{-dipath } \}\) is also known as the reachability problem.

• \(PATH\) can be solved by this algorithm.

  M = "On input <G,s,t> where G is a directed graph, s and t are vertices:
           mark s;
           while (there exists some edge, say uv, with u marked and v not marked) do {
              if (v = t) accept;
              else mark v;
           }
           reject;"

• Our encoding scheme
  • uses consecutive positive integers 1, 2, … to denote vertices
  • takes \(s\) to be 1 and \(t\) to be 2
  • represents \(G\) as adjacency matrix

• This algorithm then takes time \(O(n^2)\) on the RAM. So \(PATH\) is in \(\mathcal{P}\).

RELPRIME is in P

• Two natural numbers \(x\) and \(y\) are relatively prime if the only positive number that divides both of them is 1. E.g., 8 is relatively prime to 21, but 6 is not relatively prime to 81.
• \(RELPRIME = \{ \langle x, y\rangle : x \textrm{ and } y \textrm{ are natural numbers and } x \textrm{ is relatively prime to } y \}\)

• \(RELPRIME\) can be solved by this algorithm.

  M = "On input <x, y> where x and y are nonnegative integers:
           while (y != 0) do {
              (x, y) := (y, x mod y);
           }
           if (x = 1) accept;
           else reject;"

• On the RAM this algorithm takes \(O(n)\) time where \(n\) is the number of bits used to represent the maximum of \(x\) and \(y\).

CFG Parsing is in P

• Question. Given a CFG \(G\) in CNF and a string \(w=w_1\cdots w_n\), determine whether \(w\in L(G)\).
• Answer. If \(w=\varepsilon\), then the answer is YES if and only if \(S\to\varepsilon\) is a rule.
  Assume from now on that \(n>0\). For indices \(i\) and \(j\) (\(1\le i\le j\le n\)), define \(v(i,j)\) to be the set of variables that can derive \(w_i\cdots w_j\). We have the recurrence \[ v(i,j) = \begin{cases} \{ A : A\to w_i\} & i = j \\ \{ A : A\to BC, \text{there exists $k$ such that } B\in v(i,k) \text{and } C\in v(k+1,j) \} & i < j \end{cases} \] Now \(w\in L(G)\) iff \(S\in v(1,n)\).
• This solution takes \(O(n^3)\) operations on the RAM.

Lessons

• Many problems in \(\mathcal{P}\) have these characteristics.
  • They have naive exponential-time algorithms.
  • It takes ingenuity to design sophisticated poly-time algorithms to solve them.
• \(PATH\) is solved by graph searching, a technique for systematic exploration of graphs.
• \(RELPRIME\) is solved by divide and conquer, a technique that works by first dividing a problem into smaller subproblems, recursively solving the subproblems, then combining the solutions to the subproblems into a solution for the original problem.
• The CFG Parsing problem is solved by dynamic programming where we record and reuse solutions to all smaller problems we have already solved to avoid re-solving any of them.
• The subject matter of MCS-375 Introduction to Algorithms mostly concerns how to efficiently solve problems in \(\mathcal{P}\)!