Lecture 5: Dynamic Programming I
September 5th, 2007Today’s lecture covered the basics of dynamic programming, from Ch. 6 of KT. We looked at weighted interval scheduling, curve fitting, and SUBSET-SUM, and examined how to obtain solutions via dynamic programming for each of these problems.
The Wikipedia reference article on dynamic programming describes some of the history of the method, as well as providing examples of where it occurs out in the wild. Some of the more well known examples include
- BLAST, the genomic sequence analyzer
- The Viterbi algorithm for calculating likely paths in Markov chains
- The Duckworth/Lewis method for prorating required targets in one-day cricket games when rain intervenes.
The subtlety of why the SUBSET-SUM dynamic program does not run in polynomial time will be addressed in more detail when we cover NP-hardness. Remember though that “n”, the size of the input, is most precisely modelled (at least for complexity-theoretic purposes) as the number of bits needed to write down the input in a “reasonably compact” encoding. I could obviously pad any input string with 0s and 1s to make it seem longer, hence the “reasonably compact” requirement.
For example, consider a graph on n vertices and m edges. Since there are n vertices, we need $latex \log{n}$ bits to describe each vertex, and $latex 2log{n}$ bits to describe each edge. Summing this up, we need a total of $latex n\log{n}+2m\log{n}$ bits, which is $latex O((n+m)\log{n}$. This is what we expect, except for the extra $latex \log{n}$ factor.
It turns out that in “normal” graph algorithms, we cheat a little by assuming that we can perform operations on vertices (like comparing two vertices) in constant time, which would contradict the fact that a vertex representation takes $latex \log{n}$ bits. This is one aspect of the RAM (Random Access Machine) model of computation, where we assume that operations on single cells of memory can be performed in constant time, regardless of the size of the values. What this means is that instead of viewing the space needed by a vertex to be $latex \log{n}$ bits, we can pretend that it uses a constant number of bits, in which case the total number of bits needed to write down a graph reduces to $latex O(n+m)$. For almost all algorithms that we analyze, we implicitly use the RAM model of computation. This has its own problems: we can solve NP-hard problems in polynomial time in a RAM model (!) augmented with bitwise operators, but as long as you aren’t doing bizarre things with bits and encodings of numbers, you’re safe.
Numbers however are trickier. As I explained in class, the space needed to store a value W is $latex \log{W}$, and so any algorithm that takes W as input and runs in time super polynomial in $latex \log{W}$ does not run in polynomial time in the size of the input.
Entries