• I talked about computational geometry and output sensitivity. In particular, I mentioned convex hull algorithms, and how to make them optimally output sensitive.
• Next, we looked at LP-duality, and how the max-flow min cut theorem can be viewed as a special case of general LP-duality
• Online algorithms are fun to think about, especially the rent-buy ski problem, and the parking lot problem.
• Finally, we looked at amortized analysis, and why it’s good to double every time you reallocate a growing array.

I drew on many sources to cobble together material for this lecture.

• Quantum Computing (lecture notes) by Dave Bacon (the Quantum Pontiff)
• Quantum Computation (lecture notes) by Umesh Vazirani
• Quantum Computing and Quantum Information, by Nielsen and Chuang.

Some additional links that might be useful:

• Flynn’s categorization of machine models.
• Amdahl’s Law.
• P-completeness

From an algorithms perspective, what distinguishes a heuristic from an algorithm is the lack of formal guarantees. This deficiency can show up in a variety of ways:

• A heuristic may not have a clear stopping condition
• A heuristic will in general not guarantee any kind of quality of solution (except possibly local optimality)

If that’s the case, why use a heuristic ? Usually, it’s because the problem is too complex to analyse using formal approaches. Also, each problem often requires a completely different method of analysis if we’re looking at formal guarantees. Heuristics provide a generic approach that can “work” on a variety of problems.

Since you can’t usually evaluate a heuristic via formal methods, the questions you ask about a heuristic are usually different:

• What is a good stopping criterion ? This is especially tricky for iterative schemes: usually, you want to stop when the difference between the previous solution and the current one falls below some threshold: choosing that threshold is the hard part.
• Does the heuristic converge in a “reasonable amount of time” ? Many heuristics will make progress very quickly, and spend a lot of time towards the end, as they get closer and closer.
• How good do the solutions look ? You will often find silence on this point, mainly because to evaluate the “goodness” of a solution, you need some kind of lower bound on the optimal solution, which is also hard to find. Often, there are other ways of evaluating the quality of a solution, depending on the domain.

We’ll look at meta-heuristics in this lecture. You can think of meta-heuristics as general design principles that can be instantiated for any specific problem, giving an actual heuristic for that problem. Some examples:

• Local search (also hill climbing, gradient descent)
• The Metropolis algorithm
• Simulated annealing, and some cooling schedules
• Tabu search
• Evolutionary algorithms.

To play a Flash game on signed permutations, click here.

Extra material:

• The only paper Bill Gates ever published. (and it’s on pancake flipping)
• A pancake-flipping applet.
• Even more on pancake sorting.

I also talked about the power of two choices: the idea that when placing balls in bins, merely choosing two locations at random and picking the better one can lead to an exponential improvement in the maximum load of a bin (from $latex \log{n}$ to $latex \log\log{n}$). A nice survey that reviews this idea (and gives three different proofs of the main theorem) can be found here.

Various books have decent coverage of Markov and Chebyshev bounds: the textbook does NOT cover these bounds.The notes I linked above cover the bounds, but not the specific examples relating to occupancy. I have to update my notes, and in the meantime, you can get more information from these notes.

Two useful textbooks for randomized algorithms in general are:

• Randomized Algorithms by Motwani and Raghavan
• Probability and Computing by Mitzenmacher and Upfal.

Additional reading: Some of the material covered today is best explained in these notes.

Aftermatter:

• Uri Zwick’s website: Presentation on Jenga, and the paper.
• An article by Brian Hayes that explains the result on stacking bricks.