Graduate Algorithms

When you’ve shown a problem is NP-hard, and designed whatever approximation you can, and still aren’t close to solving a problem to the level needed, heuristics are often the next step.

From an algorithms perspective, what distinguishes a heuristic from an algorithm is the lack of formal guarantees. This deficiency can show up in a variety of ways:

• A heuristic may not have a clear stopping condition
• A heuristic will in general not guarantee any kind of quality of solution (except possibly local optimality)

If that’s the case, why use a heuristic ? Usually, it’s because the problem is too complex to analyse using formal approaches. Also, each problem often requires a completely different method of analysis if we’re looking at formal guarantees. Heuristics provide a generic approach that can “work” on a variety of problems.

Since you can’t usually evaluate a heuristic via formal methods, the questions you ask about a heuristic are usually different:

• What is a good stopping criterion ? This is especially tricky for iterative schemes: usually, you want to stop when the difference between the previous solution and the current one falls below some threshold: choosing that threshold is the hard part.
• Does the heuristic converge in a “reasonable amount of time” ? Many heuristics will make progress very quickly, and spend a lot of time towards the end, as they get closer and closer.
• How good do the solutions look ? You will often find silence on this point, mainly because to evaluate the “goodness” of a solution, you need some kind of lower bound on the optimal solution, which is also hard to find. Often, there are other ways of evaluating the quality of a solution, depending on the domain.

We’ll look at meta-heuristics in this lecture. You can think of meta-heuristics as general design principles that can be instantiated for any specific problem, giving an actual heuristic for that problem. Some examples:

• Local search (also hill climbing, gradient descent)
• The Metropolis algorithm
• Simulated annealing, and some cooling schedules
• Tabu search
• Evolutionary algorithms.

This entry was posted on Saturday, November 24th, 2007 at 11:43 pm and is filed under Lectures, News.