Algorithms (F2008)

Today’s lecture is drawn from many sources. The basic idea of a zero knowledge proof was developed by Shafi Goldwasser, Silivio Micali (MIck Ali) and Charles Rackoff in a paper first published in 1985. The explanation of ZK proofs in terms of Ali Baba and the 40 thieves is drawn from this reference.

Sudoku (in its n X n variant) was shown to be NP-Complete in 2003. The physical proof I demonstrated is due to Gradwohl et al, and a fun variant of this proof that uses playing cards can be found at Moni Naor’s Sudoku page.

The Millionaire’s problems was first coined by Andy Yao, and the solution comes from his paper. The exact demonstration (and the use of the public-private key pair) comes from this site.

If you want to try this at home, this is a useful RSA encryption applet

Notes and reading material from today’s lecture are linked here. In addition, here’s some research by Adam Teichert (thanks Adam !) on the origin of the Metropolis-Hastings algorithm:

After looking at Wikipedia for a refresher (and doing a bit of other searching), it looks like some people do refer to the Metropolis-Hastings algorithm as just the Metropolis algorithm, however some people (at least) make a distinction between the two:

Wikipedia says, “The original Metropolis algorithm calls for the proposal density to be symmetric ( Q(x;y) = Q(y;x) ); the generalization by Hastings lifts this restriction.”

This pdf describes the difference between implementing Metropolis and Metropolis-Hastings in R:
http://nitro.biosci.arizona.edu/courses/EEB581-2006/IntroR/Metropolis.pdf

Finally, I noticed that the notes in the book say that the simulated annealing was based on the 1953 work of Metropolis et al. Wikipedia tells us that Hastings generalized the Metropolis algorithm
in 1970. (http://probability.ca/hastings/).

We went through a series of examples of randomization.

1. Randomized median finding: how to apply randomization to divide and conquer
2. Randomized MAX 3SAT: The probabilistic method, and designing an approximation algorithm that uses randomness
3. The Birthday Paradox: we analyzed the well known phenomenon, and connected it to a problem in hashing (how many items do you need to guarantee a collision in n buckets).

The first two topics were taken from the textbook, and the last topic is covered in the linked notes.

We covered tail bounds: a way of determining not just the expected value of a random variable, but the probability of it exceeding the mean by some predetermined value. Notes on this can be found here.

Hi all,
sorry for the late posting. Scott’s notes for Wednesday’s lecture can be found here.

Hi all,
Scott Alfeld will be our guest lecturer for Monday’s class (and Wednesday). The material is taken from the textbook, and here are Scott’s notes.

A PTAS is a scheme to get an arbitrarily good approximation for an NP-complete problem. Given an input parameter , the goal is to find a -approximation that runs in time polynomial in n.

It’s often the case that such algorithms can be generated by designing an expensive dynamic program, and then adapting it. We demonstrate this idea with the KNAPSACK problem.

We covered material from Chapter 11 of KT, and considered two examples: Firstly, we looked at a simple scheduling problem and showed how to get a 2-approximation using a greedy algorithm. Then, we looked at the Travelling Salesman problem. After showing that in its most general form, no approximation ratio is possible unless P = NP, we showed that if we require that edge weights form a metric, then a heuristic attributed to Christofides enables us to get a 1.5 approximatino.

I’ll be using Chapter 11 from the textbook as a guide for our journey through approximation algorithms. However, I’d like to set the foundations with some basic material that I’ll draw from other sources. Since I don’t have comprehensive lecture notes yet, here are some links, with page numbers to look at:

• Rajeev Motwani’s (old) lecture notes on approximation algorithms. The notes are old, but the fundamentals haven’t really changed. I’ll be drawing on material from Chapter 1 (pages 1-33: the definitions, and the impossibility results).
• Sariel Har-Peled’s notes on approximation algorithms. The relevant material is in pages 1-2 (the greedy approximation algorithm for VERTEX COVER)

We practised reductions for a number of problems. First of all, I showed how to prove that Independent Set is NP-Complete, assuming that 3SAT is. The proof has a number of steps:

1. Show that Independent Set is in NP, by demonstrating that a guessed solution can be verified in polynomial time
2. Construct a transformation f that takes an arbitrary instance x of 3SAT and maps it to an instance f(x) of Independent Set, and does this in polynomial time.
3. Show that if x is satisfiable, then f(x) has an independent set of size k
4. Show that if f(x) has an independent set of size k, then x is satisfiable.

After that, we broke out into groups and looked at a number of NP-Completeness proofs. The problems we considered were:

• SAT (via CIRCUIT SAT)
• 3SAT (via SAT)
• SET PACKING (via INDEPENDENT SET)
• Not-All-Equal SAT (via SAT)
• TSP (via Hamiltonian Path)
• HITTING SET (via SET COVER via VERTEX COVER)

Not-All-Equal SAT is SAT, but where a clause must be satisfied without setting all literals to 1. All the other problems are defined in the textbook.

All of you worked on at least one of these problems, but I encourage you to try your hand at the others. You can be fairly sure that I’ll be asking some NP-Completeness questions either in an assignment or in the final.