Use established software package and assign a web master. I used Grunwald's Conference Review Package (CRP) . It served well. I had one of my Ph.D. students deal with the software, including implementing minor changes. Since I knew this was a large task, I gave him credit as Web Master, an official ISCA 2005 officer. In the end, he reports that he worked 120 hours on both CRP and the conference web page.
-- excerpted (with permission) from Some Advice for Program Committee Chairs Based on my ISCA 2005 Experience by
Mark D. Hill