So when I’m not working, blogging, or slacking off, I do development work at Neowin. I mostly prefer to work on making the new toys (who doesn’t?), but a few days ago I had a chance to test our site under the most traffic it has gotten in a long time.

During Wednesday afternoon, Neowin hit the homepage of Reddit and Digg. Now while this wasn’t the first time our site has been on the homepage of a social website, it is one of the first times that our forums, powered by IP.Board, was the focus of the traffic. While our main homepage has stood up to this kind of traffic before, we were unsure how the forums would handle it.

It actually didn’t turn out that great.

As soon as the forums started to see around (estimated) 10,000 users at a single time, the site crippled and neither the regular visitors or the new ones from Digg and Reddit could view the content they wanted to see. This was due allegedly to the MySQL server being abused by the thousands of queries being requested by it every second.

As a result we decided to take down the forums and put up a static page that linked to the content (images) that the visitors from the social sites wanted to see. The parts of the site that were still online instantly came to life again, but now members in our IRC chat were starting to become frustrated about the forums being shut down.

At that time I decided to make it so only traffic to the specific topic would result in a static page so everybody would be able to get what they want. It took only a couple of seconds to modify the code and we had the forums back online and the social website viewers were still getting their static page that used a total of zero queries!

We left things alone for a while as the traffic wouldn’t be going away for a while — it was on Digg’s front page for hours. Once and a while we would take our static page off to see if the forums could handle the traffic again, but we watched as the site was slowly crippled again.

During this time a post was published to our front page discussing the incident and also linking to the images that were of interest to our new visitors. We decided to see what would happen to our server if we directed all the social website traffic to this article on the front page. So with a few quick code changes, all the social website traffic was now diverted to our front page code system (a completely internal project to Neowin). What was the result? Our servers survived. We watched as the strain jumped on our servers due to increased load of all the various images, files, and queries being requested by the thousands of visitors. Did we buckle under the load? Not once.

So while we’re now looking into plans to prevent such a crippling action on our forums, it’s good to know that our front page can handle whatever is dished out to it.

Neowin: Worst hard drive crash I’ve seen

Digg: Worst hard drive crash ever [pics]
Reddit: Worst hard drive crash I’ve seen [Pics]