Problem:

One of the sites I maintain has been getting more and more traffic everyday. A very good thing for the site, not so good for the solo server which is serving those pages. The site is VERY dynamic with LAMP setup. We only have one server serving our web pages to our users. Since its a dynamic site with PHP and MySQL, it has a lot of load during peak times. Average load time of a page is between 1-2 secs during normal usage, 2-5 secs under average to heavy load.

During heavy load, we started to see our mysql stop responding to requests which is a big concern for us since we don’t show content if there is no db connection. We had to come with a solution, fast, to prevent this issue from appearing.

Solution:

So here are couple things I ended up doing on the server side to reduce load on this server without having to do much code change.

• We have couple other servers at the same location for doing other tasks. So I decided to use one of those servers to offload some of the http processing. I setup our dns to point js.domain.com and css.domain.com to go to this spare server which reduced load on our main server quite a bit. We have a lot of js and css content which is loaded every time user hits a page. Our development team took care of the including the domain in the part where it loads js/css on the page. Another benefit of doing this was to allow users’ browsers to simultaneously download from two servers at a time which in turn equates to load time decrease. Browsers only do certain # of connection per server and wait until those requests are completed before making another request. I believe both FF and IE have limit of 2 per domain.
• Second thing I did was to turn on compression at Apache level to decrease the amount of traffic sent to users. As long as users browsers support compression, apache would server compressed content. Doing this more than halved our bandwidth usage. We went from doing about 20-25 gigs/day to about 11-13 gigs.

As of now our load has been low even during high traffic times. Our bandwidth usage is low and going up with traffic as expected. Users have to wait 1-2 secs during high traffic times compared to 2-5 secs before tweaking. And during normal usage, we barely see 2 sec page load time instead its mostly only 1 sec. As of writing of this blog, we currently have 12781 users on-line with 90 sql queries per sec on average.