I’ve been using lighttpd as my main production web server since the summer of 2007. I first heard about lighttpd from a server administrator I was working with on a project a few years earlier but since my clients weren’t getting traffic that Apache couldn’t handle I didn’t really pay it too much attention. As far as I was concerned it was a cool idea, building a better webserver, but since lighttpd was so new and support and documentation being a little sparce, compared to, say, Apache’s documentation, all using lighttpd meant was that I had to stay away from Apache specific functionality and I’d be ok.

Fast forward a couple years and I’m working on projects that Apache is really struggling with. The point of removing Apache from my toolbox started because one of our projects at StreetWise, the official community for one of our biggest client’s programs, was having serious load issues. The site was very popular, getting over 2,000,000 hits a day, and Apache was having a hard time dealing. Here we were with 8 web servers, on a load balancer, 3 DB servers, and all of the web servers were just getting hammered; it was cascading, where one by one the servers would go into swap and all traffic would be shifted to a different server, only to have the same thing happen again until all servers died. It was a very, very, stressful time for my team and I.

We tried configuring the hell out of Apache, upgrading the server’s RAM and adding additional CPUs but finally came to realize that continuing with Apache meant we’d just have to get more servers. Apache can only handle so many users at one time, like all webservers really, so we needed a web server that could handle more users at one time. It’s a pretty basic While adding additional servers to a project is cheaper than throwing people at it, if it’s a software issue, there’s additional costs with hardware in my team especially because we manage the servers ourselves.

This wasn’t good; it meant explaining to the client that we would need more money outside of our projections, which is embarrassing and, worse, my team would have to manage more servers (we were already managing 18 in total at the time).

It was at this time that lighttpd came back into play. Since Apache wasn’t working out so hot in the load we were under it had to be replaced. I remembered how lighttpd was started as a proof of concept for the C10K problem and had some pretty impressive benchmarks so I started researching more about it.

While lighttpd’s penetration wasn’t even a blip compared to Apache or IIS we installed it on a one of our dev servers and started testing it.

Installing lighttpd was pretty straightforward though setting up php to run in fast-cgi mode was a little more challenging than usual. Right away, one of the biggest challenges was the configuration though. After using Apache for, well, forever really, moving to how lighttpd was configured was pretty difficult.

lighttpd.conf example

$HTTP =~ "(www.)?your-domain.com" {
 
  server.document-root = "/www/your-domain.com/html"
 
#### auth module
## read authentication.txt for more info
auth.backend               = "htpasswd"
auth.backend.htpasswd.userfile = "/www/your-domain.com/.htpasswd"
#auth.backend.plain.groupfile = "lighttpd.group"
auth.require               = ( "/" =>
                               (
                                 "method"  => "basic",
                                 "realm"   => "Server Info",
                                 "require" => "valid-user"
                               )
                             )
 
}

Compare the above to the below.
httpd.conf example

<VirtualHost *>
    ServerName your-domain.com
    DocumentRoot /www/your-domain.com/html
    ServerAdmin contact@your-domain.com
    <IfModule mod_suphp.c>
        suPHP_UserGroup nobody nobody
    </IfModule>
</VirtualHost>

Right away, you can see how Apache and Lighttpd have different philosophies on layout and structure. Apache takes an XML structure for their configuration files while lighttpd formats its configurations like an array.

Anyway, after familiarizing ourselves in how lighttpd was configured we started the process of moving all the production servers away from Apache over to lighttpd. We went one server at a time, installing lighttpd, configuring, benchmarking and then changing the server’s init so lighttpd was started and Apache wasn’t. All told the process took about a week.

Since then, I haven’t had a single issue related to load on one of lighttpd boxes. If you’re serious about performance and care about how your projects function check out lighttpd ASAP.