Regarding reverse proxys

Right now I use both Perlbal and Lighttpd as reverse proxy.

Lighttpd does a lot more than reverse proxy, and I also use it as a web-server for static content and mod_secdownload for certain content. But the current stable 1.4.x version is not that good reverse proxy. For example, it does not keep persistent connections to back-end servers, and sometimes he thinks all my application servers are dead when they are in perfect health.

Perlbal is a much much better reverse proxy. It keeps persistent connections up to back-end servers, and before using them for client requests, it makes sure the server is really ready to answer requests. It also caches file uploads to disk before allocating a back-end server to process the request. All this features keep the back-ends pretty busy and without stalls. But as a web server for static content, its just not as good as Lighttpd.

I've been working with three upgrade paths.

The first is use Perlbal for reverse proxy for the application servers and remap all static content to a different site. This also makes sense because if you move your static content to a CDN, your site will load a lot faster.

The second is using Lighttpd 1.5.x. The last time I tested it, it was still a bit unstable with my setup, so its in the back-burner for now.

The third is to use Varnish. Its wicked fast, extremely configurable, and it does caching. But it takes a bit to get into, so its a bigger time investment than the other two.

Right now, I think that Varnish would be the best bet of them all, but I don't have the time right now to get up to speed with it. So I'm going to use the first option, using Lighttpd 1.4.x to power my CDN server, just because I need modsecdownload.