Lighttpd as reverse proxy
We often recommend to set lighttpd in front of apache to handle http requests (more about https://www.percona.com/blog/2006/05/21/speedup-your-lamp-stack-with-lighttpd/ ) , redirect dynamic requests to apache and handle static files by itself. I just gathered step-by-step instruction how to do that in 10 minutes, as it may be not so obvious.
- Of course you need lighttpd by itself, it’s available on http://www.lighttpd.net/download
- You may want pcre-devel packet, which allows to use regular expressions in lighttpd.conf. For CentOS boxes we just run
1yum install pcre-devel
- Take sample config file
and put to1lighttpd-1.4.XX/doc/lighttpd.conf1/etc/lighttpd/lighttpd.conf
- Create directory to store log files, e.g.
. Despite it looks simple, lighttpd may just die trying to create files in non-existing directory.1/var/log/lighttpd
- Set
to point to actual directory with web documents1server.document-root = "/www/html/htdocs"
- Uncomment next lines in lighttpd.conf
1234567server.modules = (..."mod_auth","mod_status","mod_proxy",...)mod_status is useful to show additional information from lighttpd
mod_auth to protect mod_status from unauthorized access 🙂
and mod_proxy is proxy by itself - to enable status uncomment or add
to lighttpd.conf1status.status-url = "/server-status"
- to protect access to /server-status put next lines to lighttpd.conf
12345678910auth.backend = "htpasswd"auth.backend.htpasswd.userfile = "/var/www/.htpasswd"auth.require = ( "/server-status" =>("method" => "basic","realm" => "status","require" => "valid-user"))It enables htpasswd file based authentication, you can just manage this file with htpasswd utility from apache.
- and finally make changes related to proxy
1. Move apache from 80 port to another, e.g. 8080 or pick your own.
You need to changeand1Listendirectives in httpd.cong1VirtualHost
and meantime set, as we want apache only handling dynamic pages.1KeepAlive Off
2. put next lines to lighttpd.conf:
123456$HTTP["url"] !~ "\.(js|css|gif|jpg|png|ico|txt|swf|html|htm)$" {proxy.server = ( "" => (( "host" => "127.0.0.1", "port" => 8080 )))}
This actually directs lighttpd to redirect all requests, besides one ending(you can add your own static pages), to apache listening on 8080 port1js|css|gif|jpg|png|ico|txt|swf|html|htm - Final step to add lighttpd to list of services (actual only for RedHat / CentOS)
1cp <sourcedirt>/doc/rc.lighttpd.redhat /etc/init.d/lighttpdfix path
1lighttpd="/usr/local/lighttpd/sbin/lighttpd"and add script to autostart
12chkconfig --add lighttpdchkconfig --level 2345 lighttpd on
That’s all. For complex apache setups and VirtualHosts it may be trickier, but work just fine for simple cases.
Comments (28)
This is interesting – we actually do the opposite 🙂 – apache as reverse proxy + mod_security + static file handling in front of the lighttpd/fcgi.
ew. you should look at nginx. ncache (based on nginx) if you need a proxy that -stores- a cache of the content.
nginx is the most efficient proxy out there, hands down. once you try it you -will not- go back.
Here, we use “HAproxy” in front for reverse proxy and load balancing.
It’s VERY fast to parse HTTP requests and it handle failover.
It’s possible to do the same thing with it : send all static content requests to lighttpd and all dynamic content to apache.
If you have a lot of traffic, you should look at this 🙂
http://haproxy.1wt.eu/
we do not handle static files at all 😉 we completely rely on amazon S3 for that and lighttpd just handle the dynamic pages
to solve the ETAG, expiration of new static files, we use either a revision number in the URLs or an SVN tag
we also decide to compress all the files we put on amazon S3 as gzip mode in mainly supported by all browsers
What do you mean with compress all the files? They’re just compressed with gzip and then decompressed on the fly by the web browser???
For example, having file.txt, do you mean compressing it to file.txt.gz and be able to access http://server.com/file.txt ? I’d love to know how to do it!
Or is just enabling the on the fly server compression like mod_deflate/mod_gzip, wich is forbidden for it’s cpu usage by most hostings?
Isn’t it better to use a cache/accelerator dedicated product like squid as a front-end instead of lighttpd?
use nginx for this.
it is better then lighttpd
now 2 million hosts use this server (month ago — 1 million…)
http://nginx.net/ enjoy
( http://wiki.codemongers.com/Main )
Isn’t Lighttpd better suited as a replacement for Apache rather than a front end to it ? To serve static content I recommend using thttpd, it is very fast, very stable and very light on resources.
or tux 🙂
http://www.redhat.com/docs/manuals/tux/TUX-2.2-Manual/
If you wish to give the impression of a single domain surely using Squid in reverse proxy mode is the simplest? Get one httpd to serve both static and dynamic, but get squid to cache the static?
At fav.or.it we have a dedicated domain for static content, which is served by lighttpd. Apache then hosts the dynamic.
sent from: fav.or.it [FID179790]
I tried this on my website, but then the forum software got completely confused: all users’ IP addresses were listed as being 127.0.0.1, which caused problems (too many incorrect logins from the same IP address = all logins blocked for 15 minutes). I just switched entirely to lighttpd/FastCGI, and never looked back.
We use Apache with threaded MPM as reverse proxy (and for static files) and Apache prefork MPM for serving PHP files.
Big advantage of this solution is same configuration ( .htacces, mod_rewrite and so) for both webservers. Threaded MPM Apache has decent memory consumption and almost same performance as lighttpd with sendfile network module.
My problem with lighty is that the project is kind of stalled or improving *very* slowly. The last development release was last September and 1.4.x releases are mostly bugfixes.
It’s an interesting setup that you present here, but my question is what are your reasons to put Apache as the second layer? Why not a first layer of Lighty and a second layer of fastcgi servers (e.g. php-fastcgi)?
why on earth would you not just use lighttpd to serve all the traffic?
NagyZ
We often use lighty as a quick fix to “spoon feeding and keep alive” problem even medium size web sites get to very quickly when Apache is used. Many applications use mod_rewrite .htaccess or other apache specific features which make it longer job including QA etc. Full migration to Lighty takes longer time if desired.
Saiyine
Lighttpd can compress static files and and compressed result (to not perform compression each time)
just enable compression module and use next lines
#### compress module
#compress.cache-dir = “/var/cache/lighttpd/compress/”
#compress.filetype = (“text/plain”, “text/html”)
To all.
Regarding lighttpd vs ngnix.
Lighttpd is used by youtube.com and wikipedia, so it is proven to be able to work in high load environment.
When we started use lighttpd it seemed more advanced than ngnix, but currently ngnix is still begin active developed, while author of lighttpd is more interested in mysql-proxy development and lighttpd looks frozen, so I even agree that ngnix looks preferable.
Emil,
Why do you do that ? for mod_security ?
@6: Yes, it is. Using lighty in front of Apache instead of in parallel is a somewhat strange construct but depending on your setup/overall goal it might make sense.
Though what most ppl want to achieve is a faster and less CPU&IO intensive delivery of their static contents.
Now squid hasn’t been the best choice to do this for about 10 years now, since it’s internal architecture is somewhat 80s style (find a good description of the details at http://varnish.projects.linpro.no/wiki/ArchitectNotes). There’s been commercial alternatives like cachebox and netcache around for a while now.
A very nice, open source, alternative is varnish (http://varnish.projects.linpro.no/). It provides a pretty flexible scripting language that gets compiled on the fly and offers fine-grained control over “what to do with this http request”. We measured a throughput of about 10.000 requests/sec. on a single Core2Duo Desktop Box using just one of the two cores. On a typical modern Xeon 2x Quadcore Server you should expect to get something around 40-80k requests/sec – that is, if your servers network connection offers sufficient bandwith.
A quick apache bench from my macbook pro to the desktop box mentioned shows the following result on a GigE network:
=========================
Server Software: Apache/2.2.3
Server Hostname: cache2.lcms.c4.dev.local
Server Port: 80
Document Path: /login.html
Document Length: 2241 bytes
Concurrency Level: 200
Time taken for tests: 10.946997 seconds
Complete requests: 100000
Failed requests: 0
Write errors: 0
Total transferred: 254900000 bytes
HTML transferred: 224100000 bytes
Requests per second: 9134.93 [#/sec] (mean)
Time per request: 21.894 [ms] (mean)
Time per request: 0.109 [ms] (mean, across all concurrent requests)
Transfer rate: 22739.11 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 3 2.4 3 15
Processing: 4 18 1.8 18 27
Waiting: 0 7 1.9 8 17
Total: 6 21 3.3 21 32
Percentage of the requests served within a certain time (ms)
50% 21
66% 23
75% 24
80% 24
90% 26
95% 27
98% 27
99% 28
100% 32 (longest request)
=========================
So, if all you want to do is accelerate access to your static content, varnish might be a pretty good option. Of course lighty offers alot more than just content caching so I guess depending on what one’s goals are the one or the other might be better suited.
Cheers,
— Lukas
We just use apache. We can handle around 50k connections fairly per server easily. We run all the dynamic stuff via fastcgi or mod_jk to Tomcat for Java. The whole “apache is slow” thing doesn’t make sense if you actually try to configure apache beyond the configs most vendors supply.
Brian, I am dumbfounded that one server can handle 50k connections. Say each request is processed in a second that would equate to over 4 Billion hits a day !
FWIW, an old Wikimedia report[1] says that they maintain only 14,000-32,000 connections per *Squid*, never mind the Apaches (which are presumably much lower). Of course, they use a mod_php setup for the Apaches, apparently because nobody’s ever gotten around to changing it.
[1] http://www.scribd.com/doc/43872/Wikimedia-architecture (not the original source, but the first Google hit)
JohnSep, of course we don’t do 50k all day long… But 50k connections on a “normal” server is not that “hard” for apache. You have to tune your OS and Apache – not as much as you would think – but you have to do that anyway. The GigE interface is almost always the bottleneck.
An old presentation with 2004 data I did : http://www.akins.org/files/apachecon-cnn-good.ppt
Most of the cool stuff is in normal Apache now (or some implementation of it)
Lukas,
We tried using Varnish for one of the projects but had so many crashes we had to pull back to the squid for a while.
I think Varnish is a great project but it needs to mature still, at least for some use cases.
Brian,
I think when we speak about Apache we mainly mean apache in prefork mode w process per connection which obviously does not scale. Recent apache versions have event driven handling which should place them close to lighty and nginx. There is still more “fat” in apache though for most users it does not matter.
peter, it’s not hard to get Apache to be on a diet, just don’t load stuff you don’t need. Plus, apache 2.x, with full threading, has been out for several years. Also, the whole “a server must be event/async to perform well” mantra doesn’t hold up in the real world. I like alot about lightttpd and nginx, but the performance FUD is just that. It just seems they come with a more “reasonable” default config.
Hi,
I am just curious why don’t you just use lighthttpd as an Apache replacement ?
Isn’t is stable for serving dynamic content ?
Slavi
Hello Vadim,
I read and tested several ways of lighthttpd / apache connectivity –
lighthttpd + proxy of php files to apache
apache + static server
lighthttpd + fast cgi running on different host
etc..
even saw stupid ideas like
apache that proxy the pictures to the lighthttpd
I can congratulate you – this is the best way I had in mind and is the only one page where description is clear.