I've done some considerable changes to how Pandemic-Legion.com www frontend works.
Previously everything was hosted on an apache, with fastcgi+php, xcache it would still run slow when the db backend was lagging and use a hefty amount of RAM serving small amount of users.
I've changed this to run "pound" as reverse proxy. Pound is a load-balance and SSL accelerator proxy. SSL serving is now much faster than when apache had to do it. We can now also add more content servers easily when we decide load balancing is needed. Pound supports sticky sessions, round robin and most other load balancing techniques out of the box!
It's configured with a HTTP listener and a HTTPS listener.
The HTTP listener forwards all connections on port 80 to port 433 and the HTTPS listener.
Pound is then serving requests from/to a caching proxy. In this case varnish which in turn is configured with a couple of vcl configs to serve cached content to users that arent logged in.
Varnish passes requests and serves and caches data from apache backend.
User -> Pound (Public 80/443 -> 443) -> Varnish (Local 6081) -> Apache (Local 80)
I had some issues forwarding the real client IP of visitors from pound to varnish to apache.
This is solved by two methods.
Background:
Most proxies set a http header with X-Real_Forwarded-For and Pound is no different.
Problem is when you got a second proxy inbetween. The second proxy will see the client IP being the first proxys IP.
The solution was to define in vcl
Code:
if (req.http.X-Forwarded-For) {
set req.http.X-Forwarded-For = req.http.X-Forwarded-For;
} else {
// Simply use the client IP if nothing forwarded in X-Forwarded-For - this should not happen
set req.http.X-Real-Forwarded-For = regsub(client.ip, ":.*", "");
}
Pound will pass along a http header to varnish with the client IP in x-forwarded .
In varnish the x-forwarded-for value is updated with the already set x-forwarded-for value.
Headers arent editable while in the pipe, so Im not really worried about clients trying to spoof x-forwarded-for. Although this could use some testing.
Worst case scenario, all accesses are also logged with real IP in pound to syslog.
To make apache replace the remote_addr value in header we use an apache mod called mod_rpaf that does this for us.
The result is as following:
Cache hits are served for users who arent logged in. Improving both performance for non logged in and logged in users.
People who are logged in on forums and actively browsing the site will not get served cached results nor will their views get cached and served others.
There's also a grace period if backend (apache) doesnt respond for non logged in users. Which means varnish will serve content out of cache if apache has died for some reason.
Logged in users will still feel a bit sluggish during backups but the anonymous users wont hammer the site anymore.
I will work to improve the vcl config to varnish, adding support for purging cache when a new killmail/comment has been submitted. For now the ttl for cache is 5 minutes.
People are welcome to help improving the vcl (read varnish readme for the different vcl_recv, vcl_fetch, vcl_pipe etc etc)
Code:
# This is a basic VCL configuration file for varnish. See the vcl(7)
# man page for details on VCL syntax and semantics.
#
# Default backend definition. Set this to point to your content
# server.
#
backend default {
.host = "127.0.0.1";
.port = "80";
.first_byte_timeout = 300s;
}
sub vcl_recv {
// Rename the incoming XFF header to work around a Varnish bug.
if (req.http.X-Forwarded-For) {
// Append the client IP
//set req.http.X-Real-Forwarded-For = req.http.X-Forwarded-For ", " regsub(client.ip, ":.*", "");
set req.http.X-Forwarded-For = req.http.X-Forwarded-For;
// unset req.http.X-Forwarded-For;
}
else {
// Simply use the client IP
set req.http.X-Real-Forwarded-For = regsub(client.ip, ":.*", "");
}
# do not cache if user is logged in on vbulletin
if (req.http.cookie && req.http.cookie ~ "auth_user||IDstack") {
return (pass);
}
# do not cache wiki
if ((req.url ~ "show_wiki.php")) {
return (pass);
}
# do not cache register.php
if ((req.url ~ "register.php")) {
return (pass);
}
# do not cache profile.php
if ((req.url ~ "profile.php")) {
return (pass);
}
# do not cache manage
if ((req.url ~ "/manage")) {
return (pass);
}
# do not cache irc
if ((req.url ~ "/irc")) {
return (pass);
}
# do not cache login.php
if ((req.url ~ "login.php")) {
return (pass);
}
# Properly handle different encoding types
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$") {
# No point in compressing these
remove req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
# unkown algorithm
remove req.http.Accept-Encoding;
}
}
# Cache things with these extensions
if (req.url ~ "\.(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$") {
return (lookup);
}
# case backend lags
set req.grace = 10m;
if (req.request == "GET")
{
return (lookup);
}
if (req.url ~ "\.(gif|jpg|jpeg|swf|css|js|flv|mp3|mp4|pdf|ico)$")
{
return (lookup);
}
/* Nirvana */
remove req.http.cookie;
}
sub vcl_deliver {
# From http://varnish-cache.org/wiki/VCLExampleLongerCaching
if (resp.http.magicmarker) {
/* Remove the magic marker */
unset resp.http.magicmarker;
/* By definition we have a fresh object */
set resp.http.age = "0";
}
#add cache hit data
if (obj.hits > 0) {
#if hit add hit count
set resp.http.X-Cache = "HIT";
set resp.http.X-Cache-Hits = obj.hits;
}
else {
set resp.http.X-Cache = "MISS";
}
}
sub vcl_hash {
#vbulletin stuff
if (req.http.cookie && req.http.cookie ~ "auth_user||IDstack") { set req.hash += "auth"; }
set req.hash += req.url;
return (hash);
}
sub vcl_fetch
{
# do not cache if user is logged in on vbulletin
if (req.http.cookie && req.http.cookie ~ "bbuserid") {
return (pass);
}
# unset cookie for killboard
if ((req.url ~ "/killboard")) {
unset beresp.http.set-cookie;
}
set beresp.ttl = 600s;
set beresp.grace = 600s;
if ( beresp.http.Pragma ~ "no-cache"
|| beresp.http.Cache-Control ~ "no-cache"
|| beresp.http.Cache-Control ~ "private") {
return (pass);
}
/**
* Strip vbulletin cookies
*/
// if (beresp.http.Set-Cookie ~ "^bb.*=.*$")
// {
// set beresp.http.X-Cookie = "Cookie removed";
// remove beresp.http.Set-Cookie;
//}
}
sub vcl_hit
{
if (!obj.cacheable)
{
return (pass);
}
set obj.http.X-Cache = "Cache-Hit";
return (deliver);
}
//sub vcl_pipe {
// set bereq.http.X-Forwarded-For = req.http.X-Forwarded-For;
// set bereq.http.X-Forwarded-For = regsub(bereq.http.X-Forwarded-For, "$", ", ");
// set bereq.http.X-Forwarded-For = regsub(bereq.http.X-Forwarded-For, "$", client.ip);
// set bereq.http.Cookie = req.http.X-Оrig-Cookiе;
// set bereq.http.connection = "close";
//}
(View-All)
Members who have read this thread : 13
Linear Mode
