Varnish config – default.vcl

I’ve received some requests for a copy of my Varnish config. I’ll post it here with the following caveats:

  1. This is very much a work in progress. There may be some show-stopping bugs that could really mess up your users. Your milage may vary. Use with caution. I make no guarantees or warranties. Do not use this as-is on production site – you will need to modify it to work properly on your own site.
  2. Much of the configuration is specific to my infrastructure. Huge chunks of it are assuredly not appropriate for your own site. Do not use this as-is on production site – you will need to modify it to work properly on your own site.
  3. Do not use this as-is on production site – you will need to modify it to work properly on your own site. (Are you sensing a theme here?)

#This is a basic VCL configuration file for varnish.  See the vcl(7)
#man page for details on VCL syntax and semantics.
#
#Default backend definition.  Set this to point to your content
#server.
#
backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .connect_timeout = 600s;
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
    .max_connections = 250;
}

backend umpqua {
    .host = "10.0.0.40";
    .port = "80";
    .connect_timeout = 600s;
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
    .max_connections = 250;
}

backend coos {
    .host = "10.0.0.33";
    .port = "80";
    .connect_timeout = 600s;
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
    .max_connections = 250;
}

#Below is a commented-out copy of the default VCL logic.  If you
#redefine any of these subroutines, the built-in logic will be
#appended to your code.
#
sub vcl_recv {
    if (req.http.host ~ "umpqua") {
        # Route all umpqua traffic to umpqua
        set req.backend = umpqua;
    } else if (req.http.host ~ "oregonk-12.net$") {
        # route oregonk-12.net traffic to coos
        set req.backend = coos;
    } else if (req.http.host ~ "k12partners.org$") {
        # route k12partners.org to coos
        set req.backend = coos;
    } else if (req.http.host ~ "oregoneducationdata.org$") {
        # route oregoneducationdata.org to coos
        set req.backend = coos;
    } else {
        # Use the default backend for all other requests
        set req.backend = default;
    }

    # Allow a grace period for offering "stale" data in case backend lags
    #set req.grace = 60s;
    set req.grace = 5m;

    remove req.http.X-Forwarded-For;
    set req.http.X-Forwarded-For = client.ip;

    # Properly handle different encoding types
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
            # No point in compressing these
            remove req.http.Accept-Encoding;
        } elsif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elsif (req.http.Accept-Encoding ~ "deflate") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            # unkown algorithm
            remove req.http.Accept-Encoding;
        }
    }

    # Force lookup if the request is a no-cache request from the client
    if (req.http.Cache-Control ~ "no-cache") {
        return (pass);
    }

    ## Default request checks
    if (req.request != "GET" &&
        req.request != "HEAD" &&
        req.request != "PUT" &&
        req.request != "POST" &&
        req.request != "TRACE" &&
        req.request != "OPTIONS" &&
        req.request != "DELETE") {
            # Non-RFC2616 or CONNECT which is weird.
            return (pipe);
    }
    if (req.request != "GET" && req.request != "HEAD") {
        # We only deal with GET and HEAD by default
        return (pass);
    }
    if (req.request != "GET" && req.request != "HEAD") {
        # We only deal with GET and HEAD by default
        return (pass);
    }

    ## Modified from default to allow caching if cookies are set, but not http auth
    if (req.http.Authorization) {
        /* Not cacheable by default */
        return (pass);
    }

    # ORVSD tweaks
    ## Remove has_js and Google Analytics cookies.
    set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js)=[^;]*", "");
    ## Remove a ";" prefix, if present.
    set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
    ## Remove empty cookies.
    if (req.http.Cookie ~ "^\s*$") {
        unset req.http.Cookie;
    }
    ## Catch Drupal theme files  - THIS BREAKS UPDATE.PHP DO NOT USE
    #if (req.url ~ "^/sites/") {
    #    unset req.http.Cookie;
    #}
    # Catch Drupal misc files (like drupal.js and jquery.js)
    #if (req.url ~ "^/misc/") {
    #    unset req.http.Cookie;
    #}
    # Salkeiz-specific - salkeiz still uses some static files out of /files, cache them
    if (req.url ~ "^/files/salkeiz.*") {
        unset req.http.Cookie;
    }
    # Salkeiz-specific - enable caching of theme files (can't enable globally due to update.php problem above)
    if (req.url ~ "^/sites/www.sal.*") {
        unset req.http.Cookie;
    }
    if (req.url ~ "^/sites/salkeiz.*") {
        unset req.http.Cookie;
    }
    # Drupal js/css doesn't need cookies, cache them
    if (req.url ~ "^/modules/.*\.(js|css)\?") {
        unset req.http.Cookie;
    }
    ## Moodle themes - disabled, seems to cause random problems
    #if (req.url ~ "^/(theme|pix)/") {
    #    unset req.http.Cookie;
    #}

    ## Pass cron jobs and server-status
    if (req.url ~ "cron.php") {
       return (pass);
    }
    if (req.url ~ ".*/server-status$") {
       return (pass);
    }

    ## Don't cache install
    if (req.url ~ "install.php") {
        return (pass);
    }

    ## Moodle doesn't like to be cached, passing
    if (req.http.Cookie ~ "(MoodleSession|MoodleSessionTest)") {
        return (pass);
    }
    if (req.url ~ "/moodle") {
        return (pass);
    }
    if (req.url ~ "file.php") {
        return (pass);
    }

    ## Don't cache Drupal logged-in user sessions
    if (req.http.Cookie ~ "(VARNISH|DRUPAL_UID)") {
        return (pass);
    }

    return (lookup);
}

# More ORVSD tweaks
# Per-session cache
sub vcl_hash { if (req.http.Cookie) { set req.hash += req.http.Cookie; } }

sub vcl_fetch {
    # These status codes should always pass through and never cache.
    if (obj.status == 404 || obj.status == 503 || obj.status == 500) {
        set obj.http.X-Cacheable = "NO: obj.status";
        set obj.http.X-Cacheable-status = obj.status;
        return (pass);
    }

    # Grace to allow varnish to serve content if backend is lagged
    set obj.grace = 5m;

    ## ORVSD tweaks - currently disabled due to odd random loss of session
    #if (req.url ~ "^/(sites|misc|modules)/") {
    #    unset obj.http.Set-Cookie;
    #    set obj.cacheable = true;
    #}
    #if (req.url ~ "^/modules/.*\.(js|css)\?") {
    #    unset req.http.Set-Cookie;
    #    set obj.cacheable = true;
    #}
    if (req.url ~ "^/files/salkeiz.*$") {
        unset req.http.Set-Cookie;
        set obj.cacheable = true;
    }
    if (req.url ~ "^/sites/www.salkeiz.*") {
        unset req.http.Set-Cookie;
        set obj.cacheable = true;
    }

    if (!obj.cacheable) {
        set obj.http.X-Cacheable = "NO: !obj.cacheable";
        return (pass);
    }
    #if (obj.http.Set-Cookie) {
    #    set obj.http.X-Cacheable = "NO: !obj.Set-Cookie";
    #    return (pass);
    #}
    # No expires header set? Set one in the past to make sure it's not serving stale content.
    #if (!obj.http.expires) {
    #    set obj.http.expires = "Mon, 1 Jan 2001 23:59:59 GMT";
    #}
    #set obj.prefetch = -30s;
    #
    # All tests passed, therefore item is cacheable
    set obj.http.X-Cacheable = "YES";
    return (deliver);
}
#
sub vcl_deliver {
#    return (deliver);
   #add cache hit data
   if (obj.hits > 0) {
     #if hit add hit count
     set resp.http.X-Cache = "HIT";
     set resp.http.X-Cache-Hits = obj.hits;
   } else {
     set resp.http.X-Cache = "MISS";
   }
}

sub vcl_error {
    if (obj.status == 503 && req.restarts < 5) {
        set obj.http.X-Restarts = req.restarts;
        restart;
    }
}

5 Responses to “Varnish config – default.vcl”


  • Why are not caching of anything for logged in users? Is it just because you don’t need to worry about that optimization, thus haven’t bothered with more fine-grained tuning, or is it some fundamental problem you’ve run into?

    I ask because I’ve been running a copy of varnish locally with my Drupal site with no problems so far (it’s been 4-5 days), even when I’m logged in (and yes, I’ve checked varnishtop and it seems to be caching everything except the HTML for my logged in user).

    Thanks for this series of articles on Varnish – they’ve really helped me get things setup quite a bit.

  • I’m not worrying as much about caching for logged-in users for several reasons:

    1) The vast majority of hits on the sites I’m concentrating on come from anonymous users, so I’m focusing on optimizing the cache for those anonymous user sessions. The level of traffic we get from authenticated users is low enough that it’s not a concern for us.

    2) I don’t want to serve stale content to a logged-in user. If they post something to the site, I don’t want them to re-post because they’re looking at a cached version of the page and think their post didn’t go through. So I’m being pretty conservative when it comes to authenticated users. I’ve looked at things like authcache for caching of authenticated users, but it’s a lower priority than minimizing the load from anonymous users.

    Note that my config *is* caching a lot of the static content – especially the JS and CSS – even on authenticated sessions. When you look at the HTTP headers on an authenticated user session you’ll see that most of the requests for theme files, etc. do not have a DRUPAL_UID cookie set, so they’re served from Varnish.

  • Hello

    I am planning to configure Varnish on my server to hide me server ip
    consider my server ip is 66.66.66.66 and I want to configure varnish in such a way, when anyone tries to locate my server ip using any dns checker tool, it should show 99.99.99.99
    to users.

    can you tell me how can i do this. I tried alot and alot but no result.

    help needed.

    Thanks
    Yatin

  • Tried this setting on my server but its not working. I have mod proxy and apache 2.4 installed on my machine. does it create any conflict ?

  • in varnish reverse proxy how I can increase the compression of deflate and gzip to 9 ?
    like .htaccess in apache ?

Leave a Reply