Configure PURGE in VCL 4.0

Loading

In order to configure PURGE add this lines into your VCL_RECV:

vcl 4.0;
import std;

sub vcl_recv {

  # Allow banning  
  if (req.method == "PURGE") {
     # Same ACL check as above:
     if (!client.ip ~ purge) {
        return(synth(405, "This IP is not allowed to send PURGE requests."));
     }
     ban("req.http.host == " + req.http.host + " && req.url == " + req.url);
     # If you got this stage (and didn't error out above), purge the cached result
     return (synth(200, "Purged"));
  }

}

To test it:

$ curl -v -k -H "host: www.mydomain.com" -X PURGE http://varnish_server_ip/

Remember to add ACL into your files

Force cache and leverage browser caching for non cachable contents

Loading

Add this code in the VCL_BACKEND_RESPONSE:


sub vcl_backend_response {

  # client browser and server cache
  # Force cache: remove expires, Cache-control & Pragma header coming from the backend
  if (beresp.http.Cache-Control ~ "(no-cache|private)" || beresp.http.Pragma ~ "no-cache")  {
     unset beresp.http.Expires;
     unset beresp.http.Cache-Control;
     unset beresp.http.Pragma;
              
     # Marker for vcl_deliver to reset Age: /
     set beresp.http.magicmarker = "1";
     
     # Leveraging browser, cache set the clients TTL on this object /
     set beresp.http.Cache-Control = "public, max-age=2592000";
	 
     # cache set the clients TTL on this object /        
     set beresp.ttl = 30d;  

     # Allow stale content, in case the backend goes down.
     # make Varnish keep all objects for 6 hours beyond their TTL
     set beresp.grace = 6h;
     return (deliver);
  }

}

Also add this in the VCL_DELIVER:


sub vcl_deliver {
# Called before a cached object is delivered to the client.
	
  if (resp.http.magicmarker) {
	unset resp.http.magicmarker;
	# By definition we have a fresh object 
	set resp.http.Age = "0";
	}
    
  if (obj.hits > 0) { # Add debug header to see if it's a HIT/MISS and the number of hits, disable when not needed
    set resp.http.X-Cache = "HIT";
  } else {
    set resp.http.X-Cache = "MISS";
  }
  # Please note that obj.hits behaviour changed in 4.0, now it counts per objecthead, not per object
  # and obj.hits may not be reset in some cases where bans are in use. See bug 1492 for details.
  # So take hits with a grain of salt
  set resp.http.X-Cache-Hits = obj.hits;
  
  # Set Varnish server name
  set resp.http.X-Served-By = server.hostname;

  # Remove some headers: PHP version
  unset resp.http.X-Powered-By;

  # Remove some headers: Apache version & OS
  unset resp.http.Server;
  unset resp.http.X-Varnish;
  unset resp.http.Via;
  unset resp.http.Link;
  unset resp.http.X-Generator;

  return (deliver);
}

Retrieve WordPress logged in username, store it in a custom http x-header and clean cookie

Loading


vcl 4.0;
import std;

sub vcl_recv {

  # Retrieve WordPress logged in username, store it in a custom http x-header and clean cookie
  if (req.http.Cookie ~ "wordpress_logged_in")  {      
     set req.http.X-UserID = regsuball(req.http.Cookie, "^.*wordpress_logged_in_[^=]+[^;]=([^;]*);*.*$", "\1");
     set req.http.X-UserID = regsuball(req.http.X-UserID, "%7C.*", "");
     # Remove wordpress_logged_in and wordpress_ cookies
     set req.http.Cookie = regsuball(req.http.Cookie, "wordpress_logged_in_[^=]+[^;]+(; )?", "");
  } 

}

Customizing 404 error page

Loading

In Varnish 3, in order to customize the error page you should customize the subroutine VCL_ERROR:

Things changes in Varnish 4, errors that occur in the backend server subroutine VCL_BACKEND_ERROR are processed used the SYNTH command in the VCL and defined by creating a specific synthetic object in the subroutine VCL_SYNTH.

Following a VCL example that generate a 404 managed by a synthetic object by invoking the following URL http://www.myservname.com:port/404 When you access to the URL, Synth (status_code, reason).


vcl 4.0;
import std;

sub vcl_recv {
    if (req.url ~ "^/404") {
        return (synth(999, "Generate a 404 error explicitly"));
    }
}
 
sub vcl_backend_response {
}
 
sub vcl_deliver {
}
 
sub vcl_backend_error {
    set beresp.http.Content-Type = "text/html; charset=utf-8";
    synthetic( {"errors due to backend fetch"} );
    return (deliver);
}
 
sub vcl_synth {
    if (resp.status == 999) {
        set resp.status = 404;
        set resp.http.Content-Type = "text/plain; charset=utf-8";
        synthetic(std.fileread("/tmp/vcl_404_error.html"));
        return (deliver);
    }
    return (deliver);
}

Remember to create the error page in the correct path:


$ cat /tmp//tmp/vcl_404_error.html
errors due to vcl

To test the response do the following:


$ curl -D - http://www.varnishservername.com:port/
HTTP/1.1 503 Backend fetch failed
Date: Sun, 08 Feb 2015 09:26:35 GMT
Server: Varnish
Content-Type: text/html; charset=utf-8
X-Varnish: 2
Age: 0
Via: 1.1 varnish-v4
Content-Length: 27
Connection: keep-alive
 
errors due to backend fetch

$ curl -D - http://www.varnishservername.com:port//404/foo
HTTP/1.1 404 Not Found
Date: Sun, 08 Feb 2015 09:26:39 GMT
Server: Varnish
X-Varnish: 5
Content-Type: text/plain; charset=utf-8
Content-Length: 18
Connection: keep-alive
 
errors due to vcl

Varnish 4: only the most “wow” features and “must-know” differences from Varnish 3

Loading

This is not a full Varnish 4 presentation .Only the most “wow” features and “must-know” differences from Varnish 3 and some random cool stuff.

Varnish 4 vs Varnish 3 presentation

by Emanuelis, 2014-09-30

Normalize trailing slash for directory and manage 301 redirection internally

Loading

Add the following code into VCL_RECV:


vcl 4.0;
import std;

sub vcl_recv { 

  # Normalize trailing slash for directory & manage 301 redirection internally
  if (req.http.Host !~ "^www\." || 
              (req.url !~ {"(?x)
               (?:/$)         # last character isn't a slash
               |              # or
               (?:/\?)        # query string isn't immediately preceded by a slash
               "} 
               &&
               req.url ~ {"(?x)
               (?:/[^./]+$)   # last path segment doesn't contain a . no query string
               |              # or
               (?:/[^.?]+\?)  # last path segment doesn't contain a . with a query string
               "}
              )
      ) 
    {  
        return (synth(720,"Moved Permanently"));
    }
}

Add the following code into VCL_SYNTH:


sub vcl_synth {
    
  if (resp.status == 720) {
    # We use this special error status 720 to force redirects with 301 (permanent) redirects
    # To use this, call the following from anywhere in vcl_recv: return (synth(720, "http://host/new.html"));
    set resp.http.Location = resp.reason;
    set resp.status = 301;
    
    set resp.http.Location = "http://";
    set resp.http.Host = req.http.Host;
    
    if (resp.http.Host !~ "^www\.") { 
        set resp.http.Host = "www." + resp.http.Host;
    }
    
    set resp.http.Location = resp.http.Location + resp.http.Host;

    if (req.url ~ "(?:/[^./]+$)|(?:/[^.?]+\?)") {
        # no . in last path segment before optional query string
        if (req.url !~ "/$" && req.url !~ "\?") {
            # no trailing slash and no query string
            set resp.http.Location = resp.http.Location + req.url + "/";
        } elseif (req.url ~ "[^/]\?") {
           # no trailing slash and with query string, preserve it
           set resp.http.Location = resp.http.Location + regsub(req.url, "([^?]+)\?.*", "\1") + "/" + regsub(req.url, "[^?]+(\?.*)", "\1");
        } elseif (resp.http.Host != req.http.Host) {
           # trailing slash rule met, handle missing www. scenario
           set resp.http.Location = resp.http.Location + req.url;
        }
      } elseif (resp.http.Host != req.http.Host) {
            # last path segment contains a . so handle missing www. scenario
            set resp.http.Location = resp.http.Location + req.url;
        }
        
  return (deliver);
    
  } elseif (resp.status == 721) {
    # And we use error status 721 to force redirects with a 302 (temporary) redirect
    # To use this, call the following from anywhere in vcl_recv: return (synth(720, "http://host/new.html"));
    set resp.http.Location = resp.reason;
    set resp.status = 302;
    return (deliver);
  }

  return (deliver);
}

Configure BAN in VCL 4.0

Loading

To configure PURGE add this lines into your VCL_RECV:

vcl 4.0;
import std;

sub vcl_recv {

  # Allow banning  
  if (req.method == "BAN") {
     # Same ACL check as above:
     if (!client.ip ~ purge) {
        return(synth(405, "This IP is not allowed to send BAN requests."));
     }
	 ban("req.http.host == " + req.http.host + " && req.url == " + req.url);
     # Throw a synthetic page so the request won't go to the backend.
     return(synth(200, "Ban added"));
  }

}

To test it:

$ curl -v -k -H "host: www.mydomain.com" -X BAN http://varnish_server_ip/

Very useful Varnish admin command

Loading

Top REQUEST methods
varnishtop -i ReqMethod
Top URLs that MISS the cache
varnishtop -i BereqURL
Top URLS that HIT the cache
varnishtop -i ReqURL
All URLs that HIT the cache in real time
varnishlog -g request -q 'VCL_call eq HIT' | grep reqURL
All URLs that MISS the cache in real time
varnishlog -g request -q 'VCL_call eq MISS' | grep reqURL
All URLs that MISS the cache in real time
varnishlog -i BereqURL
Look at an incoming client request of a specific URL
varnishlog -c -q "ReqUrl eq '/'"
Look at a a backend request of a specific URL
varnishlog -i -q "BereqURL eq '/'"
See requests for one specific Hostname
varnishlog -c -q "ReqHeader eq 'Host: www.mydomain.com'"
See the age of the cache objects for a specific hostname
varnishlog -c -q "ReqHeader eq 'Host: www.mydomain.com'" | grep Age
All 404
varnishlog -b -q "ObjStatus eq 404"