HTTPie: a CLI, cURL-like tool for humans

HTTPie is available on Linux, Mac OS X and Windows. On a Debian or Ubuntu system HTTPie can be installed with apt-get install httpie. For other platforms, see http://httpie.org.

Testing httpie is simple:

http -p Hh http://www.atomictag.com
GET / HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Host: www.atomictag.com
User-Agent: HTTPie/0.9.2
   
HTTP/1.1 200 OK
CF-RAY: 250beb7295742666-FRA
Cache-Control: max-age=0, public
Content-Encoding: gzip
Content-Type: text/html; charset=UTF-8
Date: Sun, 06 Dec 2015 23:55:06 GMT
ETag: W/"7524-526422bb3df6c"
Expires: Sun, 06 Dec 2015 23:55:06 GMT
Last-Modified: Sun, 06 Dec 2015 22:22:06 GMT
Pragma: public
Server: cloudflare-nginx
Set-Cookie: __cfduid=dad70ed346cbd17091806e91a67d56c1f1449446106; expires=Mon, 05-Dec-16 23:55:06 GMT; path=/; domain=.atomictag.com; HttpOnly
Transfer-Encoding: chunked
Vary: Accept-Encoding,Cookie

The -p option to http can be used to control output. Specifically:

-p H will print request headers.
-p h will print response headers.
-p B will print request body.
-p b will print response body.

 

Easy reloading of Varnish VCL


#!/bin/bash
# Reload a varnish config

FILE="/opt/varnish/etc/config/default.vcl"

# Hostname and management port
# (defined in /etc/default/varnish or on startup)
HOSTPORT="localhost:2000"
NOW=`date +'%d/%m/%Y_%H:%M:%S:%N'`

error() {
   echo 1>&2 "Failed to reload $FILE."
   exit 1
}

varnishadm -T $HOSTPORT -S /opt/varnish/etc/secret vcl.load reload_$NOW $FILE || error
varnishadm -T $HOSTPORT -S /opt/varnish/etc/secret vcl.use reload_$NOW || error
echo Current configs:
varnishadm -T $HOSTPORT -S /opt/varnish/etc/secret vcl.list

Varnishadm: how to load a new VCL without restarting

Connect via Varnishadm to Varnish instance:


$ varnishadm -T varnish_server_ip:varnish_server_admin_port -S /opt/varnish/etc/secret
200
-----------------------------
Varnish Cache CLI 1.0
-----------------------------
Linux,3.18.7+,armv6l,-smalloc,-smalloc,-hcritbit
varnish-4.0.3 revision b8c4a34

Type 'help' for command list.
Type 'quit' to close CLI session.
varnish>

Display currently load VCL by using vcl.list command:


varnish>vcl.list
200
active          0 boot
varnish>

Load & compile new VCL by using vcl.load command:


varnish> vcl.load new_vcl "/opt/varnish/etc/config/new_vcl.vcl"
200
VCL compiled.
varnish>

Use the new VCL by using vcl.use command:


varnish> vcl.use new_vcl
200
VCL 'new_vcl' now active
varnish>

Check loaded & active VCL by using vcl.list command:


varnish> vcl.list
200
available       0 boot
active          0 new_vcl
varnish> 

Remove the old VCL by using vcl.discard command:


varnish> vcl.discard boot
200
varnish>

Varnish 4: a simple start.sh script

I have created a very simple start.sh script that helps to run Varnish 4.0 on our servers.

Please note that we also created a very simple logfile to check HIT/MISS requests.


#! /bin/sh
pkill varnishd
echo 'Killed Varnishd daemon'
pkill varnishncsa
echo 'Killed Varnishcsa log daemon'

ulimit -n 10240
ulimit -l 16384

/usr/local/sbin/varnishd \
	-a : \
	-T localhost:2000 \
        -t 120 \
	-S /opt/varnish/etc/secret \
	-n varnish \
        -p thread_pool_min=30 -p thread_pool_max=500 -p thread_pool_timeout=300 \
	-f /opt/varnish/etc/config/default.vcl \
	-s malloc,1G -l 8m,1m,+

echo 'Started Varnishd daemon'
sleep 10
/usr/local/bin/varnishncsa -F '%U%q %{Varnish:hitmiss}x' -w /opt/varnish/logs/requests.log -n varnish&
echo 'Started Varnishcsa log daemon'

Very useful Varnish admin command

Top REQUEST methods
varnishtop -i ReqMethod
Top URLs that MISS the cache
varnishtop -i BereqURL
Top URLS that HIT the cache
varnishtop -i ReqURL
All URLs that HIT the cache in real time
varnishlog -g request -q 'VCL_call eq HIT' | grep reqURL
All URLs that MISS the cache in real time
varnishlog -g request -q 'VCL_call eq MISS' | grep reqURL
All URLs that MISS the cache in real time
varnishlog -i BereqURL
Look at an incoming client request of a specific URL
varnishlog -c -q "ReqUrl eq '/'"
Look at a a backend request of a specific URL
varnishlog -i -q "BereqURL eq '/'"
See requests for one specific Hostname
varnishlog -c -q "ReqHeader eq 'Host: www.mydomain.com'"
See the age of the cache objects for a specific hostname
varnishlog -c -q "ReqHeader eq 'Host: www.mydomain.com'" | grep Age
All 404
varnishlog -b -q "ObjStatus eq 404"