varnishncsa(1)
NAME
varnishncsa - Display Varnish logs in Apache / NCSA combined log format
SYNOPSIS
varnishncsa [-a] [-b] [-c] [-C] [-d] [-D] [-F <format>] [-f <formatfile>] [-g <request|vxid>] [-h] [-L <limit>] [-n <name>] [-N <filename>] [-P <file>] [-q <query>] [-r <filename>] [-t <seconds|off>] [-V] [-w <filename>]
DESCRIPTION
The varnishncsa utility reads varnishd(1) shared memory logs and presents them in the Apache / NCSA "combined" log format. Each log line produced is based on a single Request type transaction gathered from the shared memory log. The Request transaction is then scanned for the relevant parts in order to output one log line. To filter the log lines produced, use the query language to select the applicable transactions. Non-request transactions are ignored. The following options are available: -a When writing output to a file, append to it rather than overwrite it. -b Log backend requests. If -c is not specified, then only backend requests will trigger log lines. -c Log client requests. This is the default. If -b is specified, then -c is needed to also log client requests -C Do all regular expression and string matching caseless. -d Process log records at the head of the log and exit. -D Daemonize. -F <format> Set the output log format string. -f <formatfile> Read output format from a file. Will read a single line from the specified file, and use that line as the format. -g <request|vxid> The grouping of the log records. The default is to group by vxid. -h Print program usage and exit -L <limit> Sets the upper limit of incomplete transactions kept before the oldest transaction is force completed. A warning record is synthesized when this happens. This setting keeps an upper bound on the memory usage of running queries. Defaults to 1000 transactions. -n <name> Specify the name of the varnishd instance to get logs from. If -n is not specified, the host name is used. -N <filename> Specify the filename of a stale VSM instance. When using this option the abandonment checking is disabled. -P <file> Write the process' PID to the specified file. -q <query> Specifies the VSL query to use. -r <filename> Read log in binary file format from this file. The file can be created with varnishlog -w filename. -t <seconds|off> Timeout before returning error on initial VSM connection. If set the VSM connection is retried every 0.5 seconds for this many seconds. If zero the connection is attempted only once and will fail immediately if unsuccessful. If set to "off", the connection will not fail, allowing the utility to start and wait indefinetely for the Varnish instance to appear. Defaults to 5 seconds. -V Print version information and exit. -w <filename> Redirect output to file. The file will be overwritten unless the -a option was specified. If the application receives a SIGHUP in daemon mode the file will be reopened allowing the old one to be rotated away. This option is required when running in daemon mode.
MODES
The default mode of varnishncsa is "client mode". In this mode, the
log will be similar to what a web server would produce in the absence
of varnish. Client mode can be explicitly selected by using -c.
If the -b switch is specified, varnishncsa will operate in "backend
mode". In this mode, requests generated by varnish to the backends
will be logged. Unless -c is also specified, client requests received
by varnish will be ignored.
When running varnishncsa in both backend and client mode, it is
strongly advised to include the format specifier %{Varnish:side}x to
distinguish between backend and client requests.
Client requests that results in a pipe (ie. return(pipe) in vcl), will
not generate logging in backend mode. This is because varnish is not
generating requests, but blindly passes on bytes in both directions.
However, a varnishncsa instance running in normal mode can see this
case by using the formatter %{Varnish:handling}x, which will be 'pipe'.
In backend mode, some of the fields in the format string get different
meanings. Most notably, the byte counting formatters (%b, %I, %O)
considers varnish to be the client.
It is possible to keep two varnishncsa instances running, one in
backend mode, and one in client mode, logging to different files.
FORMAT
Specify the log format to use. If no format is specified the default
log format is used:
%h %l %u %t "%r" %s %b "%{Referer}i" "%{User-agent}i"
Escape sequences \n and \t are supported.
Supported formatters are:
%b In client mode, size of response in bytes, excluding HTTP
headers. In backend mode, the number of bytes received from the
backend, excluding HTTP headers. In CLF format, i.e. a '-'
rather than a 0 when no bytes are sent.
%D In client mode, time taken to serve the request, in
microseconds. In backend mode, time from the request was sent
to the entire body had been received.
%H The request protocol. Defaults to HTTP/1.0 if not known.
%h Remote host. Defaults to '-' if not known. In backend mode this
is the IP of the backend server.
%I In client mode, total bytes received from client. In backend
mode, total bytes sent to the backend.
%{X}i The contents of request header X.
%l Remote logname. Always '-'.
%m Request method. Defaults to '-' if not known.
%{X}o The contents of response header X.
%O In client mode, total bytes sent to client. In backend mode,
total bytes received from the backend.
%q The query string. Defaults to an empty string if not present.
%r The first line of the request. Synthesized from other fields, so
it may not be the request verbatim.
%s Status sent to the client. In backend mode, status received
from the backend.
%t In client mode, time when the request was received, in HTTP
date/time format. In backend mode, time when the request was
sent.
%{X}t In client mode, time when the request was received, in the
format specified by X. In backend mode, time when the request
was sent. The time specification format is the same as for
strftime(3).
%T In client mode, time taken to serve the request, in seconds. In
backend mode, time from the request was sent to the entire body
had been received.
%U The request URL without the query string. Defaults to '-' if not
known.
%u Remote user from auth.
%{X}x Extended variables. Supported variables are:
Varnish:time_firstbyte
Time from when the request processing starts until the
first byte is sent to the client. For backend mode: Time
from the request was sent to the backend to the entire
header had been received.
Varnish:hitmiss
Whether the request was a cache hit or miss. Pipe and
pass are considered misses.
Varnish:handling
How the request was handled, whether it was a cache hit,
miss, pass, pipe or synth.
Varnish:side
Backend or client side. One of two values, 'b' or 'c',
depending on where the request was made. In pure backend
or client mode, this field will be constant.
Varnish:vxid
The VXID of the varnish transaction.
VCL_Log:key
Output value set by std.log("key:value") in VCL.
VSL:tag or VSL:tag[field]
The value of the VSL entry for the given tag. If field
is specified, only the selected part is shown. Defaults
to '-' when the tag is not seen, or when the field is out
of bounds. If a tag appears several times in a single
transaction, only the first occurrence is used.
SIGNALS
SIGHUP Rotate the log file (see -w option)
SIGUSR1
Flush any outstanding transactions
EXAMPLE
Log the second field of the Begin tag, corresponding to the VXID of the
parent transaction:
varnishncsa -F "%{VSL:Begin[2]}x"
SEE ALSO
varnishd(1) varnishlog(1) varnishstat(1) vsl(7)
HISTORY
The varnishncsa utility was developed by Poul-Henning Kamp in cooperation with Verdens Gang AS and Varnish Software AS. This manual page was initially written by Dag-Erling Smrgrav <[email protected]>, and later updated by Martin Blix Grydeland and Pl Hermunn Johansen.
COPYRIGHT
This document is licensed under the same licence as Varnish itself. See
LICENCE for details.
* Copyright (c) 2006 Verdens Gang AS
* Copyright (c) 2006-2016 Varnish Software AS
VARNISHNCSA(1)
Opportunity
Personal Opportunity - Free software gives you access to billions of dollars of software at no cost. Use this software for your business, personal use or to develop a profitable skill. Access to source code provides access to a level of capabilities/information that companies protect though copyrights. Open source is a core component of the Internet and it is available to you. Leverage the billions of dollars in resources and capabilities to build a career, establish a business or change the world. The potential is endless for those who understand the opportunity.
Business Opportunity - Goldman Sachs, IBM and countless large corporations are leveraging open source to reduce costs, develop products and increase their bottom lines. Learn what these companies know about open source and how open source can give you the advantage.
Free Software
Free Software provides computer programs and capabilities at no cost but more importantly, it provides the freedom to run, edit, contribute to, and share the software. The importance of free software is a matter of access, not price. Software at no cost is a benefit but ownership rights to the software and source code is far more significant.
Free Office Software - The Libre Office suite provides top desktop productivity tools for free. This includes, a word processor, spreadsheet, presentation engine, drawing and flowcharting, database and math applications. Libre Office is available for Linux or Windows.
Free Books
The Free Books Library is a collection of thousands of the most popular public domain books in an online readable format. The collection includes great classical literature and more recent works where the U.S. copyright has expired. These books are yours to read and use without restrictions.
Source Code - Want to change a program or know how it works? Open Source provides the source code for its programs so that anyone can use, modify or learn how to write those programs themselves. Visit the GNU source code repositories to download the source.
Education
Study at Harvard, Stanford or MIT - Open edX provides free online courses from Harvard, MIT, Columbia, UC Berkeley and other top Universities. Hundreds of courses for almost all major subjects and course levels. Open edx also offers some paid courses and selected certifications.
Linux Manual Pages - A man or manual page is a form of software documentation found on Linux/Unix operating systems. Topics covered include computer programs (including library and system calls), formal standards and conventions, and even abstract concepts.
