Apache Log Parse Pattern
Use the examples shown at the bottom of this page or create your own patterns to process Apache log files.
Use the following patterns to parse Apache log files:
Pattern | Column | Type | Description |
---|---|---|---|
%% | The percent sign. | ||
%a | remoteIpAddress | STRING | Remote IP-address |
%A | localIpAddress | STRING | Local IP-address |
%B | bytesSent | INTEGER | Bytes sent, excluding HTTP headers. |
%b | bytesSent | INTEGER | Size of response in bytes, excluding HTTP headers. In CLF format, i.e. a '-' rather than a '0' when no bytes are sent. |
%{Foobar}C | Foobar | STRING | The contents of cookie Foobar in the request sent to the server. |
%D | microsecondsToServeRequest | INTEGER | The time taken to serve the request, in microseconds. |
%{Foobar}e | envvar | STRING | The contents of the environment variable FOOBAR. |
%f | fileName | STRING | Filename |
%h | remoteHost | STRING | Remote host |
%H | requestProtocol | STRING | Request protocol |
%{Foobar}i | Foobar | STRING | The contents of Foobar header line(s) in the request sent to the server. Changes made by other modules (e.g., mod_headers) affect this pattern. |
%k | keepAlive | INTEGER | Number of keepalive requests handled on this connection. Interesting if KeepAlive is being used, so that, for example, a '1' means the first keepalive request after the initial one, '2' the second, etc...; otherwise this is always 0 (indicating the initial request). |
%l | logName | STRING | Remote logname (from identd, if supplied). This returns a dash unless IdentityCheck is set to 'on'. |
%m | requestMethod | STRING | The request method. |
%{Foobar}n | Foobar | STRING | The contents of note Foobar from another module. |
%{Foobar}o | Foobar | STRING | The contents of Foobar header line(s) in the reply. |
%p | canonicalPort | INTEGER | The canonical port of the server serving the request. |
%{format}p | canonicalPort | INTEGER | The canonical port of the server serving the request or the server's actual port or the client's actual port. Valid formats are |
%P | localPort | INTEGER | The process ID of the child that serviced the request. |
%{format}P | remotePort | INTEGER | The process ID or thread id of the child that serviced the request. Valid formats are |
%q | queryString | STRING | The query string (prepended with a ? if a query string exists, otherwise an empty string). |
%r | request | STRING | The first line of a request. |
%R | response | STRING | The handler generating the response (if any). |
%s | status | INTEGER | Status. For requests that are internally redirected, this is the status of the original request — %...>s for the last. |
%t | requestTime | DATE | Time the request was received (standard English format). |
%{format}t | requestTime | DATE | The time, in the form given by format, which should be in |
%T | requestTime | INTEGER | The time in seconds taken to serve the request. |
%u | remoteUser | STRING | Remote user (from auth; might be bogus if return status (%s) is 401) |
%U | urlPath | STRING | The URL path requested, not including any query string. |
%v | canonicalServerName | STRING | The canonical ServerName of the server serving the request. |
%V | serverName | STRING | The server name according to the UseCanonicalName setting. |
%X | connectionStatus | STRING | Connection status when completed. |
%I | bytesReceived | INTEGER | Bytes received, including request and headers, can't be zero. You need to enable |
%O | bytesSend | INTEGER | Bytes sent, including headers, can't be zero. You need to enable |
Tips
- It is not necessary to put everything in quotation marks. However, there might be some circumstances where quotation marks are necessary, as with
%r
.
With%r
, you would use quotes:- If the raw records contain quotation marks and you want the full string inside those quotation marks.
- When the parse pattern represents a real row of data and each pattern object represents the actual column value.
- To prevent Apache log parsing from using spaces as the delimiter.
- The modifiers "<" and ">" can be used for requests that have been internally redirected to choose whether the original or final request should be consulted.
- By default, the % directives
%s, %U, %T, %D
, and%r
look at the original request while all others look at the final request.
- By default, the % directives
Examples
%h %u "%r" %s %b
\"%{Referer}i\"
- The referrer URL, taken from the request's headers.
\"%{User-Agent}i\"
- The user agent, taken from the request's headers.
Common Log Format (CLF): %h %l %u %t "%r" %>s %b
Common Combined Format: %h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"
Common Log Format with Virtual Host: %v %h %l %u %t "%r" %>s %b