Skip to content

Latest commit

 

History

History
148 lines (119 loc) · 6.06 KB

logging.md

File metadata and controls

148 lines (119 loc) · 6.06 KB
Raw

Logging

FLAT Server writes most of its logs in JSON format to the standard error output (stderr). One line is written for each event. I.e. one line each for a client request, an upstream request or an error.

For comfortable log processing, events are discriminated with the type field that is one of

  • flat_access
  • flat_request
  • flat_alert

The access log can be augmented with custom fields.

All JSON log lines have a requestID field. Its value can be used to correlate log events.

Access Log

Client requests received by FLAT are recorded as type flat_access and have the following format:

{
  "timestamp": "2019-10-15T13:28:26+00:00",
  "type": "flat_access",
  "requestID": "XaXJeWA-@B@3XFNhk42H@QAAAAQ",
  "url": "http://localhost:8080/api/path",
  "path": "/api/path",
  "status": 200,
  "method": "GET",
  "agent": "curl/7.54.0",
  "referrer": "",
  "mime": "application/json",
  "realtime": 0.019845,
  "bytes": 563,
  "requestbytes": 0,
  "flow": "flow.xml",
  "requestmime": "",
  "location": ""
}

Fields

  • timestamp: Start time of the request in ISO 8601 format
  • type: Event type discriminator, fixed string flat_access
  • requestID: Log Correlation ID, accessible in flows as $request/id and $error/requestID
  • url: client request URL
  • path: path, query string and fragment of the client request URL
  • status: HTTP status code of the client response (number) (200, 404, 500, …)
  • method: HTTP method of the client request (GET, POST, …)
  • agent: User-Agent header of the client request
  • referrer: Referer [sic] header of the client request
  • mime: Content-Type header of the client response (text/html, application/json…)
  • realtime: request duration in seconds (number)
  • bytes: uncompressed size of the outgoing response body (number)
  • requestbytes: uncompressed size of the incoming request body (number)
  • flow: filename of the applied flow
  • requestmime: Content-Type header of the client request i.e. MIME type of the incoming request body
  • location: Location header of the client response

Request Log

Outgoing requests to upstream APIs are recorded as type flat_request and have the following format:

{
  "timestamp": "2019-10-15T13:28:26+00:00",
  "type": "flat_request",
  "requestID": "XaXJeWA-@B@3XFNhk42H@QAAAAQ",
  "url": "https://example.com/?foo=bar",
  "method": "get",
  "status": 200,
  "mime": "application/json",
  "bytes": 231,
  "duration": 0.607165,
  "id": "main.0",
  "cacheHit": false,
  "refresh": false,
  "timing": {
    "DNS": "17.391",
    "TCP": "96.475",
    "TTFB": "623.931"
  },
  "curlErrorCode": 28,
  "curlErrorMessage": "Operation timed out after 3000 milliseconds with 0 out of -1 bytes received"
}

Fields

  • timestamp: Start time of the request in ISO 8601 format
  • type: Event type discriminator, fixed string flat_request
  • requestID: Log Correlation ID
  • url: Absolute URL of the upstream request including query-string
  • method: HTTP method of the upstream request (GET, POST, …)
  • status: HTTP status code of the upstream response (number) (0 for failed requests)
  • mime: Content-Type header of the upstream response (text/html, application/json…)
  • bytes: Size of the (possibly compressed) response body in bytes (number) (0 if the body is empty; when read from the internal cache, the original response length is used)
  • duration: Time spent waiting for the response in seconds (number)
  • id: Request/Content identifier as configured with the request's id field or content attribute (suffixed with .N where N is incremented for each followed redirect starting with 0)
  • cacheHit: Indicating whether the response was read from FLAT's HTTP Cache (boolean) (activated with request option use-http-cache)
  • refresh: Indicating whether the response was a 304 response to a request revalidating the local cache (boolean)
  • timing: Contains timing information about request phases
    • DNS: Time to perform name lookup in ms (number)
    • TCP: Time to establish a TCP connection in ms (number) (may be near zero for keep-alive requests)
    • TTFB: Time to first byte in ms since request was sent (number)
  • curlErrorCode: cURL Error Code for failed requests (number, optional) (e.g. 28)
  • curlErrorMessage: Text error message of libcurl for failed requests (optional)

Error Log

Errors that have occurred during processing, such as Validation Errors are recorded as type flat_alert.

These log events should be monitored!

{
  "timestamp": "2019-10-15T13:28:26+00:00",
  "type": "flat_alert",
  "requestID": "XaXJeWA-@B@3XFNhk42H@QAAAAQ",
  "topic": "validation",
  "message": "Response validation failed: Type constraint violated in body: String value found, but an object is required. "
}

Fields

  • timestamp: Start time of the request in ISO 8601 format
  • type: Event type discriminator, fixed string flat_alert
  • requestID: Log Correlation ID
  • topic: Log topic of the alert, corresponds to debugging topics
  • message: An informative error message

Other Log Events

In rare cases plain text log lines may be written to stdout or stderr. For example, this happens during startup.

These lines are usually error log messages of the included Apache Web server that look like this:

[Tue Oct 15 13:11:30.190335 2019] [mpm_worker:notice] [pid 28:tid 139763282708608] - AH00292: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips configured -- resuming normal operations

See also