synthetic

A synthetic check is a short API journey: log in, take the token from the response, call an endpoint with it, and assert on each step along the way. Every step is an httpcheck request with that check’s assertions and authentication. A failure in any step stops the run and names that step.

Parameters

The check’s timeout_seconds is the deadline for the whole run, every step included. It defaults to 30 and can be at most 60. The agent stops a run at 60 seconds whatever the check says.

Steps

A step takes httpcheck’s request, assertion and authentication parameters, plus three of its own:

Variables

A step’s url, header values and body can use {{name}}. The agent fills it in just before that step sends. A name can come from one of three places:

• variables: plain values written in the check.
• An earlier step’s extract: a value read from that step’s response. A step can use only names that an earlier step extracts.
• A secret variable: a value in the check’s sealed env. See Secrets.

An extraction has a var (the name to set), a from (jsonpath or header) and a property (the JSONPath into a JSON body, or the header name). Add secret: true when the value is itself a secret, such as a session token. It is then shown as [redacted] everywhere a secret variable is: in the detail, the labels, errors and logs. An extraction from a step that uses a secret variable, its auth, or another such extraction is treated as secret too, with or without secret: true, so a session read at login is redacted like the password that got it. Each var is set only once in a check, and it can’t share a name with a plain or secret variable.

In url and oauth_token_url, {{name}} may appear only in the path, query or fragment. The scheme and host must be literal, so where every request goes is known before the run. The API refuses anything else, naming the step from 0, so steps.1 below is the second step. A local agent.yaml reports the same refusal as param steps[1].url: ….

with.steps.1.url: the scheme and host must be literal; {{var}} may appear only in the path, query or fragment

A {{...}} that isn’t a valid name is sent exactly as written.

A step is also bounded once its variables are filled in: its whole url, scheme and host included, may be at most 8 KiB, its header values together at most 16 KiB, and its body at most 256 KiB. A step past one of these fails without sending its request: the step is too large once its variables are filled in. The detail never shows the values. A plain variable or an extracted value can be up to 4 KiB, so a template that uses one many times can reach a limit even when it is short as written.

with:
  variables:
    tenant: acme
  steps:
    - name: login
      url: https://api.acme.example/v1/login
      method: POST
      headers: {Content-Type: application/json}
      body: '{"user":"probe","password":"{{PROBE_PASSWORD}}"}'
      assertions:
        - {target: status, operator: equals, value: "200"}
      extract:
        - {var: token, from: jsonpath, property: $.token, secret: true}
    - name: orders
      url: https://api.acme.example/v1/{{tenant}}/orders
      headers: {Authorization: "Bearer {{token}}"}
      assertions:
        - {target: jsonpath, property: $.count, operator: greater_than, value: "0"}
        - {target: response_time_ms, operator: less_than, value: "800"}

Secrets

Secret variables live in the check’s env, never in with. Upper-case names such as PROBE_PASSWORD can be used as {{PROBE_PASSWORD}}, or named by a step’s auth_secret. You set them from the portal or the API. They are sealed at rest and write-only: reading the check back shows which secrets are set, never their values. If a response echoes a secret back, the result shows [redacted] in its place.

A secret is tied to exactly where it is sent. That means the full URL and method of the steps that use it, and for mtls the client_cert it was sealed with. Under oauth_client_credentials the step’s auth_secret is tied to both the token URL and the step’s own URL and method, because the token it mints is sent to the step’s URL. An update that changes any of these, even only a path or a query parameter, without supplying env again, is refused with a 422: env NAME would reach a destination it was not sealed for; re-supply env (or send {} to clear it). A step with no method counts as GET. The URL is read with the check’s plain variables filled in, so changing a variable that a secret step’s URL or token URL uses is the same change and is refused the same way; a variable no such URL uses can change freely. A {{name}} taken from an extraction stays as written, but the step it is extracted from is tied too: when a secret step’s URL or token URL uses an extraction, the secret is also tied to the step that extraction comes from, and, if that step’s URL uses an extraction of its own, to that one’s step as well, back along the chain. Such a step is tied by these fields of the request it reads its value from: its URL and method, that extraction’s from, property and secret, its headers and body, its auth, auth_username, auth_secret and client_cert, its oauth_token_url, oauth_client_id, oauth_client_auth, oauth_scopes and oauth_params, and its follow_redirects. Its headers, body and oauth_token_url are read with the plain variables filled in, so changing a variable one of them uses is a change too. So changing any of these, even only a path, a JSONPath or one header, without supplying env again is refused the same way. Writing a default out, such as follow_redirects: true or oauth_client_auth: basic, is not a change. Its name, expect_status, body_matches, max_ttfb and assertions, the check’s timeout_seconds, and its extractions of names that no secret step’s URL or token URL uses can change freely. An extraction a secret step uses only in its headers or body does not tie the step it is extracted from. Adding a region or source to sources without supplying env again is refused too: env: sources now include <names> the env was not sealed for; re-supply env (or send {} to clear it). Narrowing where a secret goes, such as removing a step that uses it or a source, is fine. Minting a new agent token for a source stops the check running from that source until you supply env again: see Secrets.

A value extracted with a secret is tied the same way. That is an extraction marked secret: true, or one taken from a step that uses a secret variable or another such extraction, so a session read with a token that was read with a password is tied too. It can go back to the exact method and URL it was read from, and to those it was already sent to. Where the URL or token URL of the step it is read from, or of a step it is sent from, uses an extraction, it is tied to that extraction’s step the same way. An update that would send it anywhere else, without supplying env again, is refused with a 422: extraction NAME carries a secret and would reach a destination it was not sealed for; re-supply env (or send {} to clear it). Supply env again, or send {} to clear it. An extraction not marked secret: true, from a step that uses neither a secret variable nor a value extracted with one, is not tied anywhere.

As with httpcheck, a check that needs a secret works only when Pingtower assigns it to the agent, not as a check in a local agent.yaml.

Running it

Steps run one after another, all within one deadline. Every request dials through the same network guard as httpcheck: the step’s URL, any redirect, and any OAuth token fetch. If any of them resolves to a private or otherwise blocked address, the run stops as a check error tagged status:error, not as down. The error names the step, never the address:

synthetic could not run: check checkout-journey (step 2: refusing to dial a blocked address)

Each step’s host is also checked when you save the check. A step that carries a secret, through a {{SECRET}} reference, its auth, or a value extracted with a secret, does not follow a redirect to another origin: a different scheme, host or port. That step fails with redirected to another origin while carrying a secret.

The first step that fails stops the run, and the check is down. No later step runs. The detail names the step by its number, and by its name when it has one, followed by the reason it failed. An unnamed step reads step 2 failed: …:

step 2 "orders" failed: assertion 1 (jsonpath $.count greater_than "0") failed: got "0"

Other reasons include unexpected status 500, request timed out, connection refused, host could not be resolved, TLS certificate rejected and OAuth token request failed: …. An extraction that finds nothing fails its step too: extract token: JSONPath selected nothing, extract token: header not present, or extract token: JSONPath selected 3 values when it matches more than one value. So does a variable the step uses that has no value or contains a line break: variable token has no value, variable token contains a line break or NUL, and so does a step past its size limit once filled in: the step is too large once its variables are filled in.

A step over its max_ttfb makes the run degraded: step 1 "login": first byte after 1.2s, budget 800ms, or step 1: first byte after … for an unnamed step. A run where every step passes reads 3 steps passed in 412ms.

There is no branching or looping between steps, and there are no browser steps.

Measurements

A failed run also carries the label failed_step, and failed_assertion when the step failed on an assertion rather than on its status, the connection or an extraction. Both hold numbers as strings, counted from 1, and failed_assertion counts within the failed step. A step that never ran has no step_N_ keys. The portal and the apps show the last run step by step, with skipped steps marked.

The check runs on your own agents and on Pingtower’s probe fleet. It needs addons agent 0.4.0 or later. An older agent reports it as an unknown addon, and the tower warns when you save a synthetic check that names a source whose agent is older than 0.4.0, or reports no version (every release before 0.4.0). The save still succeeds, the portal shows the warning, and the sources API reports each source’s agent_version.

Rules for it

{"name":"journey-down","match":{"tags":["addon:synthetic"],"min_level":400},
 "fire_after":{"count":2,"window_minutes":5,"distinct_sources":true}}

{"name":"journey-slow","match":{"tags":["addon:synthetic"]},
 "conditions":[{"key":"total_ms","op":"gte","value":5000}]}