Pingtower
Docs
Log tail & templates
The actual lines behind an alert, filtered to the shape that matched, and a live tail you can follow. New sources keep their lines unless told not to.
Templates
A template is a message with the values you passed as keys substituted out. The substitution is literal: each key’s value is searched for in the message and replaced with {{ .name }}, longest value first, every occurrence.
| Message | Keys | Template |
|---|---|---|
user bob logged in | {"name":"bob"} | user {{ .name }} logged in |
timeout after 30012ms for o_48113 | {"order":"o_48113","latency_ms":30012} | timeout after {{ .latency_ms }}ms for {{ .order }} |
disk full | none | disk full |
The template’s SHA-256, as lowercase hex, is its id. Ingest returns it as template_id, every kept line carries it as template_sha256, and so does every webhook payload. An alert carries the template itself as template. Two lines with the same template are the same thing to a rule and fold into the same alert.
keys, it stays in the template, and every distinct value opens its own alert. When you see a storm of near-identical alerts, the fix is almost always to move the changing part into a key.Retaining lines
retain_logs is a per-source switch. A source created without it keeps its lines; send "retain_logs": false on create to keep none. On update it must be sent explicitly: an omitted field on PUT is a 400, not a silent false. Sources created before this default changed keep whatever they had.
Three bounds apply to kept lines. Alerting is never affected by any of them.
| Bound | Default | Past it |
|---|---|---|
Daemon -retain-days | 7 | Older lines are pruned. |
Daemon -max-logs-per-source | 10,000 | The source’s oldest lines are pruned. |
Plan’s max_logs | per plan | New lines are not kept until the team is back under it. Nothing is queued. |
Every tail response reports where retention stands, so an empty tail can say why:
"retention": {
"sources_total": 3,
"sources_retaining": 2,
"over_quota": false,
"retain_days": 7,
"max_lines_per_source": 10000
}
sources_total and sources_retaining count the project’s sources, or just the one named by source. retain_days and max_lines_per_source are the daemon’s two bounds, so a client offers only the time range that exists.
GET /v1/projects/{id}/sources adds two fields to each source: oldest_kept_at, the Unix time of its oldest kept line (null when it keeps none), so a source that covers less than the range you asked for can be labelled; and open_secret_findings, its count of open secret findings.
Reading the tail
GET /v1/projects/{id}/logtail
| Parameter | Default | Meaning |
|---|---|---|
limit | 50 | 1 to 200 lines. |
template | none | Only lines with this template id. Pass template_id from ingest or template_sha256 from a webhook. |
source | none | Only lines from this source. Repeat it for several. |
min_level | none | Only lines at or above this level. |
after | none | A cursor from an earlier response. See following. |
wait | 0 | Seconds, up to 60, to hold the request open when there is no line to return yet. |
q | none | A search query, without key terms. |
env, host, run, level | none | Repeat to allow several values. level takes a band name or number: error is 400 to 499. |
curl -s "$PT/v1/projects/checkout/logtail?limit=20&template=$TEMPLATE_ID" \
-H "Authorization: Bearer ptk_…"
{
"messages": ["…", "…"],
"lines": [
{
"id": 41201,
"message": "payment gateway timeout after 30012ms for order o_48113",
"level": 400,
"source": "api",
"template_sha256": "…",
"created_at": 1790776800,
"keys": { "order": "o_48113", "gateway": "stripe", "latency_ms": "30012" },
"tags": ["billing"],
"secret_warning": false,
"env": "prod",
"host": "api-2",
"run": "",
"trace_id": ""
}
],
"cursor": 41201,
"retention": { "…": "…" }
}
lines are oldest first, so a tail reads downward. messages is the older shape, kept for the apps already in the field: the newest limit messages as plain strings, newest first, narrowed by template and nothing else.
A read that takes longer than 10 seconds is cut off with 503 the search took too long; narrow the time range or add filters. Narrow it with template, source or a smaller limit.
In the app, an alert’s detail screen reads this with the alert’s own template already applied. A rule with investigate: true puts it first.
Following the tail
Leave after out to open the screen: you get the newest limit lines and a cursor. Pass that cursor back as after to get only what has arrived since, and add wait to hold the request open until a line lands rather than polling on a timer. A request with nothing new returns as soon as a line arrives, or with an empty lines when wait runs out.
Follow from the cursor once you have one. A read without after always jumps to the newest lines, so a burst bigger than limit between two such reads is skipped. The tail is not a durable stream either: a cursor older than retention has nothing left behind it, and a long enough pause can leave a gap.
q and the repeated filters work here as in log search, with one exception: key terms such as latency_ms:>500 are refused with 422. Search for those instead. A malformed query is 422 with the byte offset of the problem.