# Hosted checks Source: https://www.pingtower.com/docs/service/checks/ On pingtower.com, Pingtower runs your checks from its own servers in multiple regions of the world. A check names an addon, its parameters, a cadence and where it runs from; its results are ordinary events that your rules alert on. A check is a source of events, not a second alerting path. It names an [addon](/docs/addons/) and the parameters to run it with, never a command. Each run's result arrives as an ordinary event carrying the tags `addon:`, `check:` and `status:`. Every alerting decision about it is a [rule's](/docs/service/rules/).
check→runs in each region→result event→rule→alert
On pingtower.com, the regions a check runs from are Pingtower's own servers. A self-hosted tower has no fleet. There, a check names your own sources, and the [addons agent](/docs/addons/) on each one runs it. The agent holds that source's agent credential (`ptp_`, minted with POST `/v1/projects/{id}/sources/{name}/agent-token`), asks the tower what the source should run, and posts the results back. The same check, rule and quorum work in both places. A hosted project can mix the two: a check can name regions and your own sources together. ## What runs where | Kind | Addon | On Pingtower's regions | On your own agent | | --- | --- | :---: | :---: | | HTTP, with assertions and authentication | [`httpcheck`](/docs/addons/httpcheck/) | yes | yes | | Multi-step API | [`synthetic`](/docs/addons/synthetic/) | yes | yes | | Browser | [`browsercheck`](/docs/service/browser-checks/) | yes | see [Browser checks](/docs/service/browser-checks/#where-it-runs) | | TLS certificate | [`tlscert`](/docs/addons/tlscert/) | yes | yes | | DNS | [`dnscheck`](/docs/addons/dnscheck/) | yes | yes | | Ping | [`icmpping`](/docs/addons/icmpping/) | yes | yes | | TCP | [`tcpcheck`](/docs/addons/tcpcheck/) | yes | yes | | gRPC health | [`grpccheck`](/docs/addons/grpccheck/) | yes | yes | | WebSocket | [`websocket`](/docs/addons/websocket/) | yes | yes | | Domain expiry | [`domainexpiry`](/docs/addons/domainexpiry/) | one region | one agent | | UDP | [`udpcheck`](/docs/addons/udpcheck/) | no | yes | | SMTP, IMAP, POP3 | [`smtpcheck`, `imapcheck`, `popcheck`](/docs/addons/mailcheck/) | no | yes | UDP and mail checks are agent-only. Pingtower's servers share their addresses across every customer, and UDP payloads and mail-server probes from shared addresses are what get an address range listed. A check of either kind that names a Pingtower region is refused with a `422` on `sources`, under the rule `agent_only`: ```text udpcheck runs only on your own agent; source is a fleet region ``` The other addons, such as `hostmetrics` and the privileged ones, are not on the fleet either. A check naming a region for one of them is refused with `addon is not available on the fleet; run it from your own agent`. Three addons have tighter rules on Pingtower's servers than on your own agent: - `tcpcheck` may name only well-known service ports: 21, 22, 25, 53, 80, 110, 143, 443, 465, 587, 993, 995, 3306, 5432, 6379, 8080 and 8443. - `grpccheck` may name only ports 80, 443, 8080, 8443, 9090 and 50051. - `dnscheck` must leave `resolver` unset or name 1.1.1.1, 1.0.0.1, 8.8.8.8, 8.8.4.4 or 9.9.9.9. A target that resolves to a private, loopback, link-local or cloud-metadata address is refused when you save the check. The refusal says the target was refused, never what it resolved to. ## Create one Create a check in the [web app](https://app.pingtower.com) or the iOS app, each with a form per kind. The iOS app creates and edits every kind above except browser checks, which it shows read-only. Through the API, POST `/v1/projects/{id}/checks` creates one, and GET PUT DELETE `/v1/projects/{id}/checks/{name}` read, replace and remove it. A check's name is its identity and can't be changed, because it is what the event's `check` key carries and what a rule groups on. ```sh curl -s https://api.pingtower.com/v1/projects/web/checks \ -H "Authorization: Bearer ptk_…" \ -d '{ "name": "api-health", "addon": "httpcheck", "with": {"url": "https://api.acme.example/healthz"}, "interval_seconds": 60, "sources": ["", "", ""], "alert_if": {} }' ``` | Field | Default | Meaning | | --- | --- | --- | | `name`REQUIRED | | Up to 200 characters. | | `addon`REQUIRED | | An addon from the table above. | | `with` | | The addon's parameters, up to 32. An unknown parameter or one of the wrong type is refused, naming the field. | | `interval_seconds`REQUIRED | | How often it runs, up to 86,400 (a day). See [Intervals](#intervals). Domain expiry defaults to once a day. | | `timeout_seconds` | `10` | One run's limit. It must be below the interval and is at most 3,600. A multi-step check defaults to 30, a browser check to 60, a domain expiry check to 30. | | `sources`REQUIRED | | Where it runs: region names, your own sources, or both. Up to 64, no repeats. Domain expiry takes at most one, or none. | | `enabled` | `true` | | | `alert_if` | | Write a rule that pages when the check is down. See [Alerting](#alerting). | | `env` | | Sealed secrets for the kinds that take them. See [Secrets](#secrets). | GET `/v1/regions` lists the regions your plan can name, and `/v1/regions?pool=browser` the regions with browser runners. Naming a region on a check adds that region as a source in the project, along with a `-quiet` rule (see [When a region goes quiet](#when-a-region-goes-quiet)). A region's name is reserved: you can't create a source under it by hand. A check write answers with the stored check, the name of the rule `alert_if` wrote in `alert_rule`, and any `warnings`. ## Intervals | | Free | Pro | Enterprise | | --- | ---: | ---: | ---: | | Shortest interval | 300 s | 15 s | 15 s | | Checks per tenant | 5 | 100 | no cap | A browser check runs at most every 300 seconds on every plan. The plan also caps how many regions one check may name, and holds every check to a capacity budget in runs per minute: each check costs 60 ÷ interval × weight, once per region it names. Most kinds weigh 1. An `httpcheck` with a body weighs 2, an `icmpping` weighs its `count` (3 by default), a multi-step check one per step plus one per OAuth step, and a browser check 20. A write past any of these limits is refused with `429`, naming the limit (`max_checks`, `max_check_sources`, `min_check_interval_seconds` or `slot_budget`) and both numbers: ```text min_check_interval_seconds: interval_seconds 60 is below this plan's floor of 300 seconds ``` On Free, the budget fits one browser check only at an interval of 400 seconds or more from one region, and only while your other checks leave room for it. ## Regions and quorum A check that runs from several regions gets several independent answers, and one region's own network failing is not your service down. `alert_if.down_from` is how many regions must report the check down inside one window before it pages. Leave it out and it is a majority of the check's sources: | Sources | Default `down_from` | | ---: | ---: | | 1 | 1 | | 2 | 2 | | 3 | 2 | | 4 | 3 | A `down_from` above the number of sources is refused, naming both numbers. Setting it to 1 on a check with several sources is allowed, and the write carries this warning, which the web app and the iOS app show as written: ```text This check alerts when any 1 of its 3 locations reports it down, including a location whose own network failed. Requiring 2 of 3 is recommended. ``` ### Instant confirmation When a check runs from two or more regions and one of them reports it down, Pingtower asks each of the check's other regions to run it once more straight away, rather than at their next scheduled run. The quorum is then met, or not, within seconds instead of an interval. That happens once per failure streak. The streak opens with the first region to report down and stays open while any region is still reporting down. A region that reports up leaves the streak, and so does one whose last down is more than about two intervals old, so a region that stopped reporting can't hold it open. Further failures inside a streak don't ask again. A region that can't take the extra run skips it and answers at its next scheduled run. That happens when the check isn't placed there, when the region is shedding load, or when the target has used up its share of requests from Pingtower's servers. The extra run's result is an ordinary result: it counts toward the quorum and the history like any other. A check that runs from one region, and a domain expiry check, has no other region to ask. ### Domain expiry runs from one place A domain's expiry date is the same from everywhere, and registries rate-limit by address, so a `domainexpiry` check runs from exactly one source. Name one, or name none and Pingtower picks one. Naming more is refused with a `422` under the rule `single_placement`. Its `alert_if.down_from` is always 1, its interval defaults to once a day and can be no shorter than an hour (3,600 seconds; shorter is a `400` with the rule `interval_floor` on `interval_seconds`), and its timeout defaults to 30 seconds. On a self-hosted tower, Pingtower picks a live agent of yours that runs the addon. When none does, the check's `placement_note` says so. ## Alerting `alert_if` writes an ordinary rule, named ` is down`, that you can see in the rule list and edit like any other. Send `"alert_if": {}` for the default majority, or `{"down_from": 2}`. Leave `alert_if` out for no rule. The rule matches the check's own tag at level 400 or above, which is a **down** result: ```json { "name": "api-health is down", "match": { "tags": ["check:api-health"], "min_level": 400 }, "identifier_keys": ["check"], "fire_after": { "count": 2, "window_minutes": 2, "distinct_sources": true } } ``` The window is twice the interval, rounded up to whole minutes and at least one: 1 minute at 15 seconds, 2 at 60, 10 at 300. Every region re-sends a down result on each run, so two regions that are both down always have two votes in the window. The check owns `match`, `identifier_keys` and `fire_after`, and rewrites them when you edit the check. Everything else is yours from the first write: escalations, renotify, [actions](/docs/service/actions/), auto-resolve. Removing `alert_if`, deleting the check, or deleting the rule by hand detaches the rule rather than deleting it. A **degraded** result, such as an `httpcheck` over its `max_ttfb`, is level 300 and does not trip this rule. Write your own rule on the measurements for that, as on each addon's page. A run that could not happen at all, such as a probe that could not open a socket or a browser sandbox that would not start, is tagged `status:error`. It says nothing about your target. It is kept at level 300, so it shows in the log but never votes toward an alert. ### When a region goes quiet The `-quiet` rule Pingtower writes beside each region is a [`source_silent` rule](/docs/service/rules/#sources-that-go-quiet) on that region's source. Its horizon is twice the check's interval plus 30 seconds, rounded up to whole minutes. Each region reports every run, so a working region never trips it and a stopped one always does. It is your rule from the start: give it a ladder, change the horizon, or delete it. It counts toward the [watched-pairs cap](/docs/reference/limits/#counts) but is always created, so a check or a new region is never refused over it. ## Secrets Some kinds take secrets: an `httpcheck`'s authentication secret, a multi-step check's secret variables, a mail check's `PASSWORD`, a browser check's environment variables and the values a WebSocket check's `headers` reference as `{{NAME}}`. They go in the check's `env`, never in `with`. They are sealed at rest and write-only: reading the check back shows which names are set (`env_keys` for a browser or WebSocket check), never their values. Leaving `env` out of an update keeps the stored set, and `{}` clears it, unless the update changes where a secret goes: an HTTP check's `url`, `oauth_token_url` or method, a browser check's or WebSocket check's `url`, a mail check's `address`, the URL or method of a multi-step check's step that uses a secret, even only a path or a plain variable that URL uses, or of a step whose extraction that URL uses, or that step's extraction rule, headers, body, authentication or redirect setting, or `sources` gaining a region or source the env was not sealed for. That update must send `env` again, or it is refused with a `422` such as `env: with.url now points at another destination; re-supply env (or send {} to clear it)` or `env: sources now include the env was not sealed for; re-supply env (or send {} to clear it)`. Removing or reordering sources keeps the stored set. A value of `env` that turns up in a run's result, as written, JSON-escaped or URL-encoded, is stored as `[redacted]` in its message, keys and step errors; a browser check also catches more encodings, listed on [its page](/docs/service/browser-checks/#environment-variables). A value under 4 bytes is left as it is, and so is one encoded some other way. Each kind's page has its own rules for which names it takes. A create or update that finds a credential in plain text in a parameter, such as a token written as it is in a WebSocket check's `headers`, still saves, and the answer carries `secret_warnings`: `[{"field": "with.headers.Authorization", "kind": "…"}]`, naming the parameter and what was found, never the value. Where the kind takes one, use its sealed secret instead; a value that only references `env` as `{{NAME}}` is not flagged. ### A new agent token withholds secrets Secrets are sealed for each source as its agent token stood when they were set. Minting an agent token for a source, with POST `/v1/projects/{id}/sources/{name}/agent-token`, withholds every check's `env` from that source: a replacement token, the source's first token, and a token for a source deleted and created again under the same name all count, whoever mints it, an owner included. A check with `env` stops running from that source until you send its `env` again. The source records no results for it, never reports it down, and the agent is not sent the check at all. Reading the check lists those sources in `env_withheld_sources`, and the web app and the iOS app show the check as not running there. An update that sends `env` again, as on [any update](#secrets), clears the list, and the check runs from those sources again from the agent's next poll; an update that leaves `env` out keeps it. So a project admin can't read a check's secrets by taking over a source's agent: they would have to know the secrets to send them again. ## History Every run is recorded as it arrives, per source, in five-minute buckets kept for 8 days and hourly buckets kept for a year. The web app and the apps draw each check's success rate and response time over the last 24 hours, 7 days, 30 days or 90 days, for all its regions together or one at a time, and show each region's latest result. The success rate is the share of judged runs that passed: up and degraded runs, over all runs less the `status:error` runs and the runs inside a maintenance window. A probe that could not run is not a judgement of your target. ## Disabled and paused `"enabled": false` stops a check without deleting it or its history. A browser check can also be paused by the plan: when the tenant's browser minutes for the month are used up, reading the check shows `paused` with the reason `browser_minutes_exhausted` and the time it resumes. See [Browser checks](/docs/service/browser-checks/#browser-minutes). An `sshcheck` or `logwatch` check stored before Release 1 shows `paused` with the reason `local_only` and `resumes_at` `0`: Pingtower no longer assigns it, and it belongs in the agent's own config. See [Upgrading to Release 1](/posts/release-1-upgrade/#4-move-stored-sshcheck-and-logwatch-checks-into-the-agents-config).