Skip to content

Integrations and alert intake

Enterprise feature (1.19.0, helpdesk). See the Helpdesk overview.

Monitoring tools, scripts and bots can open tickets of their own. Desk settings → Integrations (Helpdesk settings or Email channel) adds an integration for each tool. Its tickets are automated: they dedupe repeats, solve themselves when the alert recovers, and never send mail, receipts or surveys to anyone.

Add integration and pick the kind:

Kind For
Prometheus Alertmanager Alertmanager’s webhook receiver
Grafana A Grafana Alerting webhook contact point
Datadog A Datadog webhook
Azure Monitor An action group’s webhook, with the common alert schema
PagerDuty-style events Any tool that speaks the PagerDuty Events API v2
Generic webhook Anything that can POST JSON
API or bot Your own scripts, bots and agents
Automated email Tools that can only send mail

Then set:

  • The team its tickets go to.
  • The severity map: which priority each alert severity gets. Out of the box Critical is Urgent, High is High, Warning is Normal and Info is Low. The desk reads the tool’s own words (“CRITICAL”, “Sev1”, “warn”, “P2”).
  • The dedup window (24 hours by default) and whether a repeat after solving reopens the ticket within it (on by default).
  • Whether a resolved alert solves the ticket.
  • A rate limit (60 requests a minute by default).

Saving shows the integration’s key once: copy it now. It is stored only as a hash, so nobody can read it back; Make a new key replaces it; the page shows the first characters of the current key. The page also shows the endpoint, the steps for that tool and a curl example.

The endpoint is https://your-domain/api/helpdesk/intake/<id>, and the key goes in Authorization: Bearer <key> or X-Api-Key: <key> (or, for Azure Monitor only, ?key=<key> in the address). Your monitoring tools must reach it over HTTPS.

The alert The desk
Fires, new Opens a ticket for the integration’s team, marked Automated · Grafana, with priority from the severity map. Assignment rules and agents’ notices run as for any ticket.
Fires again while its ticket is open Adds “Fired again (×N)” to the ticket. No new ticket, no new notices.
Fires again after its ticket was solved Within the window, with reopen on: reopens the ticket with a note. Otherwise opens a new ticket that notes the earlier one.
Resolves Adds a note and, with auto-solve on, solves the ticket, unless a person has replied publicly on it (“Alert resolved; ticket left open because a person replied”).

A tool’s retry of the same request has one effect, however close together the two arrive. Each alert’s dedup key ties its firings and its resolve together.

An automated ticket shows the alert’s links and fields beside the conversation. Its requester is the integration (“Monitoring (Grafana)”), so the composer opens on an internal note. Alert text can’t insert images, embeds, videos or disguised links: the desk writes every link itself, and a link’s target can’t break out of it. Recent deliveries on the integration’s page lists what each request became: Created, Updated, Reopened, Resolved, Ignored or Rejected. Send a test alert opens a real ticket tagged test-alert.

Add a receiver to alertmanager.yml and route the alerts you want as tickets to it. Keep send_resolved: true so a recovered alert solves its ticket. Each alert’s fingerprint is its dedup key, and its severity label picks the priority.

receivers:
- name: compassdocs-helpdesk
webhook_configs:
- url: "https://your-domain/api/helpdesk/intake/<id>"
send_resolved: true
http_config:
authorization:
type: Bearer
credentials: "<your key>"
  1. Alerting → Contact points → add a contact point of type Webhook with the endpoint as its URL.
  2. Under Optional Webhook settings, set Authorization Header Scheme to Bearer and the credentials to your key.
  3. Point a notification policy at the contact point. Resolved notifications solve the ticket.
  1. Integrations → Webhooks: add a webhook with the endpoint as its URL and this payload:

    {
    "title": "$EVENT_TITLE",
    "text_only_msg": "$TEXT_ONLY_MSG",
    "alert_transition": "$ALERT_TRANSITION",
    "alert_type": "$ALERT_TYPE",
    "alert_id": "$ALERT_ID",
    "aggreg_key": "$AGGREG_KEY",
    "priority": "$ALERT_PRIORITY",
    "hostname": "$HOSTNAME",
    "link": "$LINK",
    "tags": "$TAGS"
    }
  2. Under Custom headers, add {"Authorization": "Bearer <your key>"}.

  3. Mention @webhook-<name> in a monitor’s message. The Recovered transition solves the ticket.

  1. In an action group, add a Webhook action and turn on the common alert schema.
  2. Azure can’t send an Authorization header, so put the key in the address: https://your-domain/api/helpdesk/intake/<id>?key=<your key>.
  3. Fired opens a ticket and Resolved solves it; Sev0 to Sev4 map to the priorities you set.

For tools that speak the PagerDuty Events API v2: use the endpoint as the events URL and send the key as a Bearer token. event_action: trigger opens or repeats, resolve solves, and dedup_key ties them together. routing_key is ignored.

POST JSON with Authorization: Bearer <your key>:

Terminal window
curl -X POST "https://your-domain/api/helpdesk/intake/<id>" \
-H "Authorization: Bearer <your key>" \
-H "Content-Type: application/json" \
-d '{"title":"Disk almost full on fs-02","severity":"warning","status":"firing","dedup_key":"fs-02-disk","source":"fs-02"}'

Fields: title (required), description, severity, status (firing or resolved), dedup_key, source, links ([{ "label", "url" }]) and fields (an object of extra facts). Send the same dedup_key with status: resolved to solve the ticket.

The same fields as a generic webhook, from a script, bot or agent. Add requester_email to open the ticket for a workspace member: when it is an active account that may use Get help, the ticket is theirs, not automated, and they get the usual updates; otherwise the address is ignored. Send an Idempotency-Key header to make retries safe.

For tools that can only send mail, add an Automated email integration and list the tool’s sender addresses. Have the tool mail your help address (see Email channel) from one of them. Its mail is accepted even though it is automated, as long as your mail provider confirms the sender (see Sender authentication): results the sender wrote into its own headers are ignored. Mail with no result at all is refused unless the integration has Accept mail without SPF or DKIM results on; a failed check is always refused. The subject is the ticket’s title, and a subject tagged [RESOLVED] or [OK] solves the ticket with the same subject. A subject pattern can pick the dedup key out of the subject. A pattern that could backtrack without end is refused, and every match has a time budget.

Code Means
200 / 201 Accepted; the body says what the alert became
400 The payload isn’t one the integration understands (no title, bad JSON)
401 No key, or a wrong one
402 The helpdesk license is missing or expired
403 The integration is turned off
413 The body is over 256 KB
429 Over the integration’s rate limit; retry later

Every rejection is listed under Recent deliveries with its reason, once per reason a minute. Repeated wrong keys are rate limited before anything is written.

Action Shown as
helpdesk.integration.created / .updated / .deleted Connected / Edited / Removed a monitoring or API integration
helpdesk.integration.key_rotated Issued a new integration API key
helpdesk.intake.rejected Rejected an alert or API ticket