Install it.
One Go binary and a Postgres. Run end to end before publishing; the order of the last few steps is deliberate.
This installs 0.3.0, the current release.
The compose file pulls :latest, which moves to the newest release when one
ships. Set TOKAY_TAG=0.3.0 in .env to stay put.
Back up Postgres before changing versions either way: downgrades are not supported.
What the numbers promise.
What you need.
- Docker with Compose. Images are
linux/amd64andlinux/arm64. - An Alertmanager already sending alerts.
- A Slack workspace or a Telegram bot.
- A domain with TLS -
APP_ENV=productionsets Secure cookies and CSRF.
Postgres ships in the compose file. For a managed one, edit DB_HOST in the
compose environment block - it overrides .env - and drop the
postgres service with its depends_on.
Two files, one directory.
mkdir tokayops && cd tokayops
curl -fLO https://raw.githubusercontent.com/tokayops/tokayops/main/docker-compose.prod.yml
curl -fL -o .env https://raw.githubusercontent.com/tokayops/tokayops/main/.env.example
chmod 600 .env Generate once, then keep them.
# Refuses to run twice - a changed ENCRYPTION_KEY orphans stored secrets.
if grep -qE '^ENCRYPTION_KEY=.+' .env; then
echo "ENCRYPTION_KEY is already set. Keep it."
else
sed -i.bak \
-e "s|^ENCRYPTION_KEY=.*|ENCRYPTION_KEY=$(openssl rand -hex 32)|" \
-e "s|^JWT_SECRET=.*|JWT_SECRET=$(openssl rand -base64 32)|" \
-e "s|^DB_PASSWORD=.*|DB_PASSWORD=$(openssl rand -hex 16)|" \
-e "s|^APP_ENV=.*|APP_ENV=production|" \
.env && rm .env.bak && chmod 600 .env
fi
# Lengths only, so nothing lands in your scrollback.
awk -F= '/^(ENCRYPTION_KEY|JWT_SECRET|DB_PASSWORD)=/ \
{ printf "%-15s %2d chars\n", $1, length($2) }' .env ENCRYPTION_KEY cannot be rotated.
It decrypts every stored integration secret. Keep a copy off this server; lose it and all integrations have to be recreated.
Then set TOKAY_SELF_URL to your public HTTPS address. Without it the
Ack/Resolve buttons are hidden and Telegram linking cannot complete.
sed -i.bak -e "s|^# *TOKAY_SELF_URL=.*|TOKAY_SELF_URL=https://tokayops.example.com|" \
.env && rm .env.bak Bring it up.
API and UI on :8080, health and metrics on :9090, both bound to
loopback.
docker compose -f docker-compose.prod.yml up -d
curl -s localhost:9090/health # -> OK Scraping it with Prometheus.
Alerting and recording rules ship with the release, in
deploy/prometheus/tokayops.rules.yml.
Give the job a scrape_timeout above 5s - /metrics gives the
database five seconds and answers without those series after that - and keep 30 days
of samples, which the SLO rules read.
Create the only way in.
No sign-up page. The first account on a database with no admin becomes admin.
docker compose -f docker-compose.prod.yml exec tokay \
/app/tokayops user create admin@example.com '<your-password>' 'Admin User' Put it behind HTTPS.
Serve TOKAY_SELF_URL. This has to work before the next step - Slack and
Telegram call back into it.
tokayops.example.com {
reverse_proxy 127.0.0.1:8080
} nginx instead
server {
listen 443 ssl;
server_name tokayops.example.com;
ssl_certificate /etc/ssl/tokayops.example.com.crt;
ssl_certificate_key /etc/ssl/tokayops.example.com.key;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
} Connect Slack or Telegram.
Configure → Integrations → Add. Slack: the form generates an app manifest, install it, paste back the Bot Token and Signing Secret, invite the bot to your channels. Telegram: paste a BotFather token; the secret token is required, not optional.
Then link your own account from the profile menu. An unlinked user cannot be DM'd and cannot press Ack.
Give the alert somewhere to land.
This is what decides whether a test alert pages anybody. In Configure:
- A team. Its ID is what the alert's
teamlabel must say. - A schedule for it, with a linked user on it.
- An escalation policy with real steps - a DM to the schedule, a message to a channel the bot is in.
- The policy attached to the team, as default or per severity. An unattached policy is never chosen and the group sits there notifying nobody.
Unlabelled alerts land in triage at info, so give
triage a policy too if you want a safety net. A severity other
than critical, warning or info counts as
info as well, and a severity route can only be set for those three.
Create the Alertmanager integration.
Configure → Integrations → Add → Alertmanager Webhook. Set a secret and copy it. The endpoint validates against integrations in the database, so until this exists it rejects everything.
Consider stale after is optional: how many minutes this Alertmanager may
say nothing about a group before the group is marked stale - silenced whole, a route
changed, Alertmanager down. A route repeats between its repeat_interval and
repeat_interval + group_interval, so take the upper bound and add a margin; if
the receiver serves several routes, the longest. Left empty, no group is marked.
Point Alertmanager at it.
Add the webhook to a receiver you already use, not a new route. Every config in a receiver fires, so existing delivery keeps working. Repeat per receiver you want mirrored.
receivers:
- name: team-alerts # a receiver you already send to
slack_configs: # leave what is already here alone
- channel: '#alerts'
webhook_configs: # add only this
- url: 'https://tokayops.example.com/webhook/alertmanager?token=<secret>'
send_resolved: true
max_alerts: 0 -
send_resolved: true- without it Alertmanager never says an alert cleared, and groups do not resolve by themselves. -
max_alerts: 0, the default - a notification cut short says how many alerts it dropped, not which, so a group could resolve while one still fires. The shipped Prometheus rules warn when it happens. - No custom
payload- it replaces the message TokayOps reads.
If you add a route instead.
A child route that matches everything takes those alerts away from its parent's
receiver, and continue: true only resumes at the next sibling - an earlier
match with the default continue: false ends the walk first. Both mistakes
pass amtool check-config.
Fire one and watch it land.
curl -X POST 'https://tokayops.example.com/webhook/alertmanager?token=<secret>' \
-H 'Content-Type: application/json' \
-d '{
"groupKey": "tokayops-install-test",
"commonLabels": {
"alertname": "TestAlert",
"team": "<your-team-id>",
"severity": "warning"
},
"alerts": [{
"status": "firing",
"labels": { "alertname": "TestAlert" },
"annotations": { "summary": "install check" }
}]
}' -
401- token does not match an enabled integration. Step 09. A new or rotated token takes up to half a minute to reach every instance. 400- nogroupKeyand no alertfingerprint.- Group appears but nobody is paged - no policy attached, or nobody linked. Step 08.