Alerts and email
When 3R Monitor declares an outage, how alert emails are combined, who gets them, and how to set up outgoing mail.
Failure detection
3R Monitor declares an outage only after several checks fail in a row, so a single slow response or a lost packet doesn't send an alert.
| What | Rule (defaults) |
|---|---|
| Website / API (monitors) | Down after 3 failures in a row. While it fails, and during the outage, it rechecks every 20 seconds, to confirm quickly and to notice the recovery quickly. |
| Server offline | No report for 30 seconds. The agent reports every 3 seconds, so that is 10 missed reports in a row. |
| CPU / memory | Above 90 % for 120 seconds |
| Disk | Any partition above 90 % |
| Services | failed services, services enabled at boot that are not running, and services you mark as required: bad in 3 reports in a row |
| SSL certificate | Alert when fewer than 10 days are left; warning within 30 days |
| Monthly traffic quota | Only for servers with a quota: an alert at 80 % and another at 100 % of the quota |
| Vulnerabilities | See Vulnerability checks |
Change the defaults in Settings → Default thresholds, for one server in its settings, and for a monitor in the monitor itself.
When all servers (3 or more) stop reporting at the same time, the problem is almost always the master's own network, firewall or reverse proxy. You then get a single "the master is not receiving any agent reports" alert instead of one email per server.
Every confirmed outage is also recorded on the Incidents page, with its start, end and duration. Incidents are kept for a year.
Combined emails
Alert emails are combined, so that a bad moment doesn't flood your inbox:
- The first alert is sent after 1 minute, so things that fail together (a server and the services on it) arrive in one email.
- After that, at most one alert email every 5 minutes, with everything that happened in between.
- An outage that recovers within that time appears once, as "down and recovered". Each combined email also lists the outages that are still ongoing, which counts as their reminder.
Change the interval in Settings → Alert notifications → Combine alert emails (minutes); 0 sends every alert right away, in its own email.
The combined email also carries, when they apply:
- vulnerabilities that need action now;
- with AI analysis, the likely cause of a new outage and what to check first (the email then waits up to about two more minutes);
- with AI on-call, repairs that were done, problems that need you and operations waiting for your approval.
Emails that the AI decides to send by itself (on-call results, urgent vulnerabilities) are limited per day: see Emails from the AI. The system's own alerts above are never limited by that.
Alert emails are written in the system language (Settings → General → System language) and use the panel's time zone.
Which alerts send emails
In Settings → Alert notifications → Send emails about, each kind can be turned off:
- Server offline (or the master receiving no reports at all)
- High CPU or memory usage
- Low disk space
- Monthly traffic quota (80% and 100% used)
- System services not running
- Monitored services (HTTP, TCP, Ping, DNS)
- SSL certificates
- Recoveries (back to normal)
Alerts that are turned off here still show in the panel and in Incidents. Vulnerability emails are set under Settings → Vulnerability checks.
How often
| Setting (Settings → Alert notifications → How often) | Default | What it does |
|---|---|---|
| Combine alert emails (minutes) | 5 | At most one alert email per interval; the rest is combined. 0 = no combining. |
| Reminder interval while down (minutes) | 0 | A reminder every N minutes while something stays down. 0 = no reminders. |
| At most this many emails per hour | 10 | All emails together: alerts, security and AI reports (test emails don't count). 0 = no limit. |
When the hourly limit is reached, emails wait in the outbox and alerts keep being combined until there is room again.
Who receives alerts
| Recipients | Where | Receive |
|---|---|---|
| Alert recipients | Settings → Alert notifications → Recipients | Every alert. Separate several addresses with commas. |
| Extra alert recipients of a group | Groups → a group | Only the alerts of the servers and monitors in that group, in addition to the global recipients |
A server with Enable alerts turned off (in its settings) raises no alerts at all.
Groups
Groups organize servers and monitors by project or service, for example "Website" or "API". Each group has a name, an order, a description and its own Extra alert recipients (separate several with commas). On the Overview you can collapse a group, or locate its servers on the map. Deleting a group (⋯ → Delete) moves its servers and monitors to "Ungrouped". Share links can be limited to some groups.
Alert emails are sent through an SMTP server you configure. Each change is saved as soon as you make it.
Open Settings → Outgoing mail (SMTP).
Under Server, enter the SMTP server, the Encryption (SSL (465), STARTTLS (587) or None (25)) and the Port. Under Account, the Username and the Password. Under Sender, optionally the From address (empty = the username) and the From name (default "3R Monitor").
Enter at least one address in Settings → Alert notifications → Alert recipients.
Back in Outgoing mail (SMTP) → Test, click Send a test email, and check your inbox. The box next to it sends the test to another address; empty = the alert recipients.
| Provider | Server | Port / encryption | Notes |
|---|---|---|---|
| Gmail / Google Workspace | smtp.gmail.com | 465 SSL or 587 STARTTLS | Use an app password, not your normal password (needs 2-step verification). |
| Microsoft 365 / Outlook | smtp.office365.com | 587 STARTTLS | SMTP authentication must be enabled for the mailbox. |
| Your hosting provider | As given by the provider | Usually 465 SSL |
Failed emails are retried automatically, up to 12 times with growing intervals. The System page lists recent emails with their status (Sent, Retrying, Gave up) and the error.
You can also send a test email from the master's command line, using the panel's SMTP settings:
sudo 3rmonitor mail-test --to you@example.comIf emails don't arrive, see Troubleshooting.