Monitor restic and BorgBackup backups

File backups fail quietly. The repository password changed, the external drive wasn’t mounted, or a path was renamed and the backup now covers an empty folder. A heartbeat check with a numeric value catches these along with the usual failures: the backup reports how much data it covered after it succeeds, and Pingstack alerts you when the report is late or the amount drops below what you expect.

1. Create a heartbeat check with a size threshold

Create a heartbeat check whose schedule matches the backup job, with a grace period longer than the backup’s usual runtime.

Then turn on value alerting:

Reported value alerting: below a threshold
Warning value: somewhat below the usual size of the data you back up, in megabytes
Critical value: a size that would mean something is badly wrong

For below, the critical value must be less than the warning value. The scripts below report the total size of the files each backup covered, which stays steady from run to run, rather than the new data added, which can legitimately be close to zero. Attach a notification group.

2. Keep the ping URL private

Copy the location’s ping URL from the check in Pingstack and save it in a file only the backup user can read:

install -m 600 /dev/null ~/.pingstack-backup
read -rs URL && printf 'PINGSTACK_PING_URL=%s\n' "$URL" > ~/.pingstack-backup

Repository passwords belong in a password file with mode 600, as below, not in the script.

3. Back up, then report the size

Both scripts need jq to read the backup tool’s JSON summary.

restic

#!/usr/bin/env bash
set -euo pipefail
source ~/.pingstack-backup
export RESTIC_REPOSITORY=/mnt/backup/restic
export RESTIC_PASSWORD_FILE=~/.restic-password

BYTES=$(restic backup --json /home /etc \
  | jq -r 'select(.message_type == "summary") | .total_bytes_processed')

restic forget --keep-daily 7 --keep-weekly 4 --prune

SIZE_MB=$(( BYTES / 1024 / 1024 ))
curl --fail --silent --show-error --max-time 10 --retry 3 \
  --get --data-urlencode "value=$SIZE_MB" "$PINGSTACK_PING_URL" > /dev/null

BorgBackup (1.x)

#!/usr/bin/env bash
set -euo pipefail
source ~/.pingstack-backup
export BORG_REPO=/mnt/backup/borg
export BORG_PASSCOMMAND="cat $HOME/.borg-passphrase"

BYTES=$(borg create --json '::{hostname}-{now}' /home /etc \
  | jq -r '.archive.stats.original_size')

borg prune --keep-daily 7 --keep-weekly 4
borg compact

SIZE_MB=$(( BYTES / 1024 / 1024 ))
curl --fail --silent --show-error --max-time 10 --retry 3 \
  --get --data-urlencode "value=$SIZE_MB" "$PINGSTACK_PING_URL" > /dev/null

set -euo pipefail stops the script at the first failure, including a backup failure on the left of the pipe to jq, so a failed backup sends nothing and Pingstack alerts you when the deadline passes. The report also waits for pruning, so the check covers the whole job.

Borg exits with code 1 for warnings, such as a file that changed while it was being read. With set -e, a warning stops the script and no report is sent. If warnings are routine for your data, handle exit code 1 explicitly instead of treating it as a failure.

Schedule the script with cron or a systemd timer.

4. Confirm the first report

Run the script by hand as the backup user. Open the check in Pingstack and confirm the location received a heartbeat with the size you expect, and shows a next deadline.

Check the repository too

A backup that writes successfully can still sit in a damaged repository. Run restic check or borg check on its own schedule, weekly for example, and give it a second heartbeat check that reports only when the check passes.

Why a healthy backup might still alert

Missing mount. If the source or repository is on a drive that isn’t mounted, the backup fails or covers an empty folder. Both raise an alert.
Thresholds set too tight. Deleting a lot of files lowers the size below your warning value. Adjust the thresholds as the data changes.
Repository lock. A run that overlaps the previous one, or a stale lock after a crash, makes the backup fail. Clear stale locks with restic unlock or borg break-lock once you’re sure nothing is running.

If alerts still don’t match what you expect, see troubleshooting monitoring.