tech.backup-restore-drill·version 1.0.0·draft1 to verify
Prove the backups restore
A backup is selected at random, restored in isolation, checked against the source, and the restore time is recorded.
When to use
Quarterly, or after any change to the backup system itself (new storage target, new tool, new key holder). Not during an actual outage — under pressure you restore production, not run a drill; that is tech.website-down / agents.incident-response. Not for checking that a backup job ran — that is a daily watch, part of tech.security-audit.
Before you start
- The backup inventory is current: what is backed up, where, how often, retention.
- Key holders are known by name for every encrypted backup.
- A non-production place to restore into exists (spare VM, container, scratch folder) — never the live system.
- If the company has stated recovery targets, they are written down; if not, this drill produces the first
What a run requires2
- Approval · S2 · ownerthe run stops until a named person records the decision
- Approval · S6 · ownerthe run stops until a named person records the decision
Any step can wait until a date and reopens by itself; every closed step leaves evidence (a note, a link, a number).
The trail7 steps
Pick the targetagent
Choose one system from the inventory — rotate so a different one is drilled each quarter; prefer one that has never been restored. Note the newest backup and its timestamp.
Done when the target, the backup file/job id and its timestamp are written in the drill log.
Approve the planownerneeds approval · owner
Approval · S2 · owner — the run stops until a named person records the decision
One paragraph: what will be restored, into where, what will be checked, who is on call if something breaks.
Done when the owner has approved the named target and place.
Restore into isolationagent
Restore the chosen backup into the scratch environment. Nothing points at production; no live hostname, no live database connection.
Done when the restored copy starts or opens, and no connection string in it points at production.
⛔ A restore that reuses live credentials or a live hostname can overwrite or corrupt production — rename everything before first start.
Verify the dataagent
Compare against the source: file counts and total size, database row counts on the main tables, checksums on a sample, open one real document end to end. Check the data age — is it the backup you think it is?
Done when each check has a recorded number from both sides, and "pass" or the exact mismatch.
Time it and find the gapsagent
Record how long the restore took and how old the data was. Note anything that made it slower or harder: a key that had to be hunted for, a credential that had expired, a missing instruction.
Done when restore time and data age are in the log, with every obstacle listed.
Decide on the resultownerneeds approval · owner
Approval · S6 · owner — the run stops until a named person records the decision
Present pass/fail, the numbers, the obstacles. The owner accepts the drill or assigns fixes with owners and dates.
Done when the decision is recorded — accepted, or each gap has an owner and a due date.
Record and scheduleagent
Write the drill log entry; add fix tasks where decided; set the next drill date a quarter out; tear down the restored copy (or keep it, if the owner wants it kept).
Done when the log entry exists, tasks are created, the next date is in the calendar, and the scratch environment is cleaned or marked.
Checks — how we know it worked
- The restored copy was opened and read, not merely unpacked — a file count alone is not a check.
- Data age of the restored copy matches the newest backup's timestamp (read it from the restored data, not
- No production host, address or credential was used during the restore.
- Restore time and data age are written as numbers, next to any target the company has set.
If it goes wrong
| Symptom | Response |
|---|---|
| Backup is corrupt or will not restore | Stop, do not "fix" it in place; check when the job last truly succeeded, re-run the job, re-drill the new backup within the quarter. |
| Key or credential missing/expired | This is a finding, not a detour — record it, get the key holder named in S2 to supply it, add a key-holder check to the next drill. |
| Restore is far slower than any stated target | Record the bottleneck (transfer, size, manual steps); the fix is an owner-assigned task from S6, not a note. |
| Restored copy connects to production | Isolate immediately (network off), assess whether anything was written, record it as a failed drill, fix isolation before any retry. |
| Backup job has been failing silently | Add the backup job status to a recurring watch; re-drill after two consecutive good jobs. |
What each step leaves behind
- S1the target, the backup file/job id and its timestamp are written in the drill log.
- S2the owner has approved the named target and place.
- S3the restored copy starts or opens, and no connection string in it points at production.
- S4each check has a recorded number from both sides, and "pass" or the exact mismatch.
- S5restore time and data age are in the log, with every obstacle listed.
- S6the decision is recorded — accepted, or each gap has an owner and a due date.
- S7the log entry exists, tasks are created, the next date is in the calendar, and the scratch environment is cleaned or marked.
Evidence to keep
Drill log entry (target, backup id, timestamp) · restore output · S4 comparison numbers · restore time and data age · obstacles list · S6 decision with who and when · task ids for any fixes.
How this playbook improves
After every 4 runs (a year) ask: did every system get drilled at least once, or does the rotation skip the hard ones? Did any obstacle in S5 repeat across drills without a fix closing it? Are the recorded restore times still within what the owner would accept in a real outage? A new version changes the step that let the gap persist, and says so in its change note.
Obligātās sīkdatnes tur sarunu kopā. Analītika ir izslēgta, līdz tu atļauj — tā neliek nevienu sīkdatni un neglabā ierīces identifikatoru.Necessary storage keeps your conversation together. Analytics is off until you allow it — it sets no cookie and stores no device identifier. Ko mēs glabājamWhat we store