WDN Monitor45.13.255.214
Last Backup
Sites Checked
Sites Down
Security Flags
0
No threats detected

Backup Success Rate — Last 20 Runs

Sites Down — Last 14 Checks

Site Type Distribution

Backup Volume (Sites per Run)

Nightly Backup History

0 runs
Date / Time Run ID Total Success Failed Status Failed Sites
Loading…

Site Status

0 sites
Domain HTTP Backup Type Last Checked
Loading…

Security Flags

0 flags
Timestamp Site Type Files Run ID Status
No security flags — all clear ✓

Live Ops Log

0 entries

System Notes — AI Context Reference

🖥 Server

Server IP  : 45.13.255.214
Host       : Hostinger VPS (hPanel)
User       : u930711690
Home       : /home/u930711690
Sites dir  : /home/u930711690/domains/  (~89 live domains)
Shell      : bash

📁 Key Files

/home/u930711690/wdn-backup-engine.sh     Nightly backup engine (cron: 0 1 * * *)
/home/u930711690/site-daily-check.sh      Morning HTTP check (cron: 0 8 * * *)
/home/u930711690/firebase-push.sh         Firestore REST writer (sourced by both scripts)
/home/u930711690/wdn-backup-config.sh     GitHub tokens + Telegram credentials
/home/u930711690/sar-monitor-config.sh    Telegram bot token + chat ID
/home/u930711690/wdn-restore.sh           One-command site restore from GitHub backup
/home/u930711690/wdn-ops.log              Master ops log — append-only, human+AI readable
/home/u930711690/.wdn-backup-state/       Per-run state directory
  engine.log                              Full backup engine output log
  last_run_status.txt                     Last run summary (read by morning check)

🗄 Backup System

Schedule   : MANUAL TRIGGER — run when needed (no daily cron)
             nohup bash -c 'STAGGER_TARGET_SECONDS=60 bash ~/wdn-backup-engine.sh' \
               >> ~/.wdn-backup-state/engine.log 2>&1 &
             Single site test: TEST_SITE=example.com bash ~/wdn-backup-engine.sh
Stagger    : 60s min between sites (89 sites ≈ 2–3 hours)
GitHub     : Primary → wedefinenet / Mirror → wedefinenet1-beep
             Tokens in ~/wdn-backup-config.sh
Retention  : 7 releases per repo (pruned automatically)

Per-site repo name pattern:
  {domain-slug}-wp-backup      WordPress sites
  {domain-slug}-html-backup    Static HTML sites
  {domain-slug}-node-backup    Node.js sites

GitHub Release assets per backup:
  *-core-*.zip       Code + DB dump (plugin-format, excludes uploads)
  *-EMERGENCY-*.zip  Code + DB dump (plain restore format, excludes uploads)
  *-media-*.zip      wp-content/uploads/ only (can be 1–2 GB)
  changelog.json     File diffs, DB row changes, PHP error log, security scan

Security scan in changelog.json checks for:
  eval(base64_decode | eval(gzinflate | eval(str_rot13 | passthru($_ | system($_ | exec($_ | assert($_
  Any hit → SECURITY_FLAG logged in wdn-ops.log + Firestore + Telegram alert sent immediately

🔁 Restore a Hacked/Broken Site

One command restore (downloads from GitHub, restores files + DB):
  bash ~/wdn-restore.sh monkcreatives.com
  bash ~/wdn-restore.sh monkcreatives.com backup-2026-05-12-160625

What wdn-restore.sh does:
  1. Auto-finds the GitHub backup repo for the domain
  2. Downloads EMERGENCY zip + MEDIA zip from the latest (or named) release
  3. Saves current wp-config.php credentials (DB name, user, pass)
  4. Wipes public_html — preserves uploads/ as fallback
  5. Extracts clean files from EMERGENCY zip
  6. Restores media from MEDIA zip (freshest version)
  7. Patches wp-config.php with live server DB credentials
  8. Runs mysql import for database.sql
  9. Fixes permissions (755 dirs / 644 files / 600 wp-config)
 10. Cleans all temp files on exit

If DB restore fails → SQL saved to /tmp/wdn-db-restore-{domain}-*.sql

🌐 Site Health Check

Schedule   : 8 AM UTC daily  (0 8 * * *)
Checks     : HTTP status via curl (follows redirects, 12s timeout)
             DNS resolution → skips if domain points to different server
             Unless behind Cloudflare proxy (still tested)

Telegram alerts sent only when problems exist:
  🔴 Server Error (5xx)
  🟠 Not Found (404)
  🟡 Forbidden (403)
  ⚫ Connection Failed
  ⚪ No DNS / Expired
  🚨 Backup missed (no run in last 25h)
  Silent if all sites are UP and backup was clean

🔥 Firebase / This Dashboard

Project    : monitor-cb381
Dashboard  : https://monitor.wedefinenet.com/
Login      : superadmin@wedefinenet.com

Firestore collections written by scripts:
  backup_runs/{run_id}              One doc per nightly engine run
  site_checks/{YYYYMMDD_HHMMSS}    One doc per morning HTTP check batch
  site_status/{domain-slug}         Upserted per domain per check
  security_flags/{domain_runid}     PHP shell hits (resolved=false until cleared)

firebase-push.sh writes via Firestore REST API (no SDK — plain curl + PHP JSON builder)
Security rules: write=true (server writes unauthenticated), read requires auth

🔐 Security Context

In April 2026 the server suffered a mass compromise.
Multiple WordPress sites had eval(str_rot13(gzinflate(base64_decode(...)))) malware
injected into theme functions.php files.
Key site affected: ptdemolition.in (nordic theme functions.php wiped + replaced with malware)
Quarantine: ~/sar-quarantine-20260426/

This monitoring system was built as a direct response to that incident.
The backup engine changelog.json scans every site on every run for these patterns.

⚠️ Known Issues / Watch List

rovepackersandmovers.in   HTTP 404 since 2026-05-10 — site files may be missing or WP broken
manticinfra.com           DNS points to 93.127.169.244 (old server) — update A record to 45.13.255.214
Shell crontab blocked     Hostinger VPS blocks crontab from shell — must set via hPanel
  Site check (active) : 0 8 * * *  /bin/bash /home/u930711690/site-daily-check.sh >> ~/.wdn-backup-state/site-check.log 2>&1
  Backup engine       : MANUAL — no cron set intentionally (trigger manually when needed)