fix deployment db issue and widthrawal administration page

docs/production_incident_remediation.md

@@ -0,0 +1,104 @@
+# Production Incident Remediation — Ops Follow-ups
+
+This document covers remediation items that cannot be completed from application code alone.
+Deploy the heartbeat / filter / rate-limit code changes first, then work through this list.
+
+## Deployment wave 1 (application)
+
+1. Back up production database and current release.
+2. Deploy:
+   - throttled `/session/ping-activity` client (`public/assets/js/session_timeout.js`)
+   - filter exclusions for timezone + school-year writable on session endpoints
+   - lightweight `SessionTimeoutController`
+   - route rate limit `apiratelimit:5,60` on ping endpoints
+3. Monitor for 30–60 minutes:
+   - ping-activity requests/minute
+   - HTTP 429 / 500 / 503 rates
+   - MySQL connection errors
+
+## 3. MySQL `Operation not permitted`
+
+Do **not** assume a bad password. Check Hostinger / server logs for the incident window
+`04:12:03`–`06:21:08` and capture:
+
+```sql
+SHOW VARIABLES LIKE 'max_connections';
+SHOW STATUS LIKE 'Threads_connected';
+SHOW STATUS LIKE 'Max_used_connections';
+SHOW STATUS LIKE 'Aborted_connects';
+```
+
+Also inspect:
+
+- PHP-FPM / LiteSpeed worker limits
+- Hostinger resource throttling / inodes / CPU
+- firewall / socket restrictions
+- temporary MySQL outages
+
+Document the confirmed root cause here once known:
+
+- Cause:
+- Evidence:
+- Fix applied:
+
+## 5. Missing tables / migrations
+
+After backup:
+
+```bash
+php spark migrate:status
+php spark migrate
+```
+
+Confirm:
+
+- `settings` exists
+- `user_preferences` exists
+- no new `table doesn't exist` errors
+
+Migration `2026-08-20-010000_CreateSettingsAndUserPreferencesTables` creates both tables if missing.
+
+## 7. SMTP authentication
+
+Credentials must live in environment configuration (`mail.SMTP*` / `SMTP_*`), not in
+`app/Config/Email.php`.
+
+Verify production env:
+
+- SMTP host / port / encryption
+- username + app password
+- from address / provider restrictions
+
+If auth recently failed, rotate the Gmail app password and update production env only.
+Send one controlled registration or student-removal email from production and confirm success.
+
+## 10. Monitoring / alerts
+
+Track at least:
+
+- HTTP RPS
+- `/session/ping-activity` requests/minute
+- HTTP 500 and 429 rates
+- DB connection failures
+- active MySQL connections
+- PHP worker usage
+- SMTP failures
+
+Suggested initial alerts:
+
+- DB connection errors > 5/minute
+- 500 responses > 1% of requests
+- ping-activity above expected session-based threshold
+- MySQL connections > 80% of `max_connections`
+
+## Verification checklist
+
+- [ ] No overlapping ping requests from one tab
+- [ ] Failed pings use exponential backoff
+- [ ] Heartbeat skips timezone / school-year / settings queries
+- [ ] Server-side ping rate limiting returns 429 when exceeded
+- [ ] MySQL `Operation not permitted` cause identified
+- [ ] `settings` + `user_preferences` exist; migrations current
+- [ ] SMTP auth succeeds
+- [ ] Zero-value invoices return controlled 422 (no null `setJSON`)
+- [ ] Monitoring / alerts enabled