alrahma_sunday_school/payments_logic_and_data_repair_plan.md

Payments Logic and Data Repair Plan

1. Purpose

This plan fixes two separate problems:

1. Application logic that creates inconsistent or misleading payment rows.
2. Existing data in the payments table without deleting financial evidence or inventing transactions.

The repair must treat payments as an auditable ledger. A payment row should represent money actually received, not an invoice adjustment, fee assignment, discount, balance correction, or repeated attempt to make a balance reach zero.

---

2. Findings from the supplied dump

The dump contains 321 rows across 120 invoices.

Key findings:

• installment_seq is NULL in all 321 rows.
• number_of_installments is actually being used as the payment sequence number. It matches chronological row order for every invoice.
• 65 invoices have more than one total_amount value across their payment history.
• 168 rows have paid_amount = 10.00, mostly recorded as cash during April and May 2026.
• 116 of 120 invoice histories are internally consistent once a fixed pre-existing paid amount or credit is allowed.
• 33 internally consistent invoices have a non-zero implied opening paid amount. This means the table is not a complete ledger for those invoices.
• Four invoice histories contain an unexplained balance jump:

Invoice	Payment row where jump begins	Unexplained change
25	201	-590.00
65	304	-573.75
75	92	+90.00
96	232	-20.00

These four changes may represent missing payments, credits, reversals, adjustments, or incorrect stored balances. They must be reconciled against receipts, invoice history, audit logs, and bank or cash records before destructive correction.

---

3. Target accounting rules

After the repair, the following rules must always hold.

3.1 Payment rules

• One row in payments represents one actual receipt of money.
• A payment is immutable after posting, except for controlled metadata corrections.
• A mistaken payment is reversed with a linked reversal row. It is not deleted or silently overwritten.
• paid_amount must be greater than zero for a normal payment.
• transaction_id or an idempotency key must be unique and non-null.
• parent_id, school_year, and invoice ownership must come from the invoice, not from unchecked form input.
• Check-specific fields are required only for check payments.
• A payment cannot create a negative invoice balance unless explicit overpayment or account-credit logic is enabled.

3.2 Invoice rules

• invoices.total_amount is the current authoritative invoice total.
• invoices.paid_amount is the sum of valid posted payments plus approved opening credit or migrated paid balance.
• invoices.balance = invoices.total_amount - invoices.paid_amount.
• Invoice fees, discounts, waivers, penalties, and event charges are stored as adjustments, not payments.
• Invoice status is derived from the current balance instead of being independently guessed.

3.3 Installment rules

• installment_seq is the chronological payment sequence within one invoice.
• number_of_installments must not be used as both a sequence and a total count.
• Recommended final naming:
  • installment_seq: this payment's sequence number.
  • installment_count: total posted payment count, only if the application genuinely needs it.

---

4. Correct application logic

4.1 Record a payment atomically

All operations must run inside one database transaction. The invoice row must be locked so two users cannot post against the same balance simultaneously.

BEGIN TRANSACTION

1. Load invoice FOR UPDATE.
2. Reject missing, cancelled, or wrong-school-year invoice.
3. Read authoritative parent_id, school_year, total_amount, paid_amount, and balance from invoice.
4. Validate payment amount and payment method.
5. Reject duplicate idempotency key or transaction ID.
6. Reject amount greater than current balance unless explicit credit handling is enabled.
7. Calculate next installment_seq while the invoice is locked.
8. Insert one payment row.
9. Update invoice paid_amount, balance, and status.
10. Write an audit record.

COMMIT

Representative MySQL logic:

START TRANSACTION;

SELECT
    id,
    parent_id,
    total_amount,
    paid_amount,
    balance,
    school_year,
    status
INTO
    @invoice_id,
    @invoice_parent_id,
    @invoice_total_amount,
    @invoice_paid_amount,
    @invoice_balance,
    @invoice_school_year,
    @invoice_status
FROM invoices
WHERE id = :invoice_id
FOR UPDATE;

-- Application validations before continuing:
-- :paid_amount > 0
-- :paid_amount <= invoice.balance unless credits are explicitly supported
-- invoice.school_year = active school year
-- invoice.status is not cancelled/void
-- transaction_id/idempotency_key does not already exist

SELECT COALESCE(MAX(installment_seq), 0) + 1
INTO @next_installment_seq
FROM payments
WHERE invoice_id = :invoice_id;

SET @new_paid_amount = @invoice_paid_amount + :paid_amount;
SET @new_balance = @invoice_total_amount - @new_paid_amount;

INSERT INTO payments (
    parent_id,
    invoice_id,
    paid_amount,
    installment_seq,
    transaction_id,
    check_file,
    check_number,
    payment_method,
    payment_date,
    school_year,
    status,
    updated_by,
    created_at,
    updated_at
) VALUES (
    @invoice_parent_id,
    @invoice_id,
    :paid_amount,
    @next_installment_seq,
    :transaction_id,
    :check_file,
    :check_number,
    LOWER(:payment_method),
    :payment_date,
    @invoice_school_year,
    'recorded',
    :updated_by,
    CURRENT_TIMESTAMP,
    CURRENT_TIMESTAMP
);

UPDATE invoices
SET paid_amount = @new_paid_amount,
    balance = @new_balance,
    status = CASE
        WHEN @new_balance = 0 THEN 'paid'
        WHEN @new_balance > 0 AND @new_paid_amount > 0 THEN 'partial'
        WHEN @new_paid_amount = 0 THEN 'unpaid'
        ELSE 'credit'
    END,
    updated_by = :updated_by,
    updated_at = CURRENT_TIMESTAMP
WHERE id = :invoice_id;

COMMIT;

The exact status strings must match the values supported by the application. Do not introduce partial or credit until reporting and validation code understands them.

4.2 Record an invoice adjustment

A fee, discount, waiver, event charge, or correction changes the invoice total. It does not create a payment.

Create a dedicated adjustment table rather than forcing payments to perform accounting theatre:

CREATE TABLE invoice_adjustments (
    id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
    invoice_id INT UNSIGNED NOT NULL,
    adjustment_type ENUM(
        'charge',
        'discount',
        'waiver',
        'credit',
        'reversal',
        'correction'
    ) NOT NULL,
    amount DECIMAL(10,2) NOT NULL,
    description VARCHAR(255) NOT NULL,
    source_reference VARCHAR(100) DEFAULT NULL,
    school_year VARCHAR(9) NOT NULL,
    created_by INT UNSIGNED DEFAULT NULL,
    created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (id),
    KEY idx_invoice_adjustments_invoice (invoice_id),
    UNIQUE KEY uniq_invoice_adjustment_source (invoice_id, source_reference),
    CONSTRAINT fk_invoice_adjustments_invoice
        FOREIGN KEY (invoice_id) REFERENCES invoices(id),
    CONSTRAINT chk_invoice_adjustment_amount
        CHECK (amount <> 0)
) ENGINE=InnoDB;

Adjustment transaction:

1. Lock invoice FOR UPDATE.
2. Insert exactly one adjustment row using an idempotent source reference.
3. Recalculate total_amount from invoice base charges plus all adjustments.
4. Recalculate balance as total_amount minus paid_amount.
5. Update invoice status.
6. Commit.

This eliminates the likely cause of repeated $10.00 payment rows when the system is actually adding charges or closing small balances.

4.3 Prevent duplicate submissions

Add an idempotency key generated once by the client or server per payment request.

ALTER TABLE payments
    ADD COLUMN idempotency_key CHAR(36) DEFAULT NULL AFTER transaction_id,
    ADD UNIQUE KEY uniq_payments_idempotency_key (idempotency_key);

On retry, return the existing payment instead of inserting a second one.

4.4 Reverse rather than delete

Recommended columns:

ALTER TABLE payments
    ADD COLUMN reversal_of_payment_id INT UNSIGNED DEFAULT NULL,
    ADD COLUMN voided_at DATETIME DEFAULT NULL,
    ADD COLUMN voided_by INT UNSIGNED DEFAULT NULL,
    ADD COLUMN void_reason VARCHAR(255) DEFAULT NULL,
    ADD KEY idx_payments_reversal (reversal_of_payment_id);

A reversal should insert an explicit reversing transaction or mark the original row void while preserving the audit trail. The selected policy must be consistent across invoice totals, cash reports, and receipts.

---

5. Existing-data repair strategy

5.1 Repair principles

The migration must not automatically:

• Delete $10.00 rows.
• Merge rows merely because they share a date or amount.
• Change paid_amount without receipt evidence.
• Convert negative balances to zero using GREATEST().
• Recalculate every historical balance from the current invoice total.
• Assume the first stored payment is the first payment ever made.

Those shortcuts make reports look tidy while making the ledger less truthful. Aesthetic consistency is not accounting integrity.

5.2 Maintenance procedure

1. Put payment creation and editing into maintenance mode.
2. Take a database backup and verify restoration on a separate database.
3. Copy payments, invoices, and relevant audit tables into dated backup tables.
4. Run the audit queries in the companion SQL script.
5. Reconcile the four unexplained transitions against external evidence.
6. Apply safe sequence and normalization updates.
7. Apply only approved balance corrections.
8. Rebuild invoice summaries.
9. Run all validation queries.
10. Deploy corrected application logic before reopening writes.

---

6. Data-repair SQL design

The companion SQL script creates these working tables:

• payments_backup_20260718: exact pre-repair copy.
• payment_repair_analysis: row-level sequence, running totals, and implied opening paid amount.
• payment_repair_invoice_review: invoice-level consistency summary.
• payment_repair_transition_review: unexplained changes between consecutive rows.
• payment_repair_decisions: reviewed instructions for each anomalous transition.

6.1 Sequence repair

Safe automatic correction:

UPDATE payments p
JOIN payment_repair_analysis a ON a.payment_id = p.id
SET p.installment_seq = a.expected_installment_seq
WHERE p.installment_seq IS NULL
   OR p.installment_seq <> a.expected_installment_seq;

After the application has switched to installment_seq, repurpose the misleading field:

UPDATE payments p
JOIN (
    SELECT invoice_id, COUNT(*) AS installment_count
    FROM payments
    WHERE status = 'recorded'
    GROUP BY invoice_id
) c ON c.invoice_id = p.invoice_id
SET p.number_of_installments = c.installment_count;

Do not run the second update before confirming no application code still expects number_of_installments to be the current sequence.

6.2 Parent and school-year normalization

First inspect differences:

SELECT
    p.id,
    p.invoice_id,
    p.parent_id AS payment_parent_id,
    i.parent_id AS invoice_parent_id,
    p.school_year AS payment_school_year,
    i.school_year AS invoice_school_year
FROM payments p
JOIN invoices i ON i.id = p.invoice_id
WHERE p.parent_id <> i.parent_id
   OR NOT (p.school_year <=> i.school_year);

After review, normalize from the authoritative invoice:

UPDATE payments p
JOIN invoices i ON i.id = p.invoice_id
SET p.parent_id = i.parent_id,
    p.school_year = i.school_year
WHERE p.parent_id <> i.parent_id
   OR NOT (p.school_year <=> i.school_year);

6.3 Balance-transition audit

For every row after the first payment on an invoice, the expected transition is:

new balance
= previous balance
+ change in invoice total
- current payment
+ explicit non-payment adjustment not already reflected in total_amount

The supplied dump contains four unexplained transitions. The migration records a decision for each one:

CREATE TABLE payment_repair_decisions (
    payment_id INT UNSIGNED NOT NULL,
    invoice_id INT UNSIGNED NOT NULL,
    decision ENUM(
        'balance_is_wrong',
        'missing_payment',
        'missing_credit',
        'missing_charge',
        'missing_reversal',
        'leave_unchanged'
    ) NOT NULL,
    correction_amount DECIMAL(10,2) NOT NULL DEFAULT 0.00,
    evidence_reference VARCHAR(255) NOT NULL,
    approved_by INT UNSIGNED NOT NULL,
    approved_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
    notes TEXT,
    PRIMARY KEY (payment_id)
) ENGINE=InnoDB;

Rules:

• balance_is_wrong: correct the balance at that row, then recalculate all later balances for the invoice.
• missing_payment: insert a real payment using receipt evidence, then resequence.
• missing_credit: insert an approved credit adjustment.
• missing_charge: insert an approved charge adjustment.
• missing_reversal: insert or link a reversal.
• leave_unchanged: permitted only with written justification.

6.4 Preserve migrated opening paid balances

The dump proves that some invoices were already partially or fully paid before their first surviving payments row. Do not invent cash, card, or check transactions to fill that gap. Preserve the migrated amount separately with approval evidence.

CREATE TABLE invoice_opening_paid_balances (
    invoice_id INT UNSIGNED NOT NULL,
    amount DECIMAL(10,2) NOT NULL,
    effective_before_payment_id INT UNSIGNED DEFAULT NULL,
    school_year VARCHAR(9) NOT NULL,
    evidence_reference VARCHAR(255) NOT NULL,
    approved_by INT UNSIGNED NOT NULL,
    approved_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
    notes TEXT,
    PRIMARY KEY (invoice_id),
    CONSTRAINT fk_opening_paid_invoice
        FOREIGN KEY (invoice_id) REFERENCES invoices(id),
    CONSTRAINT chk_opening_paid_amount
        CHECK (amount >= 0)
) ENGINE=InnoDB;

The internally consistent candidates are produced by:

SELECT
    invoice_id,
    minimum_implied_opening_paid AS candidate_opening_paid
FROM payment_repair_invoice_review
WHERE is_internally_consistent = 1
  AND minimum_implied_opening_paid > 0.01
ORDER BY invoice_id;

The supplied dump produces 33 candidates. Each amount still requires approval because internal consistency proves only that the arithmetic repeats consistently, not that the original transaction actually occurred.

6.5 Correct a bad balance and recalculate forward

For an approved bad-balance decision, calculate the corrected row from the previous row:

UPDATE payments current_payment
JOIN payment_repair_transition_review review
    ON review.payment_id = current_payment.id
JOIN payment_repair_decisions decision
    ON decision.payment_id = current_payment.id
SET current_payment.balance = review.expected_balance
WHERE decision.decision = 'balance_is_wrong';

Then rebuild later balances for that invoice in chronological order. The companion script creates a staged result table first, allowing review before the final update.

Do not attempt this with an unordered multi-row variable update. SQL does not owe anyone deterministic behavior merely because the rows looked sorted in phpMyAdmin.

6.6 Rebuild invoice summaries

After payment, opening-balance, and adjustment reconciliation:

UPDATE invoices i
LEFT JOIN (
    SELECT
        invoice_id,
        SUM(CASE
            WHEN status = 'recorded' AND voided_at IS NULL
            THEN paid_amount
            ELSE 0
        END) AS ledger_paid
    FROM payments
    GROUP BY invoice_id
) p ON p.invoice_id = i.id
LEFT JOIN invoice_opening_paid_balances o
    ON o.invoice_id = i.id
SET i.paid_amount =
        COALESCE(o.amount, 0) + COALESCE(p.ledger_paid, 0),
    i.balance =
        i.total_amount
        - (COALESCE(o.amount, 0) + COALESCE(p.ledger_paid, 0)),
    i.status = CASE
        WHEN i.total_amount
             - (COALESCE(o.amount, 0) + COALESCE(p.ledger_paid, 0)) = 0
            THEN 'paid'
        WHEN COALESCE(o.amount, 0) + COALESCE(p.ledger_paid, 0) = 0
            THEN 'unpaid'
        WHEN i.total_amount
             - (COALESCE(o.amount, 0) + COALESCE(p.ledger_paid, 0)) > 0
            THEN 'partial'
        ELSE 'credit'
    END,
    i.updated_at = CURRENT_TIMESTAMP;

Discounts, waivers, and invoice credits must change invoices.total_amount through the adjustment ledger. They must not be counted as money paid. Opening paid balances are included only because they represent approved historical payments that predate the surviving ledger.

---

7. Schema hardening after cleanup

Apply only after data passes validation.

ALTER TABLE payments
    MODIFY transaction_id VARCHAR(100) NOT NULL,
    MODIFY installment_seq INT NOT NULL,
    MODIFY school_year VARCHAR(9) NOT NULL,
    ADD UNIQUE KEY uniq_payments_invoice_sequence (invoice_id, installment_seq),
    ADD CONSTRAINT chk_payments_paid_amount CHECK (paid_amount > 0),
    ADD CONSTRAINT chk_payments_method CHECK (
        payment_method IN ('cash', 'card', 'check', 'bank_transfer', 'online')
    ),
    ADD CONSTRAINT chk_payments_check_fields CHECK (
        payment_method <> 'check'
        OR check_number IS NOT NULL
    );

Recommended foreign key after confirming every reference is valid:

ALTER TABLE payments
    ADD CONSTRAINT fk_payments_invoice
        FOREIGN KEY (invoice_id) REFERENCES invoices(id);

Do not add a parent foreign key until the signed/unsigned types of both columns match.

---

8. Validation checklist

The repair is complete only when all checks pass.

Row and amount preservation

SELECT
    (SELECT COUNT(*) FROM payments_backup_20260718) AS before_rows,
    (SELECT COUNT(*) FROM payments) AS after_rows,
    (SELECT SUM(paid_amount) FROM payments_backup_20260718) AS before_paid,
    (SELECT SUM(paid_amount) FROM payments) AS after_paid;

Any difference must be explained by approved inserted reversals, recovered payments, or documented corrections.

Duplicate transaction IDs

SELECT transaction_id, COUNT(*)
FROM payments
GROUP BY transaction_id
HAVING transaction_id IS NULL OR COUNT(*) > 1;

Duplicate or missing installment sequences

SELECT invoice_id, installment_seq, COUNT(*)
FROM payments
GROUP BY invoice_id, installment_seq
HAVING installment_seq IS NULL OR COUNT(*) > 1;

Payment-to-invoice ownership mismatch

SELECT p.id, p.invoice_id, p.parent_id, i.parent_id
FROM payments p
JOIN invoices i ON i.id = p.invoice_id
WHERE p.parent_id <> i.parent_id;

Invoice summary mismatch

SELECT
    i.id AS invoice_id,
    i.total_amount,
    i.paid_amount,
    i.balance,
    SUM(CASE WHEN p.status = 'recorded' THEN p.paid_amount ELSE 0 END) AS ledger_paid
FROM invoices i
LEFT JOIN payments p ON p.invoice_id = i.id
GROUP BY i.id, i.total_amount, i.paid_amount, i.balance
HAVING ABS(i.balance - (i.total_amount - i.paid_amount)) > 0.01;

Remaining unexplained transitions

Recreate payment_repair_transition_review after corrections. It should return no unexplained transition unless the difference is linked to an explicit adjustment, credit, reversal, or approved migration record.

---

9. Deployment order

1. Add new nullable columns and adjustment/reversal structures.
2. Deploy application code that dual-writes the old and new installment fields.
3. Freeze payment writes.
4. Backup and run audit scripts.
5. Reconcile the four anomalous invoices and all opening paid balances.
6. Apply data repair.
7. Rebuild invoice summaries.
8. Run validations and business-report comparisons.
9. Deploy application code that reads only the corrected fields.
10. Add NOT NULL, UNIQUE, CHECK, and foreign-key constraints.
11. Reopen payment writes.
12. Monitor duplicate attempts, negative balances, and reconciliation failures.

---

10. Required tests

At minimum, automated tests must cover:

• Full payment.
• Partial payment.
• Final installment.
• Concurrent payments on the same invoice.
• Duplicate browser submission.
• Check payment without a check number.
• Payment greater than balance.
• Invoice charge after partial payment.
• Invoice discount after payment.
• Payment reversal.
• School-year mismatch.
• Invoice transferred to another parent or corrected ownership.
• Repeated event-fee processing with the same source reference.
• Status transitions from unpaid to partial to paid.

The concurrency test is non-negotiable. Without it, two perfectly valid requests can both read the same balance and produce one invalid ledger, because computers are extremely obedient even when asked to race into a wall.