Avoiding the Pitfall of Unchecked AI

July 2026

Status: scaffolding — planned, not yet the finished article. This page lays out the structure Matthew wants this piece built around, and pre-populates the one part that’s already available: a real inventory of the specific errors AI-assisted work introduced into this site’s own papers, later caught and corrected. The deeper layer — the actual root-cause analysis of how those errors got generated in the first place — is waiting on a Grok chat transcript export Matthew is retrieving from xAI. Once that’s available, it gets fed through for genuine root-cause analysis rather than guessed at here. The eventual goal, in Matthew’s own words: turn this into both a publishable article and a formal paper.

Why this is worth writing at all

Most of this site’s papers were built with heavy AI assistance (Grok, and later Claude for the audit pass), and that assistance introduced real, sometimes significant numerical errors — a rotation-curve claim off by 32x, a proton-decay comparison on the wrong side of its own experimental bound, a decay-rate figure with the wrong sign of exponent, borrowed physics results presented ambiguously as if independently derived. All of them were eventually caught, but only because a dedicated audit process went looking for them — they didn’t announce themselves. That’s the actual subject of this piece: not “AI makes mistakes” (uninteresting, everyone already knows this), but the specific, checkable mechanics of how confident-sounding, plausible-looking technical output can be wrong in ways that pass a casual read, and what a real process for catching it looks like in practice.

The framework

Matthew’s own manufacturing-engineering background supplies the structure, and it’s the same vocabulary already governing how this vault steers its own work (see item 11’s “Root Cause, Corrective Action, Look-Across, Poka-Yoke” in the CLAUDE.md guardrails behind this whole site) — this article is that same discipline turned outward into a public case study rather than kept as internal process:

• Problem → Reaction → Solution — a classic quality-engineering structure: state what actually went wrong, what the real response was once it was found, and what changed as a result.
• Best Practice — what the corrected version of the process actually looks like, stated as a reusable rule, not just a one-off fix.
• Case Study, in three parts — What Happened (the error itself, in concrete terms), What Did I Do (Matthew’s actual actions once he noticed something was off — this is the part the Grok transcript will fill in with real detail), and Resolution (how it was actually fixed and verified).
• Poka-yoke — the manufacturing term for an error-proofing mechanism: a structural change that makes the mistake difficult or impossible to make again, not just a reminder to be more careful next time.

Case study inventory: errors already found and fixed on this site

This is real, not hypothetical — every item below is a documented correction already live on solvetheuniverse.com, most surfaced by a systematic 2026-07-21/22 audit pass. Grouped by what kind of failure it actually was:

Numerical errors that contradicted the paper’s own stated formula or inputs:

• Paper 1’s flagship rotation-curve claim, off by ~32x when computed directly from the paper’s own stated parameters (plain Newtonian gravity, with no PBT enhancement, turned out closer to the real observed value than the paper’s own “corrected” curve).
• Paper 1’s black-hole-collapse ODE doesn’t actually stabilize near the Planck scale as claimed — solving the paper’s own equilibrium condition gives a radius that’s cosmologically enormous instead.
• Paper 1’s $G_{strong}\approx10^{29}$ figure doesn’t follow from the paper’s own hierarchical formula for any $\gamma$ in its own stated range.
• Paper 2’s drag-threshold acceleration was off by about four orders of magnitude from the formula’s own stated inputs — and a second, independent discrepancy (~1650x) was later found between that correction and an independent article’s own derivation of the same formula, traced to two different physically-motivated choices for one input rather than a simple arithmetic slip.
• Paper 4’s independently-parametrized version of the rotation-curve claim, off by ~2x.
• Paper 7’s neutron beta-decay rate, wrong by many orders of magnitude and the wrong sign of exponent relative to the paper’s own formula — repeated uncorrected in Paper 10.

A claim that was actually on the wrong side of the real experimental bound it cited as support:

• Paper 7’s proton-decay “alignment” with the Super-Kamiokande non-detection bound — taken literally, the paper’s own claimed decay rate is about 30x faster than the maximum rate that bound allows, meaning it predicts decay that should already have been seen. This is categorically different from a rounding error: the paper cited a real experimental result as supporting evidence while actually contradicting it.

Real, independently-confirmed numbers stated incorrectly:

• Paper 7’s muon $g$-2 anomaly, stated about 10x smaller than the real, published Fermilab measurement.

Real results borrowed from established physics, presented ambiguously as if independently derived:

• Paper 6 and Paper 7 both reused Schwinger’s real 1948 QED one-loop correction to the electron $g$-factor without originally flagging it as a borrowed, standard result rather than something PBT’s own aether-vorticity mechanism had produced.

Unverified statistics stated as if computed:

• Paper 7’s “80% validity” figure — not derived from any stated metric.
• Paper 8’s “8% gap” and Paper 9’s “6% gaps” — the same unverified-statistic pattern, appearing independently in two different papers.

Corrections that were made in one place but never propagated to where the same claim was repeated:

• The Reference Guide kept repeating Paper 1’s original “matches $G$” and ~220 km/s claims, Paper 4’s original figure, and the proton-decay “alignment” line for some time after the source papers themselves had already been corrected — a stale-propagation failure, not a fresh error.
• Push-Pressure Theory: An Overview silently repeated now-corrected wrong numbers with no note, for the same reason.

Structural/technical bugs, not content errors, but caught the same way — by someone actually checking:

• A markdown-rendering bug silently stripped raw HTML blocks sitewide (a <ul class="catalog-list"> element vanished from rendered output with no visible error).
• A CommonMark backslash-escape rule silently stripped \, (LaTeX thin-space) before KaTeX ever rendered it, leaving stray literal commas in formulas across 18 files and 69 occurrences, several predating any of this session’s own work.
• Several links pointed to Paper 13 as if it were published, when it’s actually draft: true and excluded from the live build.

What’s already functioning as poka-yoke on this site

Some of what came out of catching these errors wasn’t just a fix — it changed the site’s own structure so the same class of mistake is harder to make silently again:

• The automatic epistemic-status banner injected on every paper page, stating plainly that this is untested, fringe work — not a claim buried in prose that a reader has to go hunting for.
• The six-category Catalog classification (Proven / Accepted But Unproven / Unproven At Large / Unknowns / Fringe Ideas / Disproven) applied consistently, so every claim on the site has a stated evidentiary status, not just the ones someone remembered to caveat.
• The Formula Catalog’s binary Proven/Unproven label on every individual equation, specifically built to stop borrowed-but-unflagged results (like the Schwinger case above) from happening again.
• The append-only “editorial note” / “revised” convention used throughout every paper correction on this site — corrections get added with a date and attribution, the original claim stays visible rather than being quietly rewritten, and the record of what was wrong and when it got caught survives.
• The full-site internal-link checker, run before every deploy, which is what actually caught the dead-links-to-draft-Paper-13 bug above before it shipped.

What’s still pending

The genuine root-cause layer — why these specific errors got generated by the AI assistance in the first place, what patterns in the original prompting or verification (or lack of it) let them through, and what Matthew’s own actual reaction was in the moment he first suspected something was wrong — needs the real Grok transcript, not a reconstruction. That’s the part this page is deliberately leaving blank rather than inventing plausible-sounding root causes to fill the gap. Once the transcript is available, the “What Did I Do” and “Resolution” case-study sections above get filled in with Matthew’s real, specific account rather than this session’s own inference from the final corrected text.