Microcosm Dynamics: an open, scored dynamic microsimulation model
Microcosm Dynamics is an open model that follows individual Americans through time and computes each person’s Social Security benefits year by year. From the Panel Study of Income Dynamics (PSID), it learns how people earn, marry, divorce, have children, become disabled and die, and it draws claim ages from the Social Security Administration’s published awards. Its earnings, family and disability components pass tests on held-out or external data, as do its PSID transitions once we carry them onto the Current Population Survey. We fit the whole engine on data through 2014, and it passes a holdout test against the PSID’s 2015–2019 records. Mortality and claiming run without such a test, and four of the passes, the holdout among them, rest on public amendments we adopted to a test’s terms after an earlier attempt failed. The forward earnings law fails a further test that carries it from 2010 to 2014. Four blind tests compare the model with DYNASIM3’s published results, and none of them uses the tested earnings, family or disability components. Under a one-point cut to cost-of-living adjustments, which acts mechanically on benefits in payment, the model’s benefit changes land within 0.74 percentage points of DYNASIM3’s in every age group. Raising the full retirement age to 68 cuts average benefits at ages 65 to 69 in 2030 by 5.7 percent in the model and 9.9 percent in DYNASIM3. Both projections draw no earnings after 2010, check no insured status, apply no retirement earnings test, model no claiming response and compute benefits with Python rules over a fixed 35 computation years. A 13 percent benefit cut raises adjusted poverty at age 67 by 0.46 percentage points in the model and 2 in DYNASIM3. In the primary specification, all four minimum-benefit versions we test reach more women and fewer men than DYNASIM3 projects, a split that three of six alternatives break. The Axiom rules engine, which encodes the statute as machine-readable rules, so far reproduces the model’s Python rules for average indexed monthly earnings across the PSID cohort, and for the primary insurance amount on synthetic workers. We plan to add forward earnings, the earnings test and behavioral claiming to the projections and to run the full benefit rules in Axiom.
Plain-language summary
Microcosm Dynamics is a computer model of Social Security’s retirement, survivor and disability benefits that anyone can download, run and check. It follows individual Americans through time. In each simulated year, the people in the model can earn, marry, divorce, have children, become disabled, claim benefits and die. The model then applies Social Security’s rules to each life to work out what that person collects. The two projections we have compared with DYNASIM so far use only part of this. In them, nobody earns, marries, divorces or has children after 2010, and the model pays benefits without checking whether a person worked long enough to qualify or applying the retirement earnings test, which withholds benefits from early claimants who keep working. A model like this can answer questions that a snapshot of one year cannot, such as which retirees a benefit cut reaches decades from now.
Outsiders can read the results of the models that already do this work, including the Urban Institute’s DYNASIM, the Social Security Administration’s MINT, the Penn Wharton Budget Model and the Congressional Budget Office’s long-term model. CBO does not publish its model’s code, and we found no public code with which to rerun DYNASIM, MINT or the Penn Wharton Budget Model. Among other models that estimate reforms across a population, we found public code for OG-USA and the Cato Institute’s model. We publish the code, estimates and tests of Microcosm Dynamics under an open-source license.
Microcosm Dynamics is a work in progress. The end of this summary says what it cannot do yet and what we plan to add.
Where the model’s people come from
The model learns how lives unfold from the Panel Study of Income Dynamics (PSID), a University of Michigan survey that has followed a set of American families, and the new families their members form, since 1968. It takes the ages at which people claim benefits from the Social Security Administration’s published statistics. Every projection so far starts from PSID members, and all four comparisons below draw their people from the PSID. The model is also built to start from Microcosm, PolicyEngine’s data on 166,302 people drawn mainly from the Census Bureau’s Current Population Survey and weighted toward official totals, though no projection starts there yet.
The PSID’s design limits what the model can learn from it. The survey interviewed 9,152 families in 2023, while the Current Population Survey samples about 95,000 households a year. Because it follows the 1968 families and their descendants, it covers later immigrants mainly through two small added samples. Of the original 1968 members still alive in 2015, about 40 percent were still responding, and those who left differ from those who stayed. Its records of low earnings have known measurement problems. Since 1997 it has interviewed every other year, and for the 7,486 people in the projections who reach age 62 between 1979 and 2030, about half of the career years through 2010 hold values we filled in or set to zero (Section 6).
What we have tested it against
We test the model in two ways, first with pre-registered tests. Before we score a piece of the model, we post which statistics we will compare and how close counts as a pass. Each version of the piece then gets one scored run, against data set aside from what it learned from or data from outside sources, and we publish every result. We call these tests gates, a software-engineering term for the checks that code must clear before it ships. A piece that faces such a test joins the model only after it passes, though some pieces face no test yet.
The pieces that generate earnings, marriage and childbearing, work-limiting disability and who lives with whom each pass a pre-registered test, as does the step that applies the PSID’s patterns to the people in Microcosm. A test run of the model’s year-by-year projection, with its pieces running together, passes too: we let it learn from data through 2014 and compare its projected marriages, disability and earnings for 2015 to 2019 with what the PSID recorded. We had already seen the PSID’s data for those years when we chose parts of the projection’s design and the statistics that test compares. Four of the passes came only after a version failed and we publicly changed the test’s terms: those for earnings histories, marriage and childbearing, the Microcosm step and the projection. In two of them, the version that passed is the same one that failed under the old terms. A further pre-registered test, of forward earnings, has failed. It carried the earnings law forward from 2010 to 2014 and projected a zero-earnings rate about 35 percent above the PSID’s for people aged 25 to 44, so nothing that depends on that step may use it until a successor law passes.
The second way is four blind tests against published results from DYNASIM3, the previous generation of the Urban Institute’s model. In each test, a separate workstream read DYNASIM3’s published numbers twice, independently, and sealed them until we had saved the model’s answer. The AI model that coordinated the build saw the cost-of-living test’s values before that test ran, in a report that may also have held the retirement-age test’s. The builders, some of them AI agents, could reach files of DYNASIM3 material, and the records cannot rule out that they opened them. Each test is blind only as far as those records show.
- Cutting each year’s cost-of-living adjustment by one percentage point. By 2030 the cut lowers average benefits by about 4 to 18 percent in the model, and the model’s change lands within 0.74 percentage points of DYNASIM3’s in every age group. Once we choose the year from which each person’s increases are cut, the cut’s effect on benefits already being paid follows by arithmetic. The close match therefore says little about how well the model captures how lives change.
- Raising the full retirement age, the age for an unreduced benefit, to 68. For people aged 65 to 69 in 2030, the age group where the two models differ most, average benefits fall 5.7 percent in the model and 9.9 percent in DYNASIM3. In the other four age groups, the two models’ changes lie within 1.09 percentage points of each other. The model leaves out changes in when people claim and the retirement earnings test, which withholds benefits from early claimants who keep working; either could account for the gap.
- A 13 percent cut to every benefit. For people born from 1936 to 1945, the poverty rate at age 67, on a measure that counts income people could draw from their wealth, rises 0.46 percentage points in the model and 2 points in DYNASIM3. The model’s figure rests on 483 people born in odd years, so one person of average weight moves it by about 0.2 points. DYNASIM3’s report rounds poverty rates to whole percentages, so its 2-point rise could lie anywhere between 1 and 3 points.
- A minimum benefit tied to years of work. Under the headline version, 7.75 percent of beneficiaries aged 62 and older receive the minimum in the model and 6.3 percent in DYNASIM3. Under the model’s main assumptions, all four versions put a larger share of women and a smaller share of men on the minimum than DYNASIM3 does. Three of the six alternative assumptions we registered in advance break that pattern in at least one version.
DYNASIM3 is another model, so each gap measures a disagreement between two models, not an error in either. The first two tests project PSID families to 2030. The last two apply each reform to incomes people reported to the PSID. DYNASIM3’s figures for those two come from projections published in 2004 and 2006, so part of each gap can reflect the different populations. None of the tests involves DYNASIM4, the current version, and none establishes that our model matches either version.
What the model can and cannot say yet
The model can apply a reform to the incomes the PSID recorded and count whom it reaches. It can also take the 11,405 people born in 1980 or earlier in the families the PSID interviewed in 2011 and project them to 2030 under a reform, adding no newborns or immigrants. It reports how effects differ across people, such as benefit changes by age group and poverty at 67. It produces no cost estimates, no measure of Social Security’s long-run balance and no date when the trust funds run out.
None of the four blind tests uses a piece that passes a pre-registered test. In the two 2030 projections, nobody earns, marries or divorces after 2010, and the benefit formula counts every year after 2010 as a year without earnings. A piece that projects earnings forward passes its own test, fails a second one that carries it forward from 2010, and neither projection uses it yet. The projections pay a retirement benefit to every claimant with earnings on record through 2010, without checking whether that person worked long enough to qualify. They do not yet apply the retirement earnings test, and nobody in them changes when they claim in response to a reform.
The parts that decide how long people live, when they claim and who receives disability insurance run without a pre-registered test. The model has a step that would add immigrants, but no run we have published uses it. The disability piece that passes its test models who reports a work-limiting disability, a different measure from who receives disability insurance. Three of the four blind tests compute benefits with our own Python code, and the fourth cuts reported benefits directly. The Axiom rules engine, which encodes the Social Security statute as machine-readable rules, so far reproduces one step of that calculation for the people in our tests: a worker’s average monthly earnings over the highest-earning years, adjusted for wage growth. On synthetic workers it also reproduces the next step, the basic benefit.
We plan to add earnings, marriage and divorce after 2010 to the projections, apply the retirement earnings test and the family maximum (the cap on what one worker’s family can collect), let claim ages respond to reforms where the data support it, run the full benefit rules in Axiom, and compare the model with public tables from SSA’s actuaries, MINT, the Penn Wharton Budget Model and CBO (Section 7). We have compared the PSID’s earnings for 1998 to 2004 with a public file of Social Security earnings records, and we plan to compare the model’s earnings and whole careers with that file (Section 6).
1 Introduction
Some questions about Social Security need more than a snapshot of one year: who a benefit cut reaches decades from now, or how a higher retirement age plays out across cohorts. Dynamic microsimulation answers them by aging a population of individual people year by year, drawing their earnings, marriages, disabilities, deaths and claims, and applying the program’s rules to each life. The Urban Institute’s DYNASIM, the Social Security Administration’s MINT and the Congressional Budget Office’s long-term model, CBOLT, do this for Social Security. Outside analysts can read the results these models publish, but CBO does not publish CBOLT’s code, and we found no public code for DYNASIM or MINT with which outsiders could rerun them or reproduce their results (Favreault et al. 2015; Smith et al. 2021; Congressional Budget Office 2018).
Anyone can read, rerun and check Microcosm Dynamics. It extends Microcosm, PolicyEngine’s open cross-sectional microdata for the United States, with a longitudinal layer we estimate on the Panel Study of Income Dynamics (PSID). We publish its code, estimates and tests under the MIT license at github.com/PolicyEngine/microcosm-dynamics. The model produces no cost estimates. Its first-estimates report sums weighted benefits and payroll contributions over the projected sample, labels them frame-relative and states that it does not estimate national dollars (runs/first_estimates_v1.json). The model computes no actuarial balance and no trust fund depletion date. One report-only analogue on a closed frame of 1,549 PSID careers sets a starting reserve so that its baseline runs out in 2034, the year Smith (2015) reports, then measures how each provision moves that year (runs/m2_pseudo_projection_v1.json). It is not a trust fund projection.
The model’s generated and forward earnings, its family formation models and its work-limitation disability module each pass a gate: a test on held-out or external data that we register publicly before we run it. We admit a component that faces a gate to the model only after it passes. We take the term from software engineering, where code must clear quality gates before it ships, and from the go/no-go decisions of clinical trials. Before a gate’s scored run, we fix the statistics it scores, which we call cells, and a tolerance for each cell. We change them only by a public amendment, which must precede the next scored run. The processes we estimate on the PSID also pass a transport gate, which tests them once we carry them onto the Current Population Survey (CPS) frame, the representative population the model can start from. Four gates pass only under such amendments, which we adopted after an earlier attempt failed (Section 4). They cover the earnings histories the model generates, family formation, the CPS transport and the whole engine, which we fit on data through 2014 and score on its 2015–2019 projections against the PSID. Mortality, claiming and disability-insurance entitlement run without a gate, and no gate certifies how mortality rates drift over time. The engine can add immigrants on a schedule, but no committed run has added one.
Four blind tests compare the model with results published from DYNASIM3, and none of them uses a gated component. In each test, we post the plan publicly before the run, and two independent readings of DYNASIM3’s published values stay sealed until we have committed the model’s output to the repository. Each plan, which we call a registration, discloses its participants’ exposures, including any to those published values, and a test is blind only as far as those disclosures show. For a one-point cut to cost-of-living adjustments, the model’s 2030 benefit changes land within 0.74 percentage points of DYNASIM3’s in every age group, and the absolute gap averages 0.38 points. Once we fix the year from which each benefit’s cost-of-living increases run, that cut acts mechanically on benefits in payment, which makes this comparison a weak test of behavioral dynamics. When the reduced increases start at each beneficiary’s own entitlement year, the absolute gap averages 1.47 points.
Each of the other three tests finds a gap between the two models. Raising the full retirement age to 68 cuts average benefits for people aged 65 to 69 in 2030 by 5.7 percent in the model and 9.9 percent in DYNASIM3, so the model’s cut is 4.2 points shallower. A 13 percent benefit cut raises adjusted poverty at age 67 in the 1936–45 birth cohort by 0.46 percentage points in the model and 2 points in DYNASIM3. The model observes only the cohort’s odd birth years. DYNASIM3 projects that the headline version of a minimum benefit reaches 6.3 percent of beneficiaries aged 62 and older, and the model finds 7.75 percent. In the model’s primary specification, each of the four versions we test reaches more women and fewer men than DYNASIM3 projects. Three of the six alternative specifications put more men, or fewer women, on the minimum than DYNASIM3 does in at least one version. These last two tests apply each reform to incomes the PSID already recorded, while DYNASIM3’s figures come from projections its authors published in 2004 and 2006, so a gap in either test can reflect the population’s vintage as well as the two models’ rules.
The two projection tests, which cover the cost-of-living cut and the higher retirement age, draw no earnings after 2010, count every later year as zero in the benefit formula and check no insured status. The model’s forward earnings law, which draws each person’s employment and earnings rank every two years, passes its own gate against the PSID’s earnings for 2016 and 2018, which we had in view when we chose the law’s structure, and fails a second test, B2, that carries it from 2010 to 2014. We fit that law on PSID earnings through 2014 for ages 25 to 64. It has no fitted distribution above 64 and draws anyone older from its oldest age band. Both projection tests start in 2010 and need earnings for people in their sixties, the ages at which the retirement earnings test withholds benefits from beneficiaries who work before full retirement age. The model computes the projection tests’ benefits with Python rules. The Axiom rules engine so far reproduces one piece of those rules, average indexed monthly earnings computed with statutory computation years, while the projection tests count a fixed 35 years. We plan to connect the forward law to the projection tests, apply the retirement earnings test, add behavioral claiming and run the full benefit rules in Axiom (Section 7).
2 The model
The model fits most transitions of individual lives on the PSID, draws claim ages from the Social Security Administration’s published awards, and runs these transitions forward, one year at a time, on a population of people. It has three layers. A starting population supplies the people, the projection engine moves each person through one year (engine/loop.py), and the rules layer turns the simulated lives into Social Security benefits (ss/).
2.1 The starting population
The model is built to start from either Microcosm’s certified, representative frame of the United States or a closed PSID cohort; so far, only gate w1 places its processes on the frame, and every projection starts from a PSID cohort. We build the frame from the Current Population Survey (CPS), and it holds 166,302 people in 57,240 households, with a 2024 reference period (gates.yaml, gate_w1). We estimate the transitions on the PSID, a multi-decade family panel, and carry them onto the CPS frame. Gate w1, the transport gate, tests whether those transitions reproduce the frame’s cross-sectional joint distributions of earnings, age, sex, marital status and household composition.
All four blind tests against DYNASIM3 use a closed PSID cohort (cohorts/). We build each comparison’s cohort with code that reads the PSID files, and we verify the variable labels that code uses. The two projection tests, which run the engine forward, start from 11,405 members of the 2011 PSID wave. Their earnings histories run from 1968 through 2010, with PSID-reported earnings where the panel has them and imputed or zero values where it does not (Section 6).
2.2 The projection engine
Each simulated year, the engine takes every person’s record through eight steps in a fixed order (engine/loop.py): mortality, aging, the marital core (which updates marital state), fertility, disability, earnings, claiming and household composition. We build each step as a separate component that does its own calculations. The loop fixes only the order of the steps and the flow of marital state: fertility (step 4) and household composition (step 8) both read the one result the marital core (step 3) produces, so every downstream step works from the same marriage history. Scheduled immigrant cohorts enter through a dedicated entry point in the loop, and births come from the fertility step.
The engine takes every random number from a dedicated stream, keyed by draw, period and module, and by person where a step needs it (engine/rng.py). The same seed produces an identical projection.
We fit the engine’s estimated components only on data dated no later than a boundary year, now 2014 (engine/refit.py). We use the boundary to score the engine out of sample. After fitting the whole engine on data through 2014, we project across the later PSID waves and compare the projection with what the PSID recorded. Gate m6 scores that comparison over 2015–2019 (gates.yaml, gate_m6). The model treats 2020–2022 as an exogenous shock outside its scope; we publish that window as a diagnostic and keep it out of every gated cell. We hold the fitted parameters, but not every design choice, to data through 2014. We selected gate m6’s scored cells with the post-2014 waves in view (gates.yaml, gate_m6).
The engine separates scored holdout runs from open forward projections (engine/support.py). A scored holdout run may condition on which people the PSID observed in later waves. An open forward projection, which no gate scores, gets no later-wave information: the engine refuses to supply it.
2.3 The rules layer
The rules layer computes Social Security benefits from the engine’s output. Its Python oracle, the reference code in ss/ against which we check the Axiom rules engine, computes average indexed monthly earnings (AIME), the primary insurance amount (PIA), claiming-age adjustments, and spouse and survivor benefits under 42 U.S.C. 415 and 402. Its delayed retirement credits stop at age 70 or 48 months after full retirement age, whichever comes first. That matches the statute for people born in 1943 or later and falls 2 to 12 months short for people born from 1917 to 1942 (Ghenis 2026c). The statute’s window now sits beside the oracle (ss/statutory_delayed_credit.py). A diagnostic reproduced five registered results with the oracle’s window, recomputed them with the statute’s, and found that none of the values it published changes; it bounded the first estimates instead of recomputing them (docs/errata/2026-10-11-delayed-credit-window.md). It takes its parameters from the PolicyEngine-US parameter tree. The oracle counts computation years in one of two ways (ss/statutory_aime.py): a fixed 35 for every worker, or the statutory number, which falls short of 35 for workers who reach 62 with fewer elapsed years.
Axiom, a rules engine that encodes the statute as machine-readable rules, reproduces the oracle’s AIME for all 7,486 people we test, using statutory computation years. Those people are the members of the 2011-wave cohort who reach 62 between 1979 and 2030. A bridge program (axiom_benefit_bridge.py) passes the cohort’s stored earnings histories into the real Axiom engine. With a fixed count of 35 years instead, the oracle’s AIME differs from the statutory AIME for 251 of the 7,486, all born before 1929.
On the PSID cohort we compare Axiom with the oracle on AIME alone, and both compute it on an old-age basis that assumes no period of disability. We give Axiom and the oracle the same input definitions, and the match depends on them. Both treat PSID labor income as covered earnings, and both receive the same zero or imputed values for years before 1968, years after 2010 and years absent from the panel. Of the 320,375 earnings rows the bridge sends, 194,465 hold zeros or imputed values. Axiom and the oracle produce identical output from identical records. The match says nothing about whether either AIME equals a person’s true AIME. Axiom’s Social Security encodings await acceptance, so the blind tests that compute benefits run on the Python oracle (cola_track_a/benefits.py).
3 What the model covers today
None of the four blind tests uses a component that passes a gate (Table 1). Of the four blind tests, exercise 1 projects how a one-point cut to cost-of-living adjustments changes benefits in 2030, and exercise 3 does the same for a full retirement age of 68. These two are the projection tests. The other two apply a reform to incomes the PSID recorded: a uniform 13 percent benefit cut in exercise 2 and a minimum benefit in exercise 4. Each verdict we report comes from its gate’s committed result file under runs/. Gates 1, 2, w1 and m6 pass only under public amendments, which we adopted after earlier attempts failed (Section 4).
| Component | What the model does | Test and status | Used in the blind tests |
|---|---|---|---|
| Recorded earnings | PSID head and spouse labor income, 1968–2022, with odd years after 1996 filled from neighboring years (exercise 4 reads the PSID’s off-year reports for 2001–2021) and other missing years set to zero | Data, with imputed gaps; both fill rules fail gate_epuf_fill (runs/epuf_fill_gate_test_v1.json) |
Yes, in exercises 1, 3 and 4 |
| Generated earnings histories | A backcast process that generates earnings careers | gate 1: passes against held-out panel moments (runs/gate1_rank_knn_v5.json) |
No |
| Forward earnings | Rank-chain law fitted on 2014 and earlier, ages 25–64 | gate m6: passes, 4 of 5 seeds (runs/gate_m6_candidate3_v1.json); B2, carried from 2010 to 2014: fails (runs/track_b_b2_score.json) |
No |
| Earnings test | SSA’s exempt amounts for 2000–2026 and the statutory formula that projects later years | Tested against SSA’s published amounts; not applied | No |
| Mortality | Death probabilities by age band and sex | Not gated; gate m6 certifies nothing about mortality drift | Exercises 1 and 3 use SSA’s 2004 period life table; exercise 2 annuitizes wealth with NCHS 2000 tables |
| Disability | Work-limitation incidence, recovery, occupancy and exits near full retirement age | gate m4: passes, 5 of 5 seeds (runs/gate_m4_hazard_v1.json) |
No |
| SSDI entitlement | Awards, recoveries and conversion at full retirement age, at rates through 2008 | Not gated | Yes, in exercises 1 and 3 |
| Claiming | SSA’s claim-age distribution of retired-worker awards (Annual Statistical Supplement, Table 6.B5.1), by sex and entitlement year | Not gated | Yes, in exercises 1 and 3, with no response to reform |
| Marriage and fertility | Marriage, divorce, remarriage, widowhood and births | gate 2: passes, 4 of 5 seeds (runs/gate2_hazard_v16.json) |
No |
| Household composition | Who lives with whom | gate 2b: passes, 4 of 5 seeds (runs/gate2b_hazard_v9.json) |
No |
| Marriage by earnings | The joint of marital status and earnings | gate 2c: passes, 4 of 5 seeds (runs/gate2c_hazard_v2.json) |
No |
| Entrants | Scheduled immigrant cohorts, which no committed run has added; births come from the fertility step | Report-only; not gated | No |
| Base frame | Transport of PSID processes onto the CPS frame | gate w1: passes (runs/gate_w1_candidate4_v1.json) |
No |
| Benefit rules | AIME, PIA, auxiliaries in the Python oracle | Reference implementation | Yes, in exercises 1, 3 and 4 |
| Axiom execution | Statutory rules in the Axiom engine | AIME matches the oracle for 7,486 of 7,486 people; PIA matches to the cent for 240 synthetic workers | No |
| SSI, income and poverty | Static SSI response and an adjusted poverty measure | Registered test, exercise 2 | Yes, in exercise 2, which cuts reported benefits directly |
| Wealth and pensions | PSID wealth annuitized into income | No projection | Exercise 2 only |
Earnings. The model has three kinds of earnings (recorded, generated and projected), and the blind tests use only the recorded kind. Recorded earnings are the labor income of PSID heads and spouses for income years 1968–2022 (data/family.py). The PSID asks about them every year through 1996. After that it interviews every other year, and each interview centers on the year before it. The model’s main earnings panel reads only those years, so we fill the odd years from neighboring years and set other years without a usable record to zero (Section 6). For the 7,486 members of the 2011-wave cohort who reach 62 between 1979 and 2030, 128,033 of their 253,943 career years through 2010, about half, hold values we imputed or filled with zero. The oracle computes benefits directly from these recorded careers. The projection tests use careers through 2010. The minimum-benefit test, exercise 4, reads a longer span of the same panel. For the odd-numbered years from 2001 to 2021, it reads the labor income each following wave reports for the year before last.
Generated earnings come from a backcast, a backward-looking process that produces earnings careers. Gate 1 scores that process against held-out panel moments, relative to the panel’s own noise floor, and it passes (runs/gate1_rank_knn_v5.json). It scores people aged 25 to 59 in the years 1998 to 2022, so it tests the middle of careers, not their whole length.
Projected earnings come from the forward earnings law (engine/forward_earnings.py), a rank chain we fit on earnings the PSID observed through 2014 at ages 25 to 64. Every two years, the law draws whether each person works and where they rank in the earnings distribution, then converts that rank to dollars with the national average wage index, which the law projects from a log-linear trend it fits on 2005–2014. It has no fitted distribution above 64 and draws people over 64 from its oldest age band. The engine assigns fixed zero earnings to anyone without a fitted 2014 earnings state (engine/earnings_domain.py).
The forward law passes gate m6 in 4 of 5 random seeds, its only certification (runs/gate_m6_candidate3_v1.json; Section 4). Gate m6 scores the whole engine’s holdout projection (Section 2), and its earnings cells compare persistence, growth, mobility, low earnings and the zero-earnings rate in 2016 and 2018 with the PSID’s records. In all five seeds, the projection’s two-year growth in log earnings at ages 25 to 44 runs 2.7 to 5.2 log points lower than the PSID’s, and its 10th percentile of positive earnings at those ages runs 15 to 24 percent lower. The correlation of log earnings four years apart runs 0.74 to 0.76 in the projection and 0.67 to 0.71 in the PSID. We fit the law’s parameters only on data through 2014, but we selected its structure with the later waves in view (gates.yaml, gate_m6).
No blind test yet draws forward earnings, and the projection tests draw no earnings after 2010 (cola_track_a/adapters.py). Their projections begin in 2010, four years before the forward law’s 2014 fit boundary, and they reach ages above 64, where the law has no fitted distribution. The retirement earnings test withholds benefits from beneficiaries in their sixties who work before full retirement age. Both projection tests’ specifications list it among the differences from DYNASIM3 that they leave out, and no projection applies it yet. The model already holds the earnings test’s parameters: SSA’s exempt amounts for 2000–2026, which we checked against SSA’s published history, and the statutory formula that projects later years.
Family. Three family gates pass, each in 4 of 5 seeds. Gate 2 scores marriage, divorce, remarriage, widowhood, the stock in each marital state and births (runs/gate2_hazard_v16.json). Gate 2b scores household composition (runs/gate2b_hazard_v9.json), and gate 2c scores the joint distribution of marital status and earnings (runs/gate2c_hazard_v2.json). The projection tests use none of these gated family models. Their marital step sets a person’s status to widowed when a linked spouse dies in the simulation, and it draws no marriages and no divorces.
Disability, mortality and claiming. The disability module passes gate m4 in 5 of 5 seeds (runs/gate_m4_hazard_v1.json). It models self-reported work limitation: incidence, recovery, occupancy and exits near full retirement age. Gate m4 scores it against held-out PSID employment-status records and against the age shape and exit composition of SSA’s 2023 disability-insurance statistics. It never scores SSA’s levels. Work limitation is a different concept from a Social Security Disability Insurance (SSDI) award. The projection tests use a separate SSDI entitlement component, which draws awards, recoveries and conversion at full retirement age at rates observed through 2008 (engine/di_entitlement.py). No gate scores it.
The engine’s mortality step uses a table of death probabilities by age band and sex, and gate m6 certifies nothing about how mortality drifts over time. The projection tests use SSA’s 2004 period life table. Each year we multiply its death probabilities by the ratio of that year’s age-sex-adjusted death rate to the 2004 rate, both from the 2008 Trustees Report’s intermediate assumptions. We use one ratio for ages under 65 and another for 65 and over, and each applies to men and women alike. The 2008 Trustees Report prints no projected death probabilities by single year of age and sex. SSA’s Office of the Chief Actuary publishes such probabilities in downloadable files alongside its Reports, which we found for the 2014 Report onward; for the 2026 Report, the men’s intermediate file is https://www.ssa.gov/OACT/Downloadables/CY/DeathProbsE_M_Alt2_TR2026.csv. Claiming runs with no gate. The claiming step draws a claim age from SSA’s published distribution of retired-worker awards by sex and entitlement year (claiming.py). For entitlement years after 2008, the projection tests reuse the 2008 distribution. No claim responds to a reform. Exercise 3, which raises the full retirement age to 68, therefore holds each person’s projected claim age fixed under the reform. Its registration, the plan we post before the run, lists claiming responses only as alternatives that we report beside the main result (Section 5).
Population. Gate w1 passes (runs/gate_w1_candidate4_v1.json). When we carry the transitions we estimate on the PSID onto the CPS frame (Section 2), they match the frame on all 44 gated cells: 43 cross-sectional joint distributions and one ordering check. The entrant schedule lets new people enter a projection. We report its output, and no gate scores it. No fitted law takes that output as an input.
Rules. Exercises 1, 3 and 4 compute benefits with the Python oracle (cola_track_a/benefits.py). The projection tests count a fixed 35 computation years for every worker. Axiom, using the statutory count instead, reproduces the oracle’s AIME for all 7,486 cohort members we test (Section 2). Exercise 2 applies no benefit rules. It cuts reported Social Security income by 13 percent inside an income concept we define in Python, applies a static Supplemental Security Income (SSI) offset and computes an adjusted poverty measure. It also annuitizes PSID wealth into income with NCHS 2000 tables.
4 How the model is scored
We admit to the model any component that passes a gate, and four components run in it without one: mortality, claiming, disability-insurance entitlement and entrants. Each gate names its coverage, the held-out data it scores against, the cells it scores and a tolerance for each cell. We write all of a gate’s terms into gates.yaml and fix them before the scored run. We call each version of the component under test a candidate, and we register it, give it one scored run against its gate and commit the result under runs/ whatever the verdict. A failed candidate therefore stays in the record next to the one that passed. We change a gate’s terms only by public amendment, which must precede the next scored run.
We use four kinds of gate: holdout, anchor, transport and temporal holdout. A holdout gate scores a component on persons we excluded from its fit. Gate 1 holds out 20 percent of persons. The family gates hold out half, split by person for gates 2 and 2b and by couple for gate 2c, so that both members of a couple fall in the same half. Each of these gates scores every candidate on the same five random seeds, and gate 1 scores one classifier test over 20. To judge the model against the sampling variation of the data a gate scores, we set each cell’s tolerance against a noise floor: the discrepancy between two disjoint halves we draw from one real panel. A seed passes only if every gated cell stays within its tolerance, and a family gate needs at least four of its five seeds to pass. An anchor gate scores a component on a person-disjoint half of the panel and against external administrative publications. The disability gate compares the age shape and exit composition of work-limitation estimates with SSA’s 2023 disability-insurance statistics. It scores none of SSA’s levels.
The transport gate, w1, tests whether the processes we estimate on the PSID reproduce the CPS frame’s cross-sectional joint distributions. The temporal holdout, m6, fits the whole engine on data from 2014 and earlier and projects it forward. It scores projected marital flows and disability flows for 2015 to 2019, and earnings moments for 2016 and 2018, against PSID records. It publishes the 2020–2022 window as a diagnostic and scores none of it. Each gate also states what it does not certify. Gate m6 certifies nothing about how mortality and widowhood drift over time, or about entrants.
Four gates pass only under amendments we adopted after an earlier candidate failed: gates 1, 2, w1 and m6. Gate 1 passes under two amendments, and gates.yaml records both (gate_1). In a distinguishability test, a classifier tries to tell generated careers from recorded ones. The first amendment makes a failing test of this kind report-only, a status for cells we publish with no tolerance, and adds a gated block that scores the careers in benefit space, which weights earnings the way the benefit formula does. The second scores another distinguishability test by its mean over 20 seeds. When we adopted it, a committed diagnostic had already put the failed candidate’s 20-seed mean at 0.523, under the gate’s threshold of 0.53 (runs/c10_diagnostics_v1.json). The passing candidate, runs/gate1_rank_knn_v5.json, is identical to its failed predecessor. Gate 2 passes on its 16th candidate, a changed model, under an amendment to the gate’s estimator and a split of the gate into parts that we score separately. We adopted both after earlier candidates failed (gates.yaml, gate_2). The amendment scores the gate by its mean over 20 draws. The split leaves marriage and fertility in gate 2, and we score household composition and marriage by earnings in separate gates, 2b and 2c.
Gate w1 passes under three amendments, each of which we adopted after a candidate failed. Together they make 21 cells report-only, and narrow the one ordering check the gate scores. The third makes four household-size shares report-only: those for one, three, four, and five or more people. Among the gated cross-sectional cells, candidate 3 had failed only those four, and it put the one-person share at 0.19 against the frame’s 0.08. We had named those cells as risks in the forecast for the second amendment, before candidate 3 ran, and the third amendment states that candidate 3 would pass its terms. As at gate 1, the passing candidate is identical to its failed predecessor (runs/NOTES-gate_w1_candidate4.md).
At gate m6, the third candidate passes under one amendment, which restricts the earnings cells to people who have a fitted 2014 earnings state and re-derives the cells’ noise floors. Candidate 1 failed before that amendment and candidate 2 after it (runs/gate_m6_candidate1_v1.json, runs/gate_m6_candidate2_v1.json; gates.yaml, gate_m6, runs/m6_holdout_floors_v4.json). The restriction drops 2,722 of the 13,163 people the earnings cells had scored. The re-derived floors raise the tolerance on the 10th percentile of positive earnings at ages 25 to 44 from 0.221 to 0.284, and three of candidate 3’s four passing seeds score between those two values. In every seed, candidate 3’s remarriage rate runs 27 to 51 percent higher than the PSID’s. Other gates pass as well, gate m4 among them.
We run the blind tests against DYNASIM3 (Section 5) under the gates’ procedure, with no tolerances. Because DYNASIM3 is another published model and not ground truth, each comparison reports its gaps without grading them.
Over the long horizon, assumptions outweigh the model’s machinery, so we limit what the model claims according to how far ahead it projects. The 2025 Trustees Report puts the 75-year actuarial balance at −3.82 percent of taxable payroll on intermediate assumptions, −0.33 percent on low-cost assumptions and −8.60 percent on high-cost ones (Board of Trustees, Federal Old-Age and Survivors Insurance and Federal Disability Insurance Trust Funds 2025). That low-to-high spread, 8.27 percentage points of taxable payroll, exceeds twice the intermediate deficit. We therefore sort the model’s outputs into three tiers.
Tier 1 covers distributional reform analysis that holds the population and the assumption path fixed. Baseline and reform share much of the demographic uncertainty, and we expect that shared part to cancel when we difference them. No gate measures how much of it does. The tier design calls for labeling the parts that do not cancel, claiming responses among them, and for anchoring claiming-response ranges to quasi-experimental evidence (Mastrobuoni 2009; Behaghel and Blau 2012). Exercise 3’s claiming alternatives rest on no such evidence: they delay claims by the full rise in the retirement age, up to age 70, for projected claimants at 65 or older in one and for every projected claimant in the other. Tier 2 covers near-term outputs: covered earnings, beneficiary counts, claiming ages, average benefits and disability incidence. Because administrative publications report all five every year, we can score each against those publications annually. gates.yaml names gate 3 for this tier but gives it no cells or tolerances, and gate 3 has never run. Tier 3 covers the long horizon. For it, the design calls for sensitivity surfaces over documented assumption ranges, computed on named assumption paths, among them the Trustees’ intermediate path, and presents none of them as the model’s own forecast.
5 Four blind tests against DYNASIM3
Four blind tests set the model’s results against DYNASIM3’s published values. Two project benefit reforms to 2030: exercise 1 cuts each cost-of-living adjustment (COLA) by one percentage point, and exercise 3 raises the full retirement age to 68 (Urban Institute 2010). Two calculate directly on incomes the PSID observed: exercise 2 cuts every benefit by 13 percent (Butrica and Uccello 2004), and exercise 4 adds a minimum benefit (Favreault et al. 2006). The COLA test lands within 0.74 percentage points of DYNASIM3 in every age group. Each of the other three finds a place where the two models differ.
An interactive scorecard at https://microcosm.institute/dynamics/scorecard publishes every cell, each test’s timeline and every registered alternative, a variant that a test’s registration names before the run and that we report beside the test’s main result. We post every registration and every result as a public comment on issue #42 of the repository, https://github.com/PolicyEngine/microcosm-dynamics/issues/42.
5.1 The protocol
In each test, a separate comparison workstream, working apart from the model’s builders, makes two independent readings of DYNASIM3’s published values, transcribing them from tables or digitizing them from figures. It reconciles the two readings and seals the files under SHA-256 hashes, digital fingerprints that change if a file changes. We ratify the model specification and freeze it at a named commit, a fixed version of the code in the repository. A forecaster blind to the sealed values then predicts both the model’s result and its gap as intervals, and we hash the forecast. The registration carries the specification hash, the commit, the primary row that the test reports as its result, the registered alternatives we report beside it, and the forecast. No registered alternative may replace the primary row after the fact. The model runs and writes its output to a file, the artifact, and we score only that output. We check the artifact and commit it, and only then does the comparison lane open the seal. We publish the comparison whatever the outcome. No test has an acceptance threshold.
The hashes show that neither the sealed DYNASIM3 files nor the artifact files changed. Commit times and the recorded step order show that we committed each output before its test’s seal opened. Neither can show whether a participant ever saw a DYNASIM3 value. Each registration discloses its participants’ exposures, and a test is blind only as far as those disclosures show. The four tests ran on 23, 25 and 27 September 2026, in the order 1, 3, 2, 4.
The AI model that coordinated the build, Claude Opus 5.5, saw the digitized exercise 1 values on 22 September, in a report that may also have held the exercise 3 bars. It later wrote builder and reviewer briefs, and registration 13 states that none contains a comparator value or a choice informed by one. On 24 September, the exercise 3 comparison lane found five copies of restricted material from the 2010 report where builders could reach them. The same day, a scratchpad folder the builder agents shared held files whose names suggest Figure 2 material, exercise 1’s comparator reconciliation files, two PDFs the size of the 2004 and 2006 reports, and extracts of restricted pages of the 2004 report. No builder report cites these files, and the registrations cannot rule out that builders opened them. The author of the exercise 2 plan read a page of the 2004 report that includes results prose, and the author of the first exercise 4 plan read one withheld sentence, which describes a table of expenditure paths outside exercise 4’s targets.
| Test | Registration posted (UTC) | Run | Artifact committed | Result posted |
|---|---|---|---|---|
| 1. COLA cut | 2026-09-23 23:06:06 | 23:06–23:31 | 23:40:38 | 23:42:53 |
| 3. Full retirement age 68 | 2026-09-25 22:35:30 | 22:36–23:07 | 23:27:01 | 23:40:59 |
| 2. 13 percent cut | 2026-09-27 05:01:06 | 05:01:49–05:03:29 | 05:06:13 | 05:29:39 |
| 4. Minimum benefit | 2026-09-27 12:18:29 | 12:19:17–12:19:48 | 12:36:32 | 12:58:12 |
5.2 The projections
Both projection tests run the same engine configuration. The engine seeds a closed cohort from the PSID’s 2011 wave and ages it to 2030 in 20 random draws, with 2008-vintage mortality and disability rates. We take COLAs for 2008–2030 and the average wage index for 1975–2030 from the 2008 Trustees Report’s intermediate assumptions. Those assumptions put the COLAs for 2008, 2009 and 2010 at 2.7, 2.5 and 2.8 percent, where SSA’s realized COLAs were 5.8, 0 and 0 percent. The Python oracle computes benefits with a fixed 35 computation years for every worker. The projections draw no earnings after 2010, so the oracle computes each worker’s AIME from earnings for 1968 through 2010 and counts every year before 1968 or after 2010 as zero. Neither projection checks insured status, so any projected claimant whose career through 2010 gives a positive AIME becomes a retired-worker beneficiary, however few years that career covers. For people already retired when the projection starts, the oracle takes their observed 2010 benefit amounts, and for disabled workers it uses approximate benefit levels.
Each test reports the percent change in mean Social Security benefits in 2030 for five groups by age in 2030: 50–61, 62–64, 65–69, 70–79 and 80 and older. We compute the change as a weighted ratio of the reform and baseline means. We measure the model’s uncertainty two ways. The draw standard deviation captures the variation across the 20 projections. For the half-sample floor, we split the sample into two halves that share no person and no family unit, take the mean absolute difference between them, and average it over five random splits. The floor shows how far the estimate moves from sampling alone. Under normality, the expected floor is about 1.6 times the full-sample standard error. The draw standard deviation reaches 0.36 points in exercise 1’s primary row, at ages 50–61, and 0.24 points in exercise 3, at ages 62–64, where the floor is 0.09. The comparison lane digitizes DYNASIM3’s projection values from a figure, so each value carries a digitization interval and no standard error.
5.2.1 Exercise 1: a one-point COLA cut
Exercise 1 reduces each annual COLA by one percentage point, starting in 2010. DYNASIM3’s values come from Figure 2 of Details Matter, which reports DYNASIM3 run 614 on the 2008 Trustees assumptions (Urban Institute 2010).
| Age in 2030 | Microcosm Dynamics | DYNASIM3 | Gap (pp) | Half-sample floor |
|---|---|---|---|---|
| 50–61 | −7.77 | −7.15 | −0.62 | 0.27 |
| 62–64 | −3.64 | −3.79 | +0.14 | 0.34 |
| 65–69 | −6.26 | −6.36 | +0.10 | 0.12 |
| 70–79 | −12.22 | −11.94 | −0.28 | 0.16 |
| 80+ | −17.92 | −18.66 | +0.74 | 0.09 |
Every age group lands within 0.74 percentage points of DYNASIM3, and the mean absolute gap comes to 0.38 points. The two models order all five groups the same way. The gaps at 62–64 and 65–69 are smaller than the half-sample floor, while those at 50–61, 70–79 and 80 and older exceed it.
The largest gap, 0.74 points, sits in the oldest group. The model’s timing convention applies 21 reduced increases by 2030, so nobody aged 80 or older loses more than 18.56 percent. DYNASIM3’s −18.66 percent sits 0.10 points past that bound, close to the figure’s reading resolution of about 0.09 points. A registered alternative that counts the December 2030 increase brings the oldest group within 0.06 points. It also makes every other group’s cut 0.8 to 1.5 points deeper than DYNASIM3’s. No single registered timing convention fits all five groups. Two untested hypotheses could explain the oldest group’s gap: the two models may time COLAs differently for the oldest cohorts, or they may hold different mixes of survivors, including survivors whose cost-of-living increases run from an earlier year. The published figure cannot tell them apart.
The 50–61 group consists almost entirely of disabled workers, and the model’s cut there runs 0.62 points deeper. Two untested hypotheses could account for it: how long the disabled workers on the rolls at the projection’s start have already received benefits, and the 2008-vintage award and termination rates.
Exercise 1 is a weak test of behavioral dynamics: once we fix the year from which each benefit’s cost-of-living increases run, the reform acts mechanically on benefits in payment. The registration lists six alternatives, each changing one convention. An alternative that starts the reduced increases at each beneficiary’s own entitlement year widens the mean gap to 1.47 points, and one built on the 2009 PSID wave gives a mean gap of 0.37.
5.2.2 Exercise 3: a full retirement age of 68
Exercise 3 raises the full retirement age gradually from 2010 until it reaches 68 for people turning 62 in 2022 or later. DYNASIM3’s values come from the “Increase FRA to 68” bars in the same figure (Urban Institute 2010). The test applies the reform to each of exercise 1’s 20 projected paths and holds claiming ages fixed at their baseline values.
| Age in 2030 | Microcosm Dynamics | DYNASIM3 | Gap (pp) | Half-sample floor |
|---|---|---|---|---|
| 50–61 | 0.00 | 0.00 (reading) | 0.00 | 0.00 |
| 62–64 | −4.73 | −5.12 | +0.39 | 0.09 |
| 65–69 | −5.68 | −9.90 | +4.22 | 0.11 |
| 70–79 | −5.24 | −4.15 | −1.09 | 0.07 |
| 80+ | −0.75 | −0.60 | −0.14 | 0.11 |
At 65–69, DYNASIM3’s cut is 1.74 times the model’s, a gap of 4.22 points. That one group carries 72 percent of the mean absolute gap of 1.17 points. Nine of the ten pairwise orderings agree. DYNASIM3’s 50–61 bar shows no visible height, so its zero is a digitizer’s reading, and the drawing’s resolution of about 0.09 points bounds that reading. The model’s estimate is consistent with zero at that resolution.
The 65–69 group covers the 1961–1965 birth cohorts, whose full retirement age rises by 12 months. At fixed claim ages, that rise cuts a retired worker’s benefit by 6.25 to 7.69 percent and a spouse’s excess by at most 10 percent, and it leaves a disabled worker’s benefit where it was. To reach DYNASIM3’s 9.90 percent through those factors, the group would have to consist almost entirely of spouses’ excesses starting at 65. In the model, disabled and converted-disabled benefits make up 17.8 percent of the group’s baseline dollars. DYNASIM3’s 65–69 bar therefore includes more than the age-factor cut at fixed claim ages.
Three untested hypotheses could supply the rest. DYNASIM3’s 2004 primer, which predates run 614, says its benefit calculator checks insured status and applies the retirement earnings test (Favreault and Smith 2004). The primer also says that DYNASIM3’s claiming hazards include Social Security policy parameters, with separate equations by the ratio of recent earnings to the earnings test’s exempt amount, and it aligns employment at ages 65 to 69 to Trustees targets. Exercise 3’s projection does none of these. The first hypothesis is a retirement earnings test extended to 68, which would withhold benefits from people still working. The model pays gross benefits, applies no earnings test and draws no earnings after 2010, so testing this hypothesis requires forward earnings. The second is a claiming response that shifts claims later. The third is an average over all living people in place of recipients alone, which counts a postponed claimant as zero. A diagnostic that applies that averaging to the model’s own claiming alternative moves the 65–69 change to −8.68 percent.
The registration lists eight alternatives, which vary the PSID wave, the phase-in schedule, survivor rules, a claiming response, benefit types and the statistic. None comes within 3.9 points of DYNASIM3 at 65–69.
Exercise 3’s first registration, number 14 on issue #42, stopped before scoring. The test runner’s membership guard, which then halted any run in which the baseline and the reform counted different people as beneficiaries, ended it. By then the run had computed the primary row in memory. It had printed and recorded no value, no artifact existed, and the seal stayed closed. Registration 15 carried the same forecast and produced the scored run, after a reviewed change that makes the guard halt only on membership differences it does not explain.
5.3 The static calculations
Exercises 2 and 4 apply each reform to incomes the PSID observed and count who it affects. They project nothing and model no behavioral response. The model’s figures describe realized incomes, while DYNASIM3’s come from projections its authors published in 2004 for exercise 2 and in 2006 for exercise 4 (Butrica and Uccello 2004; Favreault et al. 2006). A gap in either test can reflect the population’s vintage as well as the two models’ rules.
5.3.1 Exercise 2: a uniform 13 percent benefit cut
Exercise 2 cuts every Social Security benefit by 13 percent from 2004. DYNASIM3’s values come from Tables 19 and 21 of Butrica and Uccello’s How Will Boomers Fare at Retirement?, which report what the authors call adjusted poverty at age 67 for the 1936–45 birth cohort, with and without the cut (Butrica and Uccello 2004). The test reports the change in the poverty rate in percentage points. The model’s sample holds 483 people whom the PSID observes at exactly age 67, born in 1937, 1939, 1941, 1943 and 1945. We attach each person to their family’s income and poverty threshold. The report prints poverty levels as whole percentages, so each printed change carries an open rounding interval of ±1 percentage point.
| Group | n | Microcosm Dynamics | DYNASIM3 | Gap (pp) |
|---|---|---|---|---|
| All | 483 | +0.46 | +2 | −1.54 |
| Women | 249 | +0.35 | +3 | −2.65 |
| Men | 234 | +0.60 | +1 | −0.40 |
| Married | 331 | +0.60 | +1 | −0.40 |
| Widowed | 44 | +0.45 | +2 | −1.55 |
| Divorced | 60 | 0 | +5 | −5.00 |
| Never married | 14 | 0 | +5 | −5.00 |
The cut raises adjusted poverty by 0.46 percentage points in the model and by 2 points in DYNASIM3. The model’s change comes out smaller in all seven groups. Men and married people fall inside DYNASIM3’s rounding interval, and the other five groups fall outside it. Before the cut, 25 sample members are poor, and four more cross the poverty line under it. Each average-weight observation moves the headline by 0.21 points. No divorced or never-married member crosses, so both groups change by zero. Without a crosser, we cannot estimate the uncertainty of either group’s change, however large the group. The model’s poverty levels run lower than DYNASIM3’s in six of seven groups before the cut and in all seven after it.
The model’s adjusted poverty rate before the cut is 3.1 percent, with a standard error of 0.87 points from the PSID’s sampling design. It is 4.1 percent for women, 1.9 for men, 2.2 for married people, 1.7 for widowed, 5.6 for divorced and 14.0 for never-married people. On reported family money income, which keeps reported asset income in place of the wealth annuity, the same sample’s poverty rate is 3.8 percent. The headline change has a design standard error of 0.25 points, and the change for women 0.28. The 34 members whose marital status we cannot classify count in the rows for everyone, women and men, and in no marital row.
The registration lists ten alternatives, each changing one input. They vary the birth years covered, the SSI response, the income unit, asset income, employer pension balances, the poverty standard, the life table and the threshold schedule. Five leave every poverty outcome where it was, and none brings the headline inside DYNASIM3’s interval. The closest covers all ten birth years and gives +0.62. The untested hypotheses include thinner PSID coverage of low-income unmarried people and whole-family income standing in for an individual’s resources. They also include income sources the report does not name and self-reported benefits that may be net of Medicare premiums. A fifth is timing: the model uses incomes realized from 2004 to 2012, while DYNASIM3 projects from the 2002 Trustees assumptions.
5.3.2 Exercise 4: a minimum benefit
Exercise 4 adds a minimum benefit tied to years of work. Its four versions combine price or wage indexing with a standard or generous schedule, and an across-the-board cut finances each one. They are options 2, 3, 4 and 5, and we take option 2 as the headline. DYNASIM3’s values come from Table 6 of Favreault, Mermin and Steuerle’s Minimum Benefits in Social Security, which reports the 2025 shares of beneficiaries aged 62 and older who receive the minimum, for everyone, men and women (Favreault et al. 2006). The model’s sample holds 2,135 Social Security recipients in the PSID’s 2023 wave, with incomes for 2022. The report’s options take effect in 2007. We move every policy date back three years, to 2004, so that the 2022 snapshot covers 19 entitlement years, 2004 to 2022, as DYNASIM3’s 2025 figures cover 2007 to 2025. A registered alternative that keeps the report’s 2007 dates puts the headline share at 6.83 percent. We count a person when their benefit rests on the minimum, through their own record or through a linked spouse’s or survivor’s benefit.
| Option | Group | Microcosm Dynamics | DYNASIM3 | Gap (pp) |
|---|---|---|---|---|
| 2 (headline) | All | 7.75 | 6.3 | +1.45 |
| 2 | Men | 2.36 | 4.8 | −2.44 |
| 2 | Women | 11.67 | 7.6 | +4.07 |
| 3 | All | 11.12 | 11.3 | −0.18 |
| 3 | Men | 3.23 | 8.7 | −5.47 |
| 3 | Women | 16.86 | 13.4 | +3.46 |
| 4 | All | 24.56 | 18.7 | +5.86 |
| 4 | Men | 13.42 | 14.4 | −0.98 |
| 4 | Women | 32.66 | 22.0 | +10.66 |
| 5 | All | 31.47 | 29.3 | +2.17 |
| 5 | Men | 19.24 | 24.4 | −5.16 |
| 5 | Women | 40.36 | 33.2 | +7.16 |
In every option of the primary row, the model’s share runs lower than DYNASIM3’s for men and higher for women. The rows for everyone mask the split by sex: under option 3 the model lands 0.18 points from DYNASIM3 for everyone, yet 5.47 points under it for men and 3.46 points over it for women. Both models rank the four options the same way, and both give women a higher share than men in every option. The mean absolute gap across the 12 cells comes to 4.09 points. The untested hypotheses include the difference between a realized 2022 population and a 2025 projection, the source of each person’s PIA, the completeness of PSID earnings histories, and records whose benefits began as disability benefits. They also include 60 auxiliary beneficiaries whom the model cannot link to any worker.
In the primary row, we build each person’s PIA from the earnings history. A registered alternative that takes the PIA from the benefit the person receives lowers the mean gap to 1.60 points, and under options 2 and 4 it puts more men on the minimum than DYNASIM3 does, at 7.30 and 14.96 percent against 4.8 and 14.4. For the 1,799 records with both measures, the PIA implied by the benefit is 1.11 times the one built from the earnings history at the median and 2.89 times at the 90th percentile. Two more alternatives break the split by sex. One that applies the across-the-board cut after the minimum gives women a lower share than DYNASIM3’s under options 2, 3 and 5, and one that uses a 35-year divisor for disability-origin PIAs gives men a higher share than DYNASIM3’s under option 4. No alternative can replace the primary row, whose mean gap of 4.09 points remains the test’s result.
5.4 What the four tests show and what they cannot
Under the COLA cut, the model lands within 0.74 points of DYNASIM3 in every age group. In the full-retirement-age test, the two models differ by 4.22 points in one age group, 65–69. The model’s poverty response to the 13 percent cut comes out smaller in every group, and in every option of the primary row the model puts a smaller share of men and a larger share of women on the minimum benefit than DYNASIM3 does. Three of the six registered alternatives break that split in at least one option. We score the blind forecasts separately from the comparisons. The forecast intervals contain every projection estimate, and in each projection test one gap falls outside its interval. In the static tests, the forecast intervals exclude the model’s own values more often than DYNASIM3’s.
Each gap measures a disagreement between two models. No test covers DYNASIM4, the current generation of DYNASIM. The tests establish parity with neither DYNASIM3 nor DYNASIM4, and they do not certify the model. Exercises 1, 3 and 4 compute benefits with Python rules, and exercise 2 cuts reported benefits directly. None of the four runs these rules in the Axiom rules engine. We plan to run the full benefit rules in Axiom as separate work (Section 7). The static tests set observed incomes against DYNASIM3’s projections and never touch the projection engine, while the projection tests exercise that engine without forward earnings, a retirement earnings test or behavioral claiming. Because neither the COLA cut nor the higher retirement age is current law, no observed outcome can check either model.
The registered alternatives measure how sensitive the model is to its own conventions and reveal nothing about how DYNASIM3 computes its results. Each explanation we offer for the gaps stays a hypothesis until a registered test addresses it.
6 The PSID’s limits
The PSID supplies the model’s careers, family events and disability transitions, and its design limits what the model can learn from it. Social Security’s benefit formula averages a worker’s highest 35 years of indexed earnings, so the model needs earnings across whole careers. The PSID has interviewed the same families and their descendants since 1968, every year through 1997 and every other year since, and anyone can obtain its data (McGonagle et al. 2012). MINT and CBOLT run on SSA’s administrative earnings records (Section 8), and SSA has a legal obligation to protect the confidentiality of the people in its records (Compson 2011). The PSID is small, it follows a population defined in 1968, it has lost many of its original members, its documented earnings problems concentrate among low earners, and the careers the model builds from it have gaps (Table 7).
Size. The PSID began in 1968 with 4,802 families holding 18,230 people, and in 2023 it interviewed 9,152 families holding 24,536 people (Panel Study of Income Dynamics 2025). The Current Population Survey’s Annual Social and Economic Supplement, the main source of Microcosm’s frame, draws a sample of about 95,000 households each year (U.S. Census Bureau 2024). Moffitt and Zhang count the PSID’s sample size as a weakness, because it “often does not permit much subgroup analysis or distributional analyses” (Moffitt and Zhang 2018). Exercise 2 shows the constraint. Its sample holds 483 people whom the PSID observes at exactly age 67, so each average-weight observation moves its headline by 0.21 percentage points. None of its 14 never-married members crosses the poverty line under the cut (Section 5).
Selection. The 1968 sample joined a nationally representative sample of 2,930 families from the University of Michigan’s Survey Research Center (SRC) with an oversample of 1,872 low-income families from the Census Bureau’s Survey of Economic Opportunity (SEO) (Panel Study of Income Dynamics 2025). Of the families selected in 1968, 76 percent gave an interview (Fitzgerald et al. 1998). The PSID follows everyone who lived in those families and anyone later born to or adopted by them, including when they form new families. People who marry into the sample stay in it only while they live with a sample member (Panel Study of Income Dynamics 2025). The original design did not represent families who arrived after 1968, and it records post-1968 immigrants only while they live with a sample member (Panel Study of Income Dynamics 2025). The PSID later added two immigrant refresher samples: 511 families in 1997 and 1999, too few to analyze immigrant subgroups, and a 2017 sample with 455 completed interviews (McGonagle et al. 2012; Johnson et al. 2018). In 1997 the PSID cut its core sample from about 8,500 families to about 6,300, mostly from the low-income oversample (Panel Study of Income Dynamics 2025). The PSID publishes weights for these unequal chances of selection. It calibrates its cross-sectional weights to population characteristics from the CPS or the American Community Survey for the year in question, and it does not calibrate its longitudinal weights to outside population estimates each wave (Chang et al. 2023a).
The model keeps the low-income oversample and both immigrant refreshers. Shin and Solon leave the oversample out of their estimates because of irregularities in its selection: by the account of a 1996 memo they cite, the firm that chose its low-income households sent the PSID only 55 percent of eligible nonwhite households and 21 percent of eligible white ones (Shin and Solon 2008). Dynan, Elmendorf and Sichel leave it out for the same reason (Dynan et al. 2012). The model’s earnings panel drops the Latino sample, which the PSID added in 1990 and dropped after 1995 (Panel Study of Income Dynamics 2025), because for 1990 to 1996 the panel reads the PSID’s core individual weights, which give those records zero weight (data/panels.py, data/family.py). Every blind test weights its PSID cohort with the PSID’s cross-sectional individual weights, and none recalibrates them.
Attrition. By 1989, 49 percent of the people interviewed in 1968 were still giving interviews, a count that treats deaths as losses (Fitzgerald et al. 1998). By 2015, on a weighted measure that counts only survivors, the PSID estimates that 40.4 percent of the 1968 sample responded: 41.0 percent of the SRC sample and 33.4 percent of the SEO sample (Chang et al. 2023b). Fitzgerald, Gottschalk and Moffitt found that attrition through 1989 “is highly selective and is concentrated among lower socioeconomic status individuals,” and among people with less stable earnings, marriage and migration histories. They also found that the PSID’s cross-sectional representativeness of the non-immigrant population stayed roughly intact over the same years (Fitzgerald et al. 1998). Because the men who stayed had steadier histories than those who left, the authors expected turnover, hazard rates and other second moments to run smaller in the remaining sample than in the population. That instability explains little of who leaves, however, so they judged significant bias in studies of such dynamic outcomes unlikely, and they recommended that researchers check how sensitive their results are to attrition (Fitzgerald et al. 1998). Schoeni and Wiemers find that attrition biases standard estimates of how closely children’s incomes follow their parents’, and that weights built only on earlier characteristics may not correct it (Schoeni and Wiemers 2015).
Low earnings. The PSID’s documented earnings problems concentrate at the bottom of the distribution. From 1968 to 2007, PSID and CPS family income distributions match fairly closely between the 5th and 95th percentiles and less well in the top and bottom 5 percent (Gouskova et al. 2010). Many of these problems trace to the PSID’s move to computer-assisted telephone interviewing and new processing software in 1993 and 1994. PSID staff warned that the change could leave a “double seam” in the income data, and the PSID lacked the staff to re-edit the 1990–1997 data with the editing system it adopted in 2000 (Kim et al. 1999; Kim and Stafford 2000). The share of household heads recorded with zero labor earnings despite working more than 120 hours in the year jumped right after income year 1991, stayed high through 2002 and fell back in 2004. Dynan, Elmendorf and Sichel judge it a change in measurement, not in the economy (Dynan et al. 2012). Item nonresponse for heads’ and wives’ labor income rose from about 2 percent in 1980 to over 6 percent in 2001 and 2003 (Killewald et al. 2011). The PSID’s earnings imputation rates run 3 to 4 percent, against recent rates of about 45 percent in the CPS and 49 percent in the Survey of Income and Program Participation (Moffitt et al. 2023). In its 2007 wave the PSID filled any missing wage it could not impute from job records or the prior wave with the median wage (Duffy 2011).
These problems run through the debate over whether American earnings and incomes grew less stable. Hacker and Jacobs, who report that short-term family income variance roughly doubled from 1969 to 2004, acknowledge that the PSID departs from the CPS at the bottom of the income distribution for roughly five years in the mid-1990s, and they find it representative again by the decade’s end (Hacker and Jacobs 2008). Shin and Solon find little rise in men’s earnings volatility through most of the 1990s, a reading they note grows stronger if one discounts observations drawn from the 1993 and 1994 interviews (Shin and Solon 2008). A 2008 Congressional Budget Office report, which links survey data from the Survey of Income and Program Participation to SSA’s earnings records, finds the variability of individual earnings roughly flat since the mid-1980s and repeats Dynan and coauthors’ caution about the PSID’s early-1990s data (Congressional Budget Office 2008). Gottschalk and Moffitt, by contrast, conclude from cross-sectional comparisons with the CPS that the procedural changes had little effect on trends in the PSID’s income distribution (Gottschalk and Moffitt 2009). Their comparison covers distributions within each year, while the zero-earnings reports Dynan and coauthors flag show up across a person’s consecutive years. PSID staff judged that the income series kept its basic continuity, and they read the PSID’s lower incomes at the bottom as possibly more accurate reports of poor families’ circumstances (Kim and Stafford 2000). Moffitt and Zhang, examining attrition, item nonresponse, imputation and other features of the PSID, find no evidence that any of them significantly biases its trends in men’s earnings volatility (Moffitt and Zhang 2020). A reconciliation of six survey and administrative sources on men’s earnings finds no strong overall trend in volatility among working men since the mid-1980s, and it traces most of the small remaining differences in trends across sources to the left tail of earnings. The PSID has the smallest left tail of the six: about 5 percent of its observations have earnings under $20,000 a year, against about 25 percent in the Census Bureau’s Longitudinal Employer-Household Dynamics records. The most common hypothesis for that gap, the authors write, is that survey respondents often fail to report earnings from short, part-year jobs (Moffitt et al. 2023).
The model reads the PSID’s labor income as the PSID records it, imputations included, takes reported zero earnings at face value, and corrects none of these problems. It also carries a break that its own code documents: through income year 1992, heads’ labor income includes the labor part of farm and business income, which the PSID records separately from 1993, so earnings windows that straddle 1992 and 1993 mix two concepts (data/family.py). No gate tests for these problems. Gate 1 scores zero-earnings spells and transitions against the PSID’s own records, so it inherits any error in them. Its benefit-space block scores deciles 3 through 9 of a benefit proxy and, pooled across seeds, the group with no earnings in its anchor year. It only reports the bottom two deciles, because there the discrepancy between two halves of the real PSID alone exceeds the block’s 5 percent band. Gate 1 also scores only the years 1998 to 2022 at ages 25 to 59 (gates.yaml, gate_1). Recorded earnings before 1997 cover most of the years in the 1968–2010 careers the projection tests read, and the only gate that scores them is gate 2c, which reads them for married couples: who marries whom by earnings and how earnings move around marriage and divorce (gates.yaml, gate_2). A sensitivity we report but do not gate drops the 9,286 of 126,867 person-years in gate 1’s earnings view that the PSID flags as assigned, and the ten-year autocorrelation of log earnings moves from 0.539 to 0.550 (runs/imputation_sensitivity_v1.json). Exercise 2’s untested hypotheses for its gap include thinner PSID coverage of low-income unmarried people (Section 5).
The model’s career construction leaves gaps of its own. Since the move to biennial interviews, the PSID’s income questions center on the calendar year before each interview, with at least minimal questions about the “off” year before that (Panel Study of Income Dynamics 2016). The model’s main earnings panel reads only the prior-year amounts, so after 1996 it observes earnings every other year and measures two-year changes, never one-year ones (data/family.py). Exercise 4 alone reads the off-year reports, for 2001 to 2021 (Section 3). For the 7,486 members of the 2011-wave cohort who reach 62 between 1979 and 2030, about half of the career years through 2010 hold values we imputed or set to zero (Section 3). The imputed years are the odd years from 1997, which the model fills from neighboring years. The years set to zero include years in which the person lived in no responding PSID family, odd years with no recorded neighbor, and income years through 1995 in which the PSID recorded the person’s earnings as a family head or spouse but gave the person no weight, as it did for people who married into a PSID family. They also include years in which the person lived in a responding family as neither its head nor a spouse, because the model reads labor income only for heads and spouses (estimates/career.py, data/family.py).
6.1 What the model does about each limit
The model addresses these limits in two ways, and we have begun a third step: a check against Social Security’s own earnings records, which has compared the PSID’s recorded earnings with them and scored the model’s rules for filling gaps in recorded careers, without yet scoring its generated earnings or whole careers. None removes the limits.
Transport onto Microcosm’s calibrated frame. The model is built to project from Microcosm’s frame in place of a PSID sample, so that the frame supplies the population’s size and makeup and the PSID supplies the transitions. Microcosm calibrates the frame’s household weights to administrative totals, among them IRS income statistics, Social Security payments by benefit type and Census population counts by age (PolicyEngine 2026a). Gate w1 applies the processes the model fits on the PSID to every adult in the frame, seeding each adult’s marital history from the frame’s own married share at ages 25 to 34, and checks 43 cross-sectional cells of the result against the frame: earnings by age and sex, marital shares, spouse coresidence and one household-size share. Gate w1 scores its marital cells at ages 25 to 34 against the same frame values the seeding reads, so those cells are in sample (models/transport_deployment_v3.py). The gate reports the rest without scoring them, and it certifies no transition (gates.yaml, gate_w1). Any bias in the PSID’s transitions, such as the understated turnover that selective attrition can produce, therefore survives the transport. No projection starts from the frame yet; the four blind tests each use a closed PSID cohort with the PSID’s own weights.
Gates scored on the PSID. The gates that score the model against the PSID set most of their tolerances from the discrepancy between two disjoint halves of the real PSID panel (Section 4). A smaller panel produces a wider floor and a looser tolerance. Gates 2, 2b, 2c, m4 and m6 demote to report-only any cell whose weaker half holds fewer than 20 events or whose tolerance would exceed a cap, ln(1.5) for most cells and 0.15 for gate m6’s earnings correlations (gates.yaml). These rules price the PSID’s sampling noise. They cannot detect a bias that both halves share, because both halves come from the same selected and attrited panel: such a gate tests whether the model reproduces the PSID, and leaves open how far the PSID departs from the population. Gate m6 scores only the people the PSID observed in each later wave, treats the PSID’s record of deaths above age 85 as confounded by attrition, and certifies nothing about mortality drift (gates.yaml, gate_m6).
A check against administrative earnings records, begun. We plan to set the earnings careers the model reads from the PSID, and those it generates, against the Social Security Administration’s 2006 Earnings Public-Use File. We have compared the PSID’s recorded earnings for 1998 to 2004 with it and, in a separate registered test described below, scored our rules for filling gaps in recorded careers on its held-out records. The file holds annual earnings records for 1951 through 2006 from a 1 percent random sample of all Social Security numbers issued before 2007, 4,384,254 people in all (Social Security Administration 2011a, 2011b).1 Its records come from the earnings posted to workers’ Social Security records, so they do not depend on anyone continuing to answer a survey. They hold only earnings up to each year’s taxable maximum from jobs Social Security covers, with wages and self-employment combined, and the file records each person’s birth year and sex and nothing about marriage, children, disability or death (Compson 2011). The file follows each person across years, so it can test change over time as well as single years. Moffitt and Zhang report that the PSID’s earnings data have never been matched to government administrative records (Moffitt and Zhang 2020). The comparison would run by birth year and sex and cover the share of years with no covered earnings, the share of low-earnings years, movement into and out of covered work, and how persistent a worker’s position stays over time. Selective attrition and weak low-end reporting would distort these statistics, and the PSID’s own halves cannot reveal the distortion. The comparison has to put the model’s earnings on the file’s terms, capped at each year’s taxable maximum and limited to covered work. It also has to put the two populations on common terms. The file records no deaths, so a run of years without earnings can mean retirement, disability or death (Compson 2011), and its sample of Social Security numbers includes the post-1968 immigrants the PSID largely leaves out. The file’s own low end is coarse: SSA replaces every amount under $100 with that year’s average and rounds amounts from $100 to $1,000 at random to a base of $25 (Social Security Administration 2011b; Compson 2011). We registered the rules for one part of this comparison before computing any PSID value on the file’s terms (docs/amendments/gate_epuf_registration_proposal.md). That part covers the generated earnings, which overlap the file only in the even years from 1998 to 2004. On that overlap only two cells met the rules for gating, we have not shown that they would catch anything gate 1 does not, and their bite check fell short of its target. The registration therefore does not lock and gates no cell, and we report how the PSID’s own earnings compare with the file, without a pass or fail. A second registered test, gate_epuf_fill, scores how we fill the years recorded careers leave out. It holds back a fifth of the file’s people, 878,481 in all, hides the years the PSID would not record for them, the odd years from 1997 to 2005 and every year before age 22 or before 1968, whichever is later, fills those years and compares the fills with the file’s records (src/populace_dynamics/harness/epuf_fill_gate.py, runs/epuf_fill_gate_test_v1.json). The rules the blind tests use fail it: filling each odd year from its neighbors fails 100 of 183 cells, and setting the years before a career to zero fails 131 of 136. A learned pre-career fill passes all 136. A learned odd-year fill fails 8 of 183, so the test adopts it as an improvement without certifying it. Both learned fills are optional, and no blind test uses them yet. We have not yet scored the model’s generated earnings against the file, and we have not compared whole careers with it.
| Limit | What it does to the model | What the model does today | What remains open |
|---|---|---|---|
| Size: 4,802 families in 1968 and 9,152 in 2023 | Sampling noise in fitted transitions and in small groups; exercise 2 scores 483 people, 14 of them never married | Gates scored on the PSID set tolerances from its own sampling noise and leave underpowered cells report-only; gate w1 places the PSID’s processes on Microcosm’s frame, which no projection starts from yet | Sampling error in the fitted transitions |
| Selection: families of 1968 and their descendants, two immigrant refreshers, a 1997 cut that fell mostly on the low-income oversample | Transitions learned from a population defined in 1968 | The PSID’s weights; on Microcosm’s frame, gate w1 checks the cross-sectional makeup | Transitions of groups the PSID covers thinly |
| Attrition: 49 percent of 1968 members still interviewed in 1989; 40.4 percent of survivors responding in 2015 | Selective attrition can understate turnover and hazard rates | The PSID’s weights; gate m6 scores only people the PSID observed | Checking the model’s earnings and careers against SSA’s earnings file |
| Low earnings: interviewing and processing changes from 1993, zero-earnings reports through 2002 that Dynan and coauthors attribute to measurement, rising item nonresponse, the smallest left tail of six data sets | Low earners and zero-earnings years carry the most measurement error, and minimum-benefit and poverty results depend on them | No correction; gate 1 scores zero-earnings dynamics only against the PSID’s own records and leaves the bottom two benefit-proxy deciles unscored | Checking the model’s earnings and careers against SSA’s earnings file |
| Gaps in recorded careers: biennial interviews since 1997 | About half of the career years through 2010 for the 7,486 cohort members who reach 62 between 1979 and 2030 hold imputed or zero values | Fills the odd years after 1996 from neighboring years and sets other missing years to zero; both rules fail gate_epuf_fill | Whether to rerun the blind tests with the learned fills; checking whole careers against SSA’s earnings file |
7 What remains to build
Forward earnings are the largest piece the projection tests still lack. The forward earnings law passes its own gate, m6, but fails B2, the registered test that carries it forward from 2010, and no projection test uses it yet. To compare the model with DYNASIM3’s reform results, we need projections that begin in 2010, end in 2030 and turn on people aged 60 to 69. The law does not yet reach those people in their sixties. Ages 60 to 64 sit inside its fitted range of 25 to 64, but the projection cohort starts in 2010, four years before the law’s 2014 boundary, and above 64 the law has no fitted distribution. We plan first to connect forward earnings to the projection tests, and then to build the benefit mechanics that depend on earnings. Each step is a design that joins the model only after it passes a dedicated registered gate, and none has joined yet. Several pieces already exist, all merged into the repository: the retirement earnings test’s parameters and annual ledger, the gross-benefit layer described below, and the code for a planned rerun of both projection tests.
Forward earnings in the projection. We carried the earnings state of a 2010 cohort forward to 2014, the law’s fitted boundary, and a registered transport test, B2, scored those states against earnings for 2012 and 2014, which the PSID collected in its 2013 and 2015 interviews. We had already used those earnings years to select the model, and B2’s registration said so. B2 failed (runs/track_b_b2_score.json). For 2012 and 2014 together, the projected zero-earnings rate for people aged 25 to 44 at the interview that collected those earnings is about 35 percent above the PSID’s, beyond the test’s tolerance of about 22 percent. The gap exceeds that tolerance in all five household splits the test uses. Of the four statistics B2 gated, only the 90th percentile of positive earnings at ages 25 to 44 passes every check. Nothing that depends on B2 may use it, and we will not rerun it, draw new seeds or loosen its tolerances. We are designing a successor law whose changes must rest on evidence other than B2’s result, and we will register it before it runs. If two successors fail, work on this mechanism stops until new evidence arrives (docs/design/track_b_attempt_ledger.md). The rest of the plan waits on a successor that passes. We would fit a second version of the law on a wider age range, 25 to 69, with one gate for each of three age bands: 62–64, 65–66 and 67–69. Any 2030 claim that rests on forward earnings would need evidence from a planned backtest that uses 2006 as its boundary year and spans PSID earnings years from 2006 to 2022. That backtest could not support a 2030 claim alone, because those same waves already shaped our modeling choices. We would record three separate monthly series from 2010 through 2030: labor earnings, covered earnings and the earnings that count under the retirement earnings test.
The retirement earnings test. The earnings test’s parameters agree with SSA’s published history. An annual ledger, now in the repository, applies the test to one worker’s record: it charges excess earnings against benefits, tracks provisional suspension and final deduction, and applies a grace-year decision that a separate helper makes, and its tests check it against SSA’s worked examples (track_b/ret_ledger.py, track_b/ret_grace.py). Two separate modules make the adjustment at full retirement age for the months the test withheld and recompute benefits when later earnings arrive (track_b/arf.py, track_b/recompute.py). The ledger refuses cases in which an auxiliary beneficiary’s earnings deduction would free room under a binding family maximum, and no projection applies any of it yet. One of three untested hypotheses for exercise 3’s largest gap, 4.22 percentage points at ages 65 to 69, is an earnings test extended to age 68. Once we combine the earnings-test machinery with fixed claiming, we could test that hypothesis for the first time. That test also needs forward earnings past 2010, which wait on B2’s successor.
Gross benefits and dual entitlement. A gross-benefit layer, now in the repository, computes the family maximum, age reductions, dual-entitlement reductions and the monthly family payment for one worker’s record in one month (track_b/gross_benefits.py). Its tests check it against SSA’s worked examples. Axiom had no rule for the family maximum when we built the layer, so no Axiom check has run. No gate scores the layer, it refuses some family configurations, and no projection uses it yet. A separate rerun, which we plan to register, would repeat both projection tests, exercises 1 and 3, with statutory computation years for disability benefits and with dual entitlement for disability and spouse benefits. We have merged its code into the repository, and so far we have tested it only on invented data. It has no registration yet, and because it comes after the published results, we label it post hoc.
Claiming. The engine draws claim ages from a fixed distribution, and no claim age responds to a reform. We plan to replace that distribution with a monthly claiming hazard, which we would estimate on the insured population that has not yet claimed. We would also add a claiming response identified from policy variation, but only if the data we can access permit a credible design. Even then, to model a claiming response to a full retirement age of 68, we would have to extrapolate past every policy anyone has observed.
Family and population in the projection. The family models pass their gates, 2, 2b and 2c, but none runs inside a projection test yet. We plan to combine the gated marriage and divorce models with the widowhood that a spouse’s death already triggers in the projection tests. We would redraw initial states and declare which populations the projection covers. A registered projection gate would score the result, and a joint validation would check the earnings, claiming, disability, payment and survivor pathways together. In the projection, entrants, both births and immigrants, stay report-only until we admit into the model a way of setting starting states that supports them.
Axiom execution. Axiom reproduces the Python oracle’s AIME for all 7,486 people we tested. On 240 synthetic workers who become eligible in 2020 or 2026, it also matches the oracle’s PIA to the cent (runs/pia_cross_engine_v1.json). On the PSID cohort we have compared nothing beyond AIME. Axiom’s Social Security encodings await acceptance, which must come before the full benefit rules can run in Axiom. We plan to extend Axiom from AIME onward through the PIA, auxiliary benefits and the earnings test.
Wealth, pensions and the long horizon. The model reads PSID wealth and annuitizes it for income measures, and it projects no wealth. The projection tests end in 2030. Any projection we run well past that year carries one more limit: gate m6 certifies nothing about mortality drift.
Held-out targets. We plan two more blind tests under the same protocol as the four existing ones. The first covers the 1946–55 birth cohort from the 2004 report behind exercise 2 (Butrica and Uccello 2004), and its DYNASIM3 values already sit under the existing seal, away from the model’s builders. For this test, we would carry the PSID income and wealth mappings forward into the 2015–2023 waves. The second covers the progressive price indexing option, from the 2010 figure behind exercises 1 and 3 (Urban Institute 2010). The report behind that figure omits the option’s operational formula, so we would need to register conventions for it. We will run no blind test that needs a projected population until the planned projection gates pass.
Comparisons beyond DYNASIM. We plan three comparisons with sources outside DYNASIM. Registration 18, posted on issue #42 on 30 September 2026, covers the one-point COLA cut in the first two, against OACT, PWBM and MINT5. It has not yet run, and no other part of the three comparisons is registered. The first sets the change in total 2030 benefits, from a projection re-timed to each provision’s start date, against the 2030 cost effect that OACT and PWBM each publish for the same provision texts (Social Security Administration, Office of the Chief Actuary 2026a; Penn Wharton Budget Model 2026b). OACT’s one-point COLA cut from December 2027 is the primary row. OACT’s two schedules that raise the FRA to 68, starting with people who turn 62 in 2027, are secondary rows. OACT prints its effects as percentages of taxable payroll to two decimal places, which suffices for the COLA row. By 2030 the FRA schedules reach only people who turned 62 in 2027 or later, so their effects may not register at two decimals. PWBM’s CSV files serve as the primary comparator for those rows; the one PWBM file we checked for precision, from a payroll-tax provision, stores its values at full precision. Arithmetic on the COLA text alone should land close to OACT’s and PWBM’s values for that row, because the cut gives everyone eligible before 2028 the same three reduced increases, those of December 2027, 2028 and 2029. We plan to report that calculation beside the model’s result. The second sets the model’s 2030 benefit changes by age against SSA’s archived MINT5 tables for a one-point COLA cut and an FRA of 68, against MINT8’s FRA of 68 and against PWBM’s two FRA-68 grids (Social Security Administration, Office of Retirement Policy 2011b, 2011a; Social Security Administration, Office of Research, Evaluation, and Statistics 2024a; Penn Wharton Budget Model 2026b). Each of these needs a projection run re-timed to that source’s schedule. MINT5’s COLA cut starts in 2011, against the increase effective December 2009 (first paid in January 2010) in the model’s primary row. MINT5’s FRA reaches 68 for people turning 62 in 2038, against 2022 in the model. MINT reports medians where the model reports a ratio of means. The MINT8 and PWBM schedules start in 2024 and 2027, so by 2030 they reach only the youngest beneficiaries. PWBM’s grids cover ages 62 to 75 in 2030, and their caption says some changes before 70 can come from delayed claiming, while the model’s primary FRA row holds claiming fixed. The same set adds MINT8’s reconfigured minimum benefit by sex, whose share of beneficiaries with a higher benefit is only a proxy for the share receiving a minimum. The third uses CBO where its precision allows (Congressional Budget Office 2015, 2024). CBO’s 2015 workbook stores every value as a percentage of GDP to one decimal place, so its FRA-68 option can check at most the direction of the 2030 change. CBO’s 2024 FRA-70 option enters at its fiscal-2030 column, which sets a fiscal year against a calendar year. Every one of these tables is public, so these tests can be blind only in procedure: we plan to seal each comparator’s pages from the people who build the model’s runs. Agreement would bear on the composition of 2030 benefits and on COLA mechanics. It would not test long-run dynamics. The model’s repository also holds one check against a source outside DYNASIM that we registered but did not run blind. It sets the model’s cost ordering of four benefit reductions against CBO’s 2005 estimates as Mermin (2005) reports them, and our registered ordering forecast did not hold. The committed artifact labels those CBO estimates as DYNASIM3, and an erratum in the repository corrects the label (Ghenis 2026a).
References
Footnotes
SSA’s article introducing the file gives 4,348,254 in three places and 4,384,254 in another; its own count of the underlying sample less the records it removed yields 4,384,254 (Compson 2011).↩︎