Medicare-Synth treats fake data as research infrastructure: small enough to inspect, controllable enough to experiment with, and large enough to expose computational failure.
The Pipeline Passed Locally and Failed on the Data Structure
Consider an illustrative team preparing an emergency-department-to-admission cohort before its final run in a restricted Medicare environment. The definition sounds ordinary: identify emergency care, connect it to a qualifying inpatient admission, then follow the beneficiary into post-acute care. The analyst needs enrollment information, outpatient facility claims, professional carrier claims, inpatient claims, and perhaps skilled nursing or home-health records.
The team has a local fixture, so it starts building. One outpatient row stands in for an emergency visit. One inpatient row stands in for the admission. The join is clean. The episode count matches the expected answer. Unit tests pass, and the cohort code appears ready for the larger study.
The fixture has done almost nothing to test the part of the problem that makes Medicare claims work difficult. In a claims-shaped dataset, the facility record can have multiple lines. A professional carrier record can describe services around the same encounter. An admission might require a separate temporal rule. Enrollment can determine whether the person was eligible at the time. A single join that was harmless in the local CSV can multiply records and change the number of apparent episodes.
When the pipeline reaches a more complete extract, the code still runs. The cohort is wrong. The defect is not a missing semicolon or an unfamiliar file format. The local test data omitted the relationships in which the incorrect assumption lived.
That failure has practical consequences. Debugging inside a restricted environment is slower to share, harder to reproduce, and often occurs after important choices about files, cohort rules, and system architecture have already been made. A team may be able to repair the query, but it has lost a chance to make the structure of the work explicit earlier.
Synthetic Medicare data is useful in this setting precisely because it can make those relationships available before the final restricted data arrives. Its usefulness does not depend on a broad claim that the rows stand in for the Medicare population. It depends on whether the artifact contains the feature that a proposed analysis, method, or implementation needs to confront.
“Realistic” Does Not Name the Requirement
“Realistic synthetic data” sounds like a clear request until a team tries to decide whether a particular file is fit for a particular job. The phrase can refer to plausible ages and diagnoses, familiar claim layouts, a longitudinal pattern of use, the behavior of a treatment-effect estimator, or the memory pressure of a multi-table join. Those are different targets. A dataset that helps with one can be a poor substitute for another.
That distinction matters especially in Medicare work because the analysis often lives in relationships rather than in single fields. A beneficiary’s enrollment history can decide cohort eligibility. A claim header and its lines determine the unit counted by an algorithm. Service dates decide whether a sequence is possible. A professional claim, a facility claim, and an admission can each be part of one care episode without being interchangeable rows. An extract with plausible marginal distributions can still be a weak environment for testing those operations.
The useful question is therefore narrower than whether a release looks like Medicare data in the abstract:
Which structures, mechanisms, and workloads must this artifact preserve for the decision it is meant to support?
Five forms of fidelity help make that question concrete.
Structural fidelity concerns tables, grain, keys, and cross-file relationships. It asks whether a carrier claim line has an intelligible parent claim, whether the beneficiary identifiers can be joined, and whether a code path encounters the same one-to-many relationships it will face later.
Temporal and administrative fidelity concerns the rules through which care becomes claims data. Enrollment, service, admission, discharge, death, coverage, care setting, and code compatibility all enter here. A service date can be a valid calendar date and still be incompatible with the surrounding beneficiary history.
Scientific controllability concerns an experiment’s assumptions. A methods researcher may need to define a target population, vary measurement error, record a generator-defined outcome, or hold a confounding mechanism fixed across repeated runs. Those capabilities make a simulation interpretable. They do not establish that the simulation represents the Medicare population.
Computational fidelity concerns the shape of work presented to an implementation: beneficiaries with many claims, claims with multiple lines, repeated observation over years, concentrated providers, and skewed utilization. Total row count is only one part of that shape.
Distributional and inferential fidelity concerns resemblance to a target population and the claims that can be made from the output. It is the dimension most likely to be understood as “realism,” and it is also the one that requires the strongest empirical evidence.
The dimensions are related, but they should not be collapsed into a single grade. A ten-beneficiary fixture may be visibly artificial and still be the best possible test for a post-death claim or an orphaned line. A very large table can exercise an execution engine while offering little support for descriptive or causal conclusions. A careful methods simulation can specify its own truth without claiming that the mechanism was calibrated to national Medicare experience.
This decomposition changes the work that follows. Research engineers can ask which keys, cardinalities, and temporal rules must be exercised before an analysis reaches a restricted environment. Biostatisticians can identify which parts of an estimand’s data-generating process are fixed by design and which remain only approximations. Health-policy researchers and data scientists can distinguish a test artifact from a dataset that supports an empirical estimate. The data does not need to satisfy every task at once. It needs a declared purpose and a defensible fidelity profile.
CMS Records and CMS Documentation Answer Different Questions
CMS has long provided synthetic claims resources for development and training. The SynPUF program was designed to let analysts and software developers learn the structure and variable names of Medicare claims files without gaining access to beneficiary records. CMS is equally direct about the limitation: synthetic files have very limited inferential value for conclusions about Medicare beneficiaries.
The newer Synthetic Medicare Enrollment, Fee-for-Service Claims, and Prescription Drug Event collection remains an important public reference. Its user guide describes a linked, claims-shaped collection with beneficiary enrollment and multiple claim families. For a developer, that is far more valuable than a generic tabular simulator. It supplies recognizable file grain, variables, and relationships that an implementation needs to handle.
It still does not supply every artifact a research program might need. A public synthetic release can be too small for a workload study, too fixed for a sensitivity analysis, too old for a given schema question, or too broad for a five-row regression test. The right response is not to discard the baseline and generate every table independently. It is to preserve the baseline’s role, record how an extension was made, and be candid about which evidence changes at each step.
Documentation has a different role. A CMS or ResDAC definition can establish that a field exists, name its intended meaning, and describe an allowed value set. It may leave implementation decisions unresolved. Documentation alone may not decide a physical type, a null representation, whether two tables share a key, which arithmetic identities should hold, or how a generator should produce a missing value.
Take a service date. Its documented definition supports an important semantic claim. An executable schema must add operational claims: which record carries the date, which other dates it must precede or follow, whether it is allowed after a recorded death date, and how it relates to enrollment or admission. Those claims need review. Treating a documentation page as executable truth hides the engineering decisions that actually determine whether a release is valid.
Medicare-Synth turns that distinction into a release practice. The project pins CMS source manifests and the versioned ResDAC Knowledge Base evidence snapshot (rkb-v1.0-20211231.json) that enforces variable definitions, types, formats, and valid value sets against the 2021 baseline. Schemas, provenance statuses, assumptions, and validators live in the project’s own domain model, separate from the source releases. The 2022 additive path imports a bounded CMS PUF slice of beneficiary and carrier files only, invoked as:
|
|
That command does not silently extend the 2021 context into a universal contemporary claims environment; it produces a bounded, evidence-stamped slice that a reader can inspect against its own manifest. This separation gives a reader a way to ask where a value or rule came from: preserved source material, normalized representation, derived value, synthesized extension, re-keyed record, or scenario-generated anomaly.
The resulting system resembles a compiler more than a population simulator. Source records and documentation are inputs. Executable schema and relationship rules are the intermediate representation. Fixtures, expanded workloads, validation findings, and release disclosures are the outputs. The analogy has a limit: a compiler can establish that an artifact follows declared rules; it cannot establish that the rules constitute a nationally representative model of care.
The diagram separates what the system can validate from what a future generator might model. A release can be structurally coherent and fully reproducible while its scientific mechanism remains intentionally simple. Keeping those layers distinct prevents provenance from becoming a vague assurance word.
Events and Relationships Come Before Dependent Columns
The hard part of a claims-like generator is seldom making a field look plausible in isolation. The hard part is deciding which entities and events exist before their administrative representations are filled in.
Consider a beneficiary who arrives at an emergency department. The care episode might create an outpatient facility record containing emergency revenue-center lines, a carrier claim for professional services, perhaps an ambulance record, and then an inpatient admission. Some of those records can be absent in a given case. Some can have multiple lines. Their dates should follow a defensible order. They should attach to one beneficiary, and an episode algorithm must be able to find them without manufacturing extra episodes through an accidental join.
The event is the organizing object. Its records are observations of that event through different administrative surfaces.
An event-first design begins with the beneficiary and the relevant enrollment interval. It then establishes the encounter, the participating setting or provider, the possible downstream admission, and the required claim relationships. Only after those choices are fixed does it fill dependent values such as procedure codes, modifiers, units, or payment components. The ordering is important because those values are conditional on an already coherent record structure.
|
|
Within a table, dependencies still matter. A place of service may constrain the procedures that can appear. A procedure and unit count help determine which amount fields are meaningful. A payment component belongs downstream of the claim and line decisions that give it context. That is a table-local dependency graph:
|
|
The two graphs solve different problems. The first graph establishes entities, cardinalities, keys, and temporal relationships across the dataset. The second generates attributes within an already chosen record. Reversing the order leads to data that can pass simple column checks while failing when records are joined.
Independent table generation creates several familiar failure modes. A facility record and a professional record can refer to the same apparent visit but disagree on dates. A claim can attach to a person with no qualifying enrollment. An inpatient admission can precede the encounter that allegedly caused it. Claim lines can have no header, or headers can have totals unrelated to their lines. A downstream cohort query can count one episode several times because the local fixture never included real multiplicity.
These failures are not minor data-cleaning inconveniences. They change the unit of analysis. A pipeline that assumes one outpatient row per visit might return a tidy answer in a handmade CSV and a distorted episode count once it encounters line-level records. The right repair is not to add a random extra row. The fixture needs the relationship that makes the assumption visible.
This is the core architectural argument in Medicare-Synth. Its executable model identifies beneficiary, claim, enrollment, and table-grain contracts; its validator checks field, record, relational, temporal, administrative, and accounting conditions; and its scenario compiler creates small named datasets whose expected outcomes are known. The project records the intended generation order in its architecture: relation-level entities and events first, table-local columns afterward.
The valid_baseline_cohort scenario makes that order concrete. Beneficiary BENE_001 exists in the summary table before any claims are generated. An outpatient record (CLM_O001) establishes an emergency-adjacent visit on January 10, 2021, followed by a carrier claim (CLM_C001) for professional services on March 1–5, and then an inpatient admission (CLM_I001) admitted May 10 with DRG 291 and a five-day length of stay. The downstream SNF stay (CLM_S001) begins June 1, after discharge. Running the full validator over that slice looks like this:
|
|
Because the scenario is built in generation order, the validator finds no orphaned claims (rule REL-001), no temporal inversions (TMP-001: CLM_FROM_DT ≤ CLM_THRU_DT), and no post-mortality service dates. A reviewer can trace BENE_001 from enrollment through each administrative surface and confirm the expected output without touching a restricted file.
The distinction between a valid scenario and an anomaly scenario is sharpest in negative testing. A deterministic scenario can deliberately contain an orphaned claim, a temporal inversion, or a service date after a recorded death. The desired result is not a plausible population table. It is an expected validator finding or a known behavior in the downstream pipeline. Such scenarios are valuable because a bug has a name, a small reproducer, and an assertion that can run in continuous integration.
The project also identifies a boundary between the relational model and table-local feature synthesis. Its current TabDatSynthAdapter is a deterministic built-in adapter that adds seeded synthetic attributes; it is not evidence of a fitted external synthesis model. That limitation is worth stating directly. The architectural boundary is useful because it gives future conditional generators a place to attach without granting them authority over keys, events, or temporal constraints.
For a research engineer, the practical rule is simple: write fixtures around the relationship that can invalidate the algorithm, then generate the dependent fields. For a methods researcher, the same ordering prevents an experiment from treating administrative records as independent observations when the method eventually depends on their shared history.
A Fixture, a Methods Population, and a Benchmark Answer Different Questions
Once the relation graph is explicit, it becomes easier to see why a single frozen “synthetic Medicare dataset” is an awkward target. Different research tasks call for different artifacts.
The smallest artifact is a deterministic fixture. It may include only a few beneficiaries, a few claims, and one intentionally awkward condition. Its value comes from inspection and an expected result. A fixture can tell a team whether a query rejects a claim after death, whether a carrier line requires a parent claim, whether an episode definition handles a same-day facility and professional record, or whether an enrollment exclusion is applied before the outcome is counted.
The next artifact is a tailored methods population. Here the question is not only whether an implementation runs. The researcher needs to state who can enter the study, how exposures and outcomes arise, and how those underlying events appear in claims-like data. A heart-failure study illustrates the difference.
Suppose the target population is beneficiaries with a recent heart-failure admission, continuous Parts A and B enrollment, no Medicare Advantage enrollment during a defined lookback, multiple chronic conditions, and follow-up until death, coverage change, or study end. That statement is already richer than a one-row simulation. Eligibility requires longitudinal enrollment. The index event is defined through an administrative record. The follow-up clock can be censored by information that belongs in another table.
The scientific process adds another layer. A researcher might specify baseline severity, a treatment decision, an outcome, and a confounding mechanism. The care process determines when people seek care or receive treatment. The administrative observation process determines which parts of that history become enrollment fields, claim settings, diagnosis codes, procedure lines, and incomplete measures of comorbidity.
|
|
A conventional one-row simulation remains useful when the goal is to isolate an estimator under a clean, transparent data-generating process. It is often the right choice for a theoretical comparison. Its limitation becomes relevant when the proposed method must also survive enrollment-derived eligibility, code-based exposure definitions, repeated claims, irregular observation, or a follow-up period that ends for administrative reasons.
For example, a phenotype study might hold the clinical status fixed while varying the sensitivity and specificity of code-based observation. A causal simulation might hold a treatment effect fixed while allowing a covariate to be measured only through utilization that also varies by severity. In both cases, the claims structure does work that a flat table does not. It can make the cohort construction and observation process part of the experiment.
The word known needs care here. Known truth exists only for the parts of the mechanism chosen by the generator. If a simulation defines a latent disease status, treatment, and outcome, those values are known within that simulation. They do not validate the simulation against real Medicare beneficiaries, and they do not turn a synthetic estimate into a policy estimate.
Medicare-Synth currently offers the clearest support for the first artifact class: deterministic valid and invalid scenarios, schema contracts, relational validation, provenance records, and bounded expansion. Its documentation describes rich target populations, latent truth, and administrative observation models as an architectural direction rather than a completed calibrated population generator. That boundary should remain visible in any methods-oriented use of the project.
The distinction protects both audiences. A biostatistician can use claims-aware simulation as a design target without mistaking it for a completed source of empirical truth. A research engineer can use deterministic scenarios today without waiting for a full disease-progression model. Neither group has to call every useful artifact “realistic.”
Scale Is a Property of the Relationship Graph
Small fixtures are excellent at finding logical mistakes. They are poor at revealing many computational assumptions.
An episode algorithm can be correct on ten beneficiaries and still become unusable when each beneficiary has years of records, claims contain multiple lines, a few high utilizers dominate the data, and provider concentration changes the size of groups and joins. Repeating the same row a million times does not recreate that workload. It changes the row count while leaving the relationship graph almost empty.
For claims pipelines, runtime is better understood as a function of several workload dimensions:
$$ T = f(N, C, L, Y, S), $$where $N$ is the number of beneficiaries, $C$ is claims per beneficiary, $L$ is lines per claim, $Y$ is years of follow-up, and $S$ is a description of skew or concentration. This is not an asymptotic theorem or a fitted performance model. It is a reminder that two datasets with the same number of rows can exercise very different algorithms.
Medicare-Synth exposes those dimensions through its expand subcommand. Horizontal mode copies supplied beneficiary-centered subgraphs and deterministically re-keys beneficiary, claim, and prescription-event IDs across as many copies as requested, keeping all declared foreign-key relationships intact:
|
|
A horizontal run with high line density per claim is more likely to reveal accidental quadratic work, repeated scans, wide materialization, and a join that grows far beyond the intended unit of analysis than an equal-row count with sparse relationships. Vertical mode adds within-table attribute variation without changing cardinality or keys.
Neither operation creates new population evidence. Neither calibrates utilization patterns, provider networks, or cross-setting care trajectories. The expander is a relationship-preserving workload tool; it lets an engineer vary $N$, $C$, $L$, $Y$, and $S$ deliberately rather than discovering their interaction in a stalled production-like job.
The distinction gives a benchmark designer a better set of controls. Instead of reporting one runtime against one arbitrary extract, the designer can vary beneficiary count, claims-per-beneficiary, line multiplicity, observation length, and high-utilizer skew. Each run then answers a specific implementation question. Does the query scale with line density? Does a partitioning strategy survive provider concentration? Does a cohort definition materialize too much intermediate data?
Those results would still describe behavior under a generated workload. They would not prove the same behavior on every restricted Medicare extract. The point of a controlled benchmark is narrower and more useful: it makes the computational assumptions visible before a team learns about them from a stalled production-like job.
The Artifact Should State the Question It Can Answer
The three artifact classes can share schemas, evidence snapshots, validation rules, and export machinery. They should not share one undifferentiated claim of realism.
Each artifact carries a different, explicit fidelity claim. A deterministic fixture should be small enough for a reviewer to inspect each relevant relationship and should identify the validator result or analytic output expected from the scenario. A methods population should publish its inclusion rule, data-generating assumptions, seeds, and the location of any generator-defined truth. A computational benchmark should publish the workload parameters and the measurement protocol rather than a single runtime chosen for convenience.
Each release therefore needs an intent statement. A file called “synthetic Medicare data” says too little. A fixture for testing an emergency-to-admission definition says what the developer is expected to learn. A claims-aware simulation for a specified measurement-error study says what the biostatistician controls. A benchmark with a stated line-density and utilization-skew regime says what a systems comparison measures.
This is also a useful division of labor. Conventional simulations remain the best tool when an investigator needs a short path from a data-generating mechanism to an estimand. Claims-shaped artifacts add administrative complexity when that complexity is part of the proposed workflow. A large relational benchmark adds scale when the question concerns the implementation. The artifacts can complement one another without competing for the title of “the realistic dataset.”
Provenance and Validation Define the Boundary of a Release
Synthetic releases often flatten every value into the same label. That loses information. A field preserved from an official synthetic baseline, a release identifier re-keyed during expansion, an age derived from dates, an imputed contemporary field, and a deliberately invalid post-death claim do not have the same evidentiary status. A user who cannot distinguish them cannot decide which part of an analysis is supported by a source and which part rests on a design choice.
Medicare-Synth represents those distinctions with provenance statuses that include preserved, normalized, re-keyed, derived, imputed, synthesized, externally calibrated, and scenario-generated. The taxonomy will not make a value empirically true. It makes the path to that value inspectable. That is a much more useful promise for a research artifact.
Validation has a similarly bounded role. A field check enforces type, format, nullability, or an allowed code domain — for example, that every PRVDR_NPI is exactly ten digits (rule FLD-NPI) and that ICD diagnosis codes match the declared format contract (FLD-ICD). A record check enforces a date order or accounting identity — the validator flags any claim where CLM_FROM_DT > CLM_THRU_DT (TMP-001) and any inpatient admission where CLM_ADMSN_DT > NCH_BENE_DSCHRG_DT (TMP-002). A relational check (REL-001) identifies carrier or outpatient claims whose BENE_ID has no matching row in the beneficiary summary table. Temporal rules connect enrollment, service, admission, discharge, and death — service dates that fall after a recorded death date raise a mortality constraint violation. Scenario validation confirms that a deliberately invalid record, such as invalid_temporal_inversion or invalid_orphaned_claim, produces the expected finding category and severity rather than passing silently.
The scope of the check should travel with the result. A successful foreign-key validation does not verify a clinical care pathway. A payment total that reconciles to lines does not validate Medicare payment policy. A k-anonymity audit can describe a property of the released table; it is not a formal privacy guarantee. A release that passes a set of declared checks has met those contracts. It has not become suitable for population estimates, causal inference, or actuarial audit by implication.
This is why validation should precede expansion. If an input relationship is already invalid, multiplying it spreads the defect across a larger release. If a baseline contains an anomaly, the release process should report the exception or reject the artifact rather than silently repair it and erase the evidence. The test suite becomes part of the release boundary: it says which conditions were enforced and leaves the remaining uncertainty visible.
Medicare-Synth Integrates the Foundation, With Clear Boundaries
The current repository contains the foundations needed to make the argument operational: executable contracts across 19 domain tables (10 MBSF segments and 9 claims and event tables), Pydantic record checks, Polars-backed relational validation, deterministic scenarios, provenance statuses, connected-subgraph re-keying, release export, audit reporting, and a structured limitations profile. Its role is to connect those pieces so that an artifact carries more than a collection of plausible rows.
The project is strongest when it stays within that boundary. The 2021 source manifest and evidence snapshots define the default schema context. The 2022 path adds a source-verified beneficiary-and-carrier slice while leaving other claim families in their existing scope. The scenario catalogue gives developers named cases that can be inspected and tested. The expander preserves relationships of the tables it receives. The limitations profile separates structural, relational, temporal, accounting, distributional, and inferential boundaries.
Several claims remain outside the project’s current authority. Schema coverage does not establish full parity with every contemporary CCW layout. Relationship-preserving expansion does not create a calibrated population. Synthetic payment approximations should not be used for formal CMS actuarial work. The built-in table-attribute adapter is deterministic rather than a fitted tabular synthesis engine. The audit utilities do not confer formal disclosure protection. A mature methods layer would need its own explicit scientific assumptions and validation evidence.
Those limits define the design. They tell a team when the artifact belongs in a test suite, a methods experiment, or a workload benchmark, and when the question has moved beyond what the release can establish.
Earlier Testing Is Useful Only When Its Limits Remain Visible
The opening cohort pipeline failed because its local fixture had hidden the relationships that determined the answer. Adding more random-looking rows would have obscured the failure. A better test artifact would make the grain, the keys, the dates, and the expected behavior explicit.
That is the practical value of synthetic Medicare research infrastructure. It lets a team inspect schema assumptions before writing a large pipeline, turn a past defect into a named scenario, vary a workload without breaking its declared relationships, and document the conditions that a methods experiment holds fixed. More of the research can be developed before restricted-data access becomes the first time an assumption is tested.
The boundary is equally important. Synthetic output can be structurally useful while remaining unsuitable for empirical Medicare estimates. A credible release identifies what the data is faithful to, what it has been tested against, and which questions still require other evidence.
The data fidelity argument here is deliberately scoped. The repository also contains an 11-subcommand CLI, an autonomous workflow engine that runs 40-plus verification checks, and an agent harness for domain-level releases. Those pieces belong to a different essay about how a research infrastructure project is itself developed and validated.