6.7 KiB
Generator
Purpose
PHI.Data.Generator creates labeled contextual medical-language examples. It
does not generate a graph of nodes and edges. Its central domain concepts are
categories, category values/keywords, patterns, subjects, and training items.
Projects
| Project | Role |
|---|---|
RLHF.Generator |
Catalog parsing, planning, LM client, persistence, generation, keyword services |
RLHF.Generator.Api |
ASP.NET Core host and HTTP endpoints |
RLHF.Generator.Tests |
xUnit unit/integration tests using SQLite for test persistence |
All three target .NET 10. Production persistence uses EF Core SQL Server.
Taxonomy
The fixed categories are:
MEDICAL_ALLERGYMEDICAL_DIAGNOSISMEDICAL_FAMILY_HISTORYMEDICAL_MEDICATIONMEDICAL_MENTAL_HEALTHMEDICAL_OTHERMEDICAL_PHYSICAL_HEALTHMEDICAL_PROCEDUREMEDICAL_SUBSTANCE_USEMEDICAL_SYMPTOMMEDICAL_TESTMEDICAL_TREATMENT
Lookup is case-insensitive; stored/returned names use these exact values.
Input layout and grammar
RLHF.Generator/Data/
Subjects.txt
Categories/
MEDICAL_ALLERGY.txt
...
Patterns/
MEDICAL_ALLERGY.positive.txt
MEDICAL_ALLERGY.negative.txt
...
Blank lines and lines beginning with # are ignored. A category line is:
canonical form|allowed variant|another allowed variant
Forms must be non-empty and case-insensitively unique within a category. The first is canonical. The remainder are alternative forms available to the normalization model.
The exact subjects are He, She, They, and Member.
Pattern tokens include:
| Token | Meaning |
|---|---|
[x] |
Canonical category value; allowed variants are supplied to LM Studio |
[sub] |
A subject selected from the shuffled subject deck |
[is|are|was|were] |
Copula selected/rephrased for subject and tense |
[has|have|had] |
Possession/auxiliary selected for subject and tense |
[a|an] |
Article resolved locally from the following word |
The legacy [s] placeholder is rejected.
Positive and negative examples
Every category has an independent positive and negative pattern set. A positive pattern describes real medical context; a negative pattern uses the same domain language without asserting medical information about the person being analyzed.
Representative intended behavior:
| Input | Expected label | Why |
|---|---|---|
Has diabetes |
Positive | Subjectless clinical fragment with an implied member/patient |
He had a reaction to shellfish |
Positive | Actual allergy/reaction assertion |
She wrote a book about diabetes |
Negative | Creative/educational context, not her diagnosis |
Had a reaction to the horrible cover of Van Halen's Jump |
Negative | Idiomatic non-medical reaction |
Those examples describe the intended learned behavior; they are not hard-coded rules or keyword matchers in the inference pipeline.
Plan construction
For each category, the service runs the positive pass and then the negative pass. It partitions patterns into:
- value-bound: contains
[x]; - generic: does not contain
[x].
It rejects values without any value-bound pattern and value-bound patterns when the category has no values. It then:
- shuffles values, value-bound patterns, and generic patterns;
- creates shuffled/cycling subject and tense decks;
- emits every generic pattern once;
- emits
max(value-bound pattern count, value count)value-bound plans; - reshuffles a deck whenever it is exhausted.
The tense deck is null, past, present, and future. The canonical category
value is placed in [x]; variants are not independently randomly sampled.
Because each polarity is planned from its own pattern/value counts, positive and negative totals are not guaranteed to be equal.
Subject and fragment behavior
The precise behavior is:
- A pattern containing
[sub]gets one shuffled explicit subject. - A pattern without
[sub]is marked as a fragment and gets no inserted subject. Memberis one of the four explicit subject choices; it is not a fallback value.- For a fragment, the LM system prompt uses an implied singular member/patient only to choose grammar. It explicitly forbids returning or inventing that subject.
- Some fragments are completed locally without an LM call when no unresolved choice, protected-form decision, or meaningful tense rewrite remains.
Therefore the safe summary is: subjectless patterns remain subjectless, while grammar assumes an implied singular member/patient.
LM Studio normalization
Default settings:
| Setting | Value |
|---|---|
| Endpoint | http://localhost:1234/v1/chat/completions |
| Model | qwen/qwen3.5-9b |
| Temperature | 0.0 |
| Top-p | 1.0 |
| Max tokens | 96 |
| Seed | 42 |
| Timeout | 300 seconds |
Each request carries fragment status, optional protected subject, protected medical term and allowed forms, requested tense, and the sentence/template. The system prompt requires exactly one unwrapped line, preserves meaning and polarity, resolves all bracket expressions, and forbids invented fragment predicates.
Generation rejects empty output, newlines, unresolved square brackets, quotation/JSON/code-fence wrappers, and common response labels.
Concurrency and durability
Defaults are seed 42, parallelism 8, save batches of 24, and progress logs every 5 seconds.
Workers generate concurrently into a bounded channel. A single persistence writer places results into a sorted buffer and commits only the next contiguous plan ordinals. Each transaction inserts the batch and advances the generation checkpoint together.
A run can resume only when its seed, input fingerprint, configuration fingerprint, expected count, committed row count, and last ordinal agree. Changed inputs/configuration or inconsistent persisted state are rejected.
Keyword catalog
Keyword loading uses each category file's canonical value and variants. Loading is additive, case-insensitive, and idempotent. It does not update existing training items. Backfill is a separate operation:
- dry-run is available;
- only unique matches are assigned;
- unmatched or ambiguous rows remain without a keyword.
Database migrations
The migrations progress through:
- initial
CategoryandTrainingItemschema; - durable generation-run/checkpoint support;
Keyword,KeywordVariant, and training-item keyword linkage.
AddKeywordCatalog.sql provides a SQL alternative for the keyword phase.
Configuration changes that invalidate resume
Treat these as generation identity:
- category, pattern, or subject file contents/paths;
- random seed;
- concurrency and save/progress options included in the fingerprint;
- LM endpoint/model/prompt and generation settings.
Start a clean run rather than bypassing fingerprint checks.