PHI-Model-8B/docs/generator.md

6.7 KiB

Generator

Purpose

PHI.Data.Generator creates labeled contextual medical-language examples. It does not generate a graph of nodes and edges. Its central domain concepts are categories, category values/keywords, patterns, subjects, and training items.

Projects

Project Role
RLHF.Generator Catalog parsing, planning, LM client, persistence, generation, keyword services
RLHF.Generator.Api ASP.NET Core host and HTTP endpoints
RLHF.Generator.Tests xUnit unit/integration tests using SQLite for test persistence

All three target .NET 10. Production persistence uses EF Core SQL Server.

Taxonomy

The fixed categories are:

  1. MEDICAL_ALLERGY
  2. MEDICAL_DIAGNOSIS
  3. MEDICAL_FAMILY_HISTORY
  4. MEDICAL_MEDICATION
  5. MEDICAL_MENTAL_HEALTH
  6. MEDICAL_OTHER
  7. MEDICAL_PHYSICAL_HEALTH
  8. MEDICAL_PROCEDURE
  9. MEDICAL_SUBSTANCE_USE
  10. MEDICAL_SYMPTOM
  11. MEDICAL_TEST
  12. MEDICAL_TREATMENT

Lookup is case-insensitive; stored/returned names use these exact values.

Input layout and grammar

RLHF.Generator/Data/
  Subjects.txt
  Categories/
    MEDICAL_ALLERGY.txt
    ...
  Patterns/
    MEDICAL_ALLERGY.positive.txt
    MEDICAL_ALLERGY.negative.txt
    ...

Blank lines and lines beginning with # are ignored. A category line is:

canonical form|allowed variant|another allowed variant

Forms must be non-empty and case-insensitively unique within a category. The first is canonical. The remainder are alternative forms available to the normalization model.

The exact subjects are He, She, They, and Member.

Pattern tokens include:

Token Meaning
[x] Canonical category value; allowed variants are supplied to LM Studio
[sub] A subject selected from the shuffled subject deck
[is|are|was|were] Copula selected/rephrased for subject and tense
[has|have|had] Possession/auxiliary selected for subject and tense
[a|an] Article resolved locally from the following word

The legacy [s] placeholder is rejected.

Positive and negative examples

Every category has an independent positive and negative pattern set. A positive pattern describes real medical context; a negative pattern uses the same domain language without asserting medical information about the person being analyzed.

Representative intended behavior:

Input Expected label Why
Has diabetes Positive Subjectless clinical fragment with an implied member/patient
He had a reaction to shellfish Positive Actual allergy/reaction assertion
She wrote a book about diabetes Negative Creative/educational context, not her diagnosis
Had a reaction to the horrible cover of Van Halen's Jump Negative Idiomatic non-medical reaction

Those examples describe the intended learned behavior; they are not hard-coded rules or keyword matchers in the inference pipeline.

Plan construction

For each category, the service runs the positive pass and then the negative pass. It partitions patterns into:

  • value-bound: contains [x];
  • generic: does not contain [x].

It rejects values without any value-bound pattern and value-bound patterns when the category has no values. It then:

  1. shuffles values, value-bound patterns, and generic patterns;
  2. creates shuffled/cycling subject and tense decks;
  3. emits every generic pattern once;
  4. emits max(value-bound pattern count, value count) value-bound plans;
  5. reshuffles a deck whenever it is exhausted.

The tense deck is null, past, present, and future. The canonical category value is placed in [x]; variants are not independently randomly sampled.

Because each polarity is planned from its own pattern/value counts, positive and negative totals are not guaranteed to be equal.

Subject and fragment behavior

The precise behavior is:

  • A pattern containing [sub] gets one shuffled explicit subject.
  • A pattern without [sub] is marked as a fragment and gets no inserted subject.
  • Member is one of the four explicit subject choices; it is not a fallback value.
  • For a fragment, the LM system prompt uses an implied singular member/patient only to choose grammar. It explicitly forbids returning or inventing that subject.
  • Some fragments are completed locally without an LM call when no unresolved choice, protected-form decision, or meaningful tense rewrite remains.

Therefore the safe summary is: subjectless patterns remain subjectless, while grammar assumes an implied singular member/patient.

LM Studio normalization

Default settings:

Setting Value
Endpoint http://localhost:1234/v1/chat/completions
Model qwen/qwen3.5-9b
Temperature 0.0
Top-p 1.0
Max tokens 96
Seed 42
Timeout 300 seconds

Each request carries fragment status, optional protected subject, protected medical term and allowed forms, requested tense, and the sentence/template. The system prompt requires exactly one unwrapped line, preserves meaning and polarity, resolves all bracket expressions, and forbids invented fragment predicates.

Generation rejects empty output, newlines, unresolved square brackets, quotation/JSON/code-fence wrappers, and common response labels.

Concurrency and durability

Defaults are seed 42, parallelism 8, save batches of 24, and progress logs every 5 seconds.

Workers generate concurrently into a bounded channel. A single persistence writer places results into a sorted buffer and commits only the next contiguous plan ordinals. Each transaction inserts the batch and advances the generation checkpoint together.

A run can resume only when its seed, input fingerprint, configuration fingerprint, expected count, committed row count, and last ordinal agree. Changed inputs/configuration or inconsistent persisted state are rejected.

Keyword catalog

Keyword loading uses each category file's canonical value and variants. Loading is additive, case-insensitive, and idempotent. It does not update existing training items. Backfill is a separate operation:

  • dry-run is available;
  • only unique matches are assigned;
  • unmatched or ambiguous rows remain without a keyword.

Database migrations

The migrations progress through:

  1. initial Category and TrainingItem schema;
  2. durable generation-run/checkpoint support;
  3. Keyword, KeywordVariant, and training-item keyword linkage.

AddKeywordCatalog.sql provides a SQL alternative for the keyword phase.

Configuration changes that invalidate resume

Treat these as generation identity:

  • category, pattern, or subject file contents/paths;
  • random seed;
  • concurrency and save/progress options included in the fingerprint;
  • LM endpoint/model/prompt and generation settings.

Start a clean run rather than bypassing fingerprint checks.