Models-Phi-8B/docs/architecture.md

114 lines
5.3 KiB
Markdown

# Architecture
## Goals
The system produces a domain-adapted model that classifies contextual medical
PHI and returns a strict JSON response. Its principal design goals are:
- create reproducible positive and negative training examples;
- preserve a controlled 12-category medical taxonomy;
- use contextual negatives so keywords alone do not determine classification;
- durably generate large datasets with resumable SQL persistence;
- fine-tune an 8B model on commodity GPU hardware through QLoRA;
- evaluate JSON validity, classification, and category extraction separately;
- publish both an adapter and a standalone merged model.
## System boundary
Inside the repository are:
- the .NET 10 generator library, API, EF Core migrations, catalogs, and tests;
- curated category values, grammar variants, subjects, and positive/negative
patterns;
- Python 3.12 data preparation, training, evaluation, and merge scripts;
- PowerShell/Bash orchestration scripts;
- generated JSONL data and model/evaluation artifacts.
External dependencies are:
- SQL Server (`RLHFGenerator`);
- LM Studio exposing an OpenAI-compatible chat completions endpoint;
- the `qwen/qwen3.5-9b` normalization model configured in the API;
- the local `Llama3-OpenBioLLM-8B` base model;
- WSL/Linux, CUDA 12.8-compatible PyTorch, and a BF16-capable NVIDIA GPU;
- NIH, WHO, and CDC as owner-reported origins of the terminology catalogs.
Microsoft Presidio is an intended downstream integration, not a runtime
component of this repository.
## End-to-end data flow
1. The generator validates 12 category files, 24 polarity-specific pattern
files, and the exact four-subject catalog.
2. For each category and polarity, it deterministically shuffles values,
patterns, subjects, and tenses using the configured random seed.
3. It substitutes protected medical values, resolves some grammar locally, and
asks LM Studio to normalize remaining expressions without changing meaning or
polarity.
4. It validates one-line output and writes ordered `TrainingItem` rows to SQL
Server in atomic batches, updating a durable generation checkpoint in the
same transaction.
5. PowerShell exports `TrainingItem` plus `Category` to `data/source.jsonl`.
6. Python validates the schema and taxonomy, then deterministically stratifies
by `(category, is_positive)` into train, validation, and development sets.
7. The SFT builder replaces each split record with a three-message conversation
containing the system policy, input text, and exact target JSON.
8. Pilot and full training load the base model in 4-bit NF4 and train LoRA
adapters using completion-only loss.
9. Evaluation loads the base model plus final adapter and generates development
predictions deterministically.
10. The merge step loads the unquantized base model on CPU, merges the adapter,
and saves a BF16 Hugging Face model.
## Persistence model
| Entity | Responsibility |
|---|---|
| `Category` | Stable seeded category ID/name |
| `Keyword` | Canonical category-specific medical value |
| `KeywordVariant` | Alternate grammatical/surface form |
| `GenerationRun` | Seed, fingerprints, status, checkpoint, counts, errors |
| `TrainingItem` | Generated text, label, category, optional keyword, plan ordinal |
Important constraints include unique category names, unique canonical values
within a category, unique variants within a keyword, unique training record
keys, and unique `(GenerationRunId, PlanOrdinal)`.
## Reliability model
- **Deterministic plan:** `System.Random` plus seed 42 creates a repeatable plan
for unchanged inputs and configuration.
- **External-model caveat:** repeatable planning does not guarantee byte-identical
LM Studio responses if model files, runtime, or service behavior change.
- **Fingerprinting:** generation hashes source files and effective settings;
changed inputs/settings reject an unsafe resume.
- **Ordered parallelism:** generation is concurrent, but a single ordered writer
buffers out-of-order results.
- **Atomic checkpoints:** inserted rows and the checkpoint advance commit in the
same database transaction.
- **Fail closed:** invalid LM output, request failures, persistence failures,
cancellation, mismatched durable state, or ambiguous incomplete runs fail the
operation rather than silently producing success-shaped output.
## Classification model
The model is trained as a binary classifier plus category reporter:
- A positive SFT target has `is_medical_phi: true`, score `1.0`, a single result,
and an entity type equal to one of the 12 categories.
- A negative SFT target has `is_medical_phi: false`, score `0.0`, and no results.
- Positive targets currently mark the entire input as `text` and `evidence`.
This is classification with whole-input evidence, not span extraction.
## Known architecture gaps
- No tracked Presidio recognizer, registry, mapping, or deployment integration.
- No tracked GGUF conversion or quantization workflow, although GGUF files are
present.
- Shell scaffolds have a legacy hard-coded checkout path.
- The SFT conversion overwrites split files and is not idempotent.
- The generator depends on an external normalization model whose exact artifact
version is not fingerprinted.
- Terminology source URLs, versions, retrieval dates, transformations, and
licenses are not captured in a machine-readable manifest.