PHI-Model-8B/docs/architecture.md

5.3 KiB

Architecture

Goals

The system produces a domain-adapted model that classifies contextual medical PHI and returns a strict JSON response. Its principal design goals are:

  • create reproducible positive and negative training examples;
  • preserve a controlled 12-category medical taxonomy;
  • use contextual negatives so keywords alone do not determine classification;
  • durably generate large datasets with resumable SQL persistence;
  • fine-tune an 8B model on commodity GPU hardware through QLoRA;
  • evaluate JSON validity, classification, and category extraction separately;
  • publish both an adapter and a standalone merged model.

System boundary

Inside the repository are:

  • the .NET 10 generator library, API, EF Core migrations, catalogs, and tests;
  • curated category values, grammar variants, subjects, and positive/negative patterns;
  • Python 3.12 data preparation, training, evaluation, and merge scripts;
  • PowerShell/Bash orchestration scripts;
  • generated JSONL data and model/evaluation artifacts.

External dependencies are:

  • SQL Server (RLHFGenerator);
  • LM Studio exposing an OpenAI-compatible chat completions endpoint;
  • the qwen/qwen3.5-9b normalization model configured in the API;
  • the local Llama3-OpenBioLLM-8B base model;
  • WSL/Linux, CUDA 12.8-compatible PyTorch, and a BF16-capable NVIDIA GPU;
  • NIH, WHO, and CDC as owner-reported origins of the terminology catalogs.

Microsoft Presidio is an intended downstream integration, not a runtime component of this repository.

End-to-end data flow

  1. The generator validates 12 category files, 24 polarity-specific pattern files, and the exact four-subject catalog.
  2. For each category and polarity, it deterministically shuffles values, patterns, subjects, and tenses using the configured random seed.
  3. It substitutes protected medical values, resolves some grammar locally, and asks LM Studio to normalize remaining expressions without changing meaning or polarity.
  4. It validates one-line output and writes ordered TrainingItem rows to SQL Server in atomic batches, updating a durable generation checkpoint in the same transaction.
  5. PowerShell exports TrainingItem plus Category to data/source.jsonl.
  6. Python validates the schema and taxonomy, then deterministically stratifies by (category, is_positive) into train, validation, and development sets.
  7. The SFT builder replaces each split record with a three-message conversation containing the system policy, input text, and exact target JSON.
  8. Pilot and full training load the base model in 4-bit NF4 and train LoRA adapters using completion-only loss.
  9. Evaluation loads the base model plus final adapter and generates development predictions deterministically.
  10. The merge step loads the unquantized base model on CPU, merges the adapter, and saves a BF16 Hugging Face model.

Persistence model

Entity Responsibility
Category Stable seeded category ID/name
Keyword Canonical category-specific medical value
KeywordVariant Alternate grammatical/surface form
GenerationRun Seed, fingerprints, status, checkpoint, counts, errors
TrainingItem Generated text, label, category, optional keyword, plan ordinal

Important constraints include unique category names, unique canonical values within a category, unique variants within a keyword, unique training record keys, and unique (GenerationRunId, PlanOrdinal).

Reliability model

  • Deterministic plan: System.Random plus seed 42 creates a repeatable plan for unchanged inputs and configuration.
  • External-model caveat: repeatable planning does not guarantee byte-identical LM Studio responses if model files, runtime, or service behavior change.
  • Fingerprinting: generation hashes source files and effective settings; changed inputs/settings reject an unsafe resume.
  • Ordered parallelism: generation is concurrent, but a single ordered writer buffers out-of-order results.
  • Atomic checkpoints: inserted rows and the checkpoint advance commit in the same database transaction.
  • Fail closed: invalid LM output, request failures, persistence failures, cancellation, mismatched durable state, or ambiguous incomplete runs fail the operation rather than silently producing success-shaped output.

Classification model

The model is trained as a binary classifier plus category reporter:

  • A positive SFT target has is_medical_phi: true, score 1.0, a single result, and an entity type equal to one of the 12 categories.
  • A negative SFT target has is_medical_phi: false, score 0.0, and no results.
  • Positive targets currently mark the entire input as text and evidence. This is classification with whole-input evidence, not span extraction.

Known architecture gaps

  • No tracked Presidio recognizer, registry, mapping, or deployment integration.
  • No tracked GGUF conversion or quantization workflow, although GGUF files are present.
  • Shell scaffolds have a legacy hard-coded checkout path.
  • The SFT conversion overwrites split files and is not idempotent.
  • The generator depends on an external normalization model whose exact artifact version is not fingerprinted.
  • Terminology source URLs, versions, retrieval dates, transformations, and licenses are not captured in a machine-readable manifest.