# Medical PHI detection model This repository contains two connected systems: 1. A .NET generator that combines curated medical terminology with positive and negative language patterns, normalizes the results through an OpenAI-compatible LM Studio endpoint, and persists labeled examples to SQL Server. 2. A Python/WSL training pipeline that exports those examples, validates and splits them, converts them to supervised fine-tuning (SFT) conversations, trains and evaluates a QLoRA adapter for `Llama3-OpenBioLLM-8B`, and merges the adapter into a standalone Hugging Face model. The repository also contains generated data, trained adapters, a merged model, evaluation results, and GGUF artifacts. The tracked automation creates the Hugging Face model; a GGUF conversion command is not currently tracked. ## Documentation Start with [the documentation index](docs/README.md). - [Architecture](docs/architecture.md) - [Generator](docs/generator.md) - [Data and provenance](docs/data-and-provenance.md) - [Training pipeline](docs/training-pipeline.md) - [API reference](docs/api-reference.md) - [Testing and validation](docs/testing-and-validation.md) - [Step-by-step runbook](docs/runbook.md) - [Terminology and verified claims](docs/claims-and-terminology.md) - [Artifacts and repository map](docs/artifacts.md) - [Editable C4 diagrams](docs/diagrams/) ## Important semantics - `Category` is the label/taxonomy term used throughout this repository. The names are compatible with the intended Presidio extension/override, but this repository does not contain the Presidio integration itself. - A pattern with `[sub]` receives a shuffled subject (`He`, `She`, `They`, or `Member`). A pattern without `[sub]` is a subjectless fragment. The LM prompt treats that fragment as having an implied singular member/patient for grammar only; it must not insert that subject into the output.
***"She had a reaction to the shellfish"*** gets tagged and redacted but ***"had a reaction to the horrible cover of Van Halen's 'Jump'"*** and ***"He wrote a book about diabetes"*** would not.
Training Patterns https://git.gaj3t.com/GAJ3T-Demo/PHI-Model-8B/src/branch/main/PHI.Data.Generator/RLHF.Generator/Data/Patterns - Positive and negative patterns teach contextual classification. Medical words can therefore appear in both classes: a real medical assertion is positive, while educational, fictional, historical, quoted, or unrelated use can be negative. - The medical terminology catalogs were assembled from NIH, WHO, and CDC data, according to the project owner. See the provenance documentation for the verification and licensing work still required. ## Quick start The complete, ordered procedure is in [the runbook](docs/runbook.md). Do not run the numbered shell wrappers without first correcting their legacy hard-coded project path.