Specialized medical Protected Health Information (PHI) detection, classification, and redaction decision support using a fine-tuned Llama3-OpenBioLLM-8B model. Goes beyond traditional keyword and pattern matching by evaluating the semantic meaning of statements, distinguishing actual medical disclosures from nonmedical references, even in sentence fragments where the subject is implied rather than explicitly stated. Features .NET 10 synthetic training-data generation, QLoRA fine-tuning, model evaluation, and deployment-ready artifacts for automated privacy screening and redaction workflows.
Go to file
gaj3t 8422e6e97f Update README.md 2026-10-30 14:44:25 +00:00
PHI.Data.Generator Copying Repo to Demo 2026-10-26 04:03:52 -04:00
config Copying Repo to Demo 2026-10-26 04:03:52 -04:00
data Copying Repo to Demo 2026-10-26 04:03:52 -04:00
docs Copying Repo to Demo 2026-10-26 04:03:52 -04:00
outputs Copying Repo to Demo 2026-10-26 04:03:52 -04:00
scaffolding Copying Repo to Demo 2026-10-26 04:03:52 -04:00
scripts Copying Repo to Demo 2026-10-26 04:03:52 -04:00
.gitignore Copying Repo to Demo 2026-10-26 04:03:52 -04:00
.python-version Copying Repo to Demo 2026-10-26 04:03:52 -04:00
README.md Update README.md 2026-10-30 14:44:25 +00:00
pyproject.toml Copying Repo to Demo 2026-10-26 04:03:52 -04:00
uv.lock Copying Repo to Demo 2026-10-26 04:03:52 -04:00

README.md

Medical PHI detection model

This repository contains two connected systems:

  1. A .NET generator that combines curated medical terminology with positive and negative language patterns, normalizes the results through an OpenAI-compatible LM Studio endpoint, and persists labeled examples to SQL Server.
  2. A Python/WSL training pipeline that exports those examples, validates and splits them, converts them to supervised fine-tuning (SFT) conversations, trains and evaluates a QLoRA adapter for Llama3-OpenBioLLM-8B, and merges the adapter into a standalone Hugging Face model.

The repository also contains generated data, trained adapters, a merged model, evaluation results, and GGUF artifacts. The tracked automation creates the Hugging Face model; a GGUF conversion command is not currently tracked.

Documentation

Start with the documentation index.

Important semantics

  • Category is the label/taxonomy term used throughout this repository. The names are compatible with the intended Presidio extension/override, but this repository does not contain the Presidio integration itself.
  • A pattern with [sub] receives a shuffled subject (He, She, They, or Member). A pattern without [sub] is a subjectless fragment. The LM prompt treats that fragment as having an implied singular member/patient for grammar only; it must not insert that subject into the output.
    "She had a reaction to the shellfish" gets tagged and redacted but "had a reaction to the horrible cover of Van Halen's 'Jump'" and "He wrote a book about diabetes" would not.
    Training Patterns https://git.gaj3t.com/GAJ3T-Demo/PHI-Model-8B/src/branch/main/PHI.Data.Generator/RLHF.Generator/Data/Patterns
  • Positive and negative patterns teach contextual classification. Medical words can therefore appear in both classes: a real medical assertion is positive, while educational, fictional, historical, quoted, or unrelated use can be negative.
  • The medical terminology catalogs were assembled from NIH, WHO, and CDC data, according to the project owner. See the provenance documentation for the verification and licensing work still required.

Quick start

The complete, ordered procedure is in the runbook. Do not run the numbered shell wrappers without first correcting their legacy hard-coded project path.