Models-Phi-8B/docs/README.md

3.2 KiB

Documentation index

System overview

This project creates a medical-context PHI classifier in two stages:

NIH/WHO/CDC-derived terminology + curated patterns
                         |
                         v
                  .NET generator
                         |
                         v
               SQL Server TrainingItem
                         |
                         v
       export -> validate -> split -> SFT
                         |
                         v
             QLoRA train and evaluate
                         |
                         v
       adapter -> merged Hugging Face model

The generator creates labeled natural-language examples. The Python pipeline uses those examples to teach a model to return a strict JSON decision describing whether the input contains medical PHI and, for positives, its category.

Documents

Document Purpose
Architecture System boundaries, data flow, persistence, and design decisions
Generator Catalogs, patterns, planning, randomization, LM normalization, durability
Data and provenance Schemas, categories, terminology sources, lineage, and governance
Training pipeline Every Python script and scaffold step
API reference Generator HTTP endpoints, contracts, and failure behavior
Testing and validation .NET tests, Python verification, evaluation semantics, current metrics
Runbook Prerequisites and complete operator procedure
Terminology and verified claims Corrections and precise wording for the project explanation
Artifacts and repository map Source tree, inputs, outputs, and reproducibility boundaries

C4 diagrams

The editable draw.io documents are:

Open the files with diagrams.net, the draw.io desktop application, or the VS Code Draw.io Integration extension. The .drawio files are the authoritative editable diagrams. Mermaid companions provide text-based, diff-friendly views; PDF companions provide portable rendered views.

Documentation conventions

  • Verified means behavior is directly represented by tracked code, tests, or configuration.
  • Owner-provided provenance means the fact was supplied by the project owner but source URLs, versions, retrieval dates, and licenses are not tracked here.
  • Paths in commands are examples. The checked-in shell wrappers still use the legacy WSL path /d/ai/training/openbiollm-phi; update them before use if the checkout is elsewhere.