2.7 KiB
2.7 KiB
Documentation index
System overview
This project creates a medical-context PHI classifier in two stages:
NIH/WHO/CDC-derived terminology + curated patterns
|
v
.NET generator
|
v
SQL Server TrainingItem
|
v
export -> validate -> split -> SFT
|
v
QLoRA train and evaluate
|
v
adapter -> merged Hugging Face model
The generator creates labeled natural-language examples. The Python pipeline uses those examples to teach a model to return a strict JSON decision describing whether the input contains medical PHI and, for positives, its category.
Documents
| Document | Purpose |
|---|---|
| Architecture | System boundaries, data flow, persistence, and design decisions |
| Generator | Catalogs, patterns, planning, randomization, LM normalization, durability |
| Data and provenance | Schemas, categories, terminology sources, lineage, and governance |
| Training pipeline | Every Python script and scaffold step |
| API reference | Generator HTTP endpoints, contracts, and failure behavior |
| Testing and validation | .NET tests, Python verification, evaluation semantics, current metrics |
| Runbook | Prerequisites and complete operator procedure |
| Terminology and verified claims | Corrections and precise wording for the project explanation |
| Artifacts and repository map | Source tree, inputs, outputs, and reproducibility boundaries |
C4 diagrams
The editable draw.io documents are:
Open the files with diagrams.net, the draw.io desktop application, or the VS Code Draw.io Integration extension.
Documentation conventions
- Verified means behavior is directly represented by tracked code, tests, or configuration.
- Owner-provided provenance means the fact was supplied by the project owner but source URLs, versions, retrieval dates, and licenses are not tracked here.
- Paths in commands are examples. The checked-in shell wrappers still use the
legacy WSL path
/d/ai/training/openbiollm-phi; update them before use if the checkout is elsewhere.