PHI-Model-8B/docs/README.md

79 lines
3.2 KiB
Markdown

# Documentation index
## System overview
This project creates a medical-context PHI classifier in two stages:
```text
NIH/WHO/CDC-derived terminology + curated patterns
|
v
.NET generator
|
v
SQL Server TrainingItem
|
v
export -> validate -> split -> SFT
|
v
QLoRA train and evaluate
|
v
adapter -> merged Hugging Face model
```
The generator creates labeled natural-language examples. The Python pipeline
uses those examples to teach a model to return a strict JSON decision describing
whether the input contains medical PHI and, for positives, its category.
## Documents
| Document | Purpose |
|---|---|
| [Architecture](architecture.md) | System boundaries, data flow, persistence, and design decisions |
| [Generator](generator.md) | Catalogs, patterns, planning, randomization, LM normalization, durability |
| [Data and provenance](data-and-provenance.md) | Schemas, categories, terminology sources, lineage, and governance |
| [Training pipeline](training-pipeline.md) | Every Python script and scaffold step |
| [API reference](api-reference.md) | Generator HTTP endpoints, contracts, and failure behavior |
| [Testing and validation](testing-and-validation.md) | .NET tests, Python verification, evaluation semantics, current metrics |
| [Runbook](runbook.md) | Prerequisites and complete operator procedure |
| [Terminology and verified claims](claims-and-terminology.md) | Corrections and precise wording for the project explanation |
| [Artifacts and repository map](artifacts.md) | Source tree, inputs, outputs, and reproducibility boundaries |
## C4 diagrams
The editable draw.io documents are:
- Level 1: System context
([draw.io](diagrams/01-system-context.drawio),
[Mermaid](diagrams/01-system-context.mmd),
[PDF](diagrams/01-system-context.pdf))
- Level 2: Containers
([draw.io](diagrams/02-containers.drawio),
[Mermaid](diagrams/02-containers.mmd),
[PDF](diagrams/02-containers.pdf))
- Level 3: Components
([draw.io](diagrams/03-generator-components.drawio),
[Mermaid](diagrams/03-generator-components.mmd),
[PDF](diagrams/03-generator-components.pdf))
- Level 4: Code
([draw.io](diagrams/04-generator-code.drawio),
[Mermaid](diagrams/04-generator-code.mmd),
[PDF](diagrams/04-generator-code.pdf))
Open the files with diagrams.net, the draw.io desktop application, or the VS
Code Draw.io Integration extension. The `.drawio` files are the authoritative
editable diagrams. Mermaid companions provide text-based, diff-friendly views;
PDF companions provide portable rendered views.
## Documentation conventions
- **Verified** means behavior is directly represented by tracked code, tests, or
configuration.
- **Owner-provided provenance** means the fact was supplied by the project owner
but source URLs, versions, retrieval dates, and licenses are not tracked here.
- Paths in commands are examples. The checked-in shell wrappers still use the
legacy WSL path `/d/ai/training/openbiollm-phi`; update them before use if the
checkout is elsewhere.