65 lines
2.7 KiB
Markdown
65 lines
2.7 KiB
Markdown
# Documentation index
|
|
|
|
## System overview
|
|
|
|
This project creates a medical-context PHI classifier in two stages:
|
|
|
|
```text
|
|
NIH/WHO/CDC-derived terminology + curated patterns
|
|
|
|
|
v
|
|
.NET generator
|
|
|
|
|
v
|
|
SQL Server TrainingItem
|
|
|
|
|
v
|
|
export -> validate -> split -> SFT
|
|
|
|
|
v
|
|
QLoRA train and evaluate
|
|
|
|
|
v
|
|
adapter -> merged Hugging Face model
|
|
```
|
|
|
|
The generator creates labeled natural-language examples. The Python pipeline
|
|
uses those examples to teach a model to return a strict JSON decision describing
|
|
whether the input contains medical PHI and, for positives, its category.
|
|
|
|
## Documents
|
|
|
|
| Document | Purpose |
|
|
|---|---|
|
|
| [Architecture](architecture.md) | System boundaries, data flow, persistence, and design decisions |
|
|
| [Generator](generator.md) | Catalogs, patterns, planning, randomization, LM normalization, durability |
|
|
| [Data and provenance](data-and-provenance.md) | Schemas, categories, terminology sources, lineage, and governance |
|
|
| [Training pipeline](training-pipeline.md) | Every Python script and scaffold step |
|
|
| [API reference](api-reference.md) | Generator HTTP endpoints, contracts, and failure behavior |
|
|
| [Testing and validation](testing-and-validation.md) | .NET tests, Python verification, evaluation semantics, current metrics |
|
|
| [Runbook](runbook.md) | Prerequisites and complete operator procedure |
|
|
| [Terminology and verified claims](claims-and-terminology.md) | Corrections and precise wording for the project explanation |
|
|
| [Artifacts and repository map](artifacts.md) | Source tree, inputs, outputs, and reproducibility boundaries |
|
|
|
|
## C4 diagrams
|
|
|
|
The editable draw.io documents are:
|
|
|
|
- [Level 1: System context](diagrams/01-system-context.drawio)
|
|
- [Level 2: Containers](diagrams/02-containers.drawio)
|
|
- [Level 3: Components](diagrams/03-generator-components.drawio)
|
|
- [Level 4: Code](diagrams/04-generator-code.drawio)
|
|
|
|
Open the files with diagrams.net, the draw.io desktop application, or the VS
|
|
Code Draw.io Integration extension.
|
|
|
|
## Documentation conventions
|
|
|
|
- **Verified** means behavior is directly represented by tracked code, tests, or
|
|
configuration.
|
|
- **Owner-provided provenance** means the fact was supplied by the project owner
|
|
but source URLs, versions, retrieval dates, and licenses are not tracked here.
|
|
- Paths in commands are examples. The checked-in shell wrappers still use the
|
|
legacy WSL path `/d/ai/training/openbiollm-phi`; update them before use if the
|
|
checkout is elsewhere.
|