39 lines
1.5 KiB
Markdown
39 lines
1.5 KiB
Markdown
# Data model and checkpoint ownership
|
|
|
|
The initial Alembic revision creates the SQLAlchemy metadata and the PgVector
|
|
extension. The models live in [`src/graphrag_platform/models.py`](../src/graphrag_platform/models.py).
|
|
|
|
## Pipeline data
|
|
|
|
| Table/model | Owner stage | Meaning |
|
|
|---|---:|---|
|
|
| `raw_documents` | 1 | original source payload and provenance |
|
|
| `documents` | 1 | normalized text and source identity |
|
|
| `chunks` | 2 | cleaned, overlapping retrieval units |
|
|
| `knowledge_models` | 3 | configured entity/relationship vocabulary and templates |
|
|
| `extractions` | 4 | validated LLM payload per chunk |
|
|
| `resolved_entities` | 5 | canonical entities and aliases |
|
|
| `resolved_relationships` | 5 | relationships linked to canonical keys |
|
|
| `nodes` | 6 | graph nodes |
|
|
| `edges` | 6 | graph edges |
|
|
| `communities` | 7 | detected groups and summaries |
|
|
| `search_items` | 8 | text, metadata, source URL, and embedding |
|
|
|
|
## Run tracking
|
|
|
|
| Table/model | Purpose |
|
|
|---|---|
|
|
| `pipeline_runs` | requested range, status, attempt, caller, correlation, and errors |
|
|
| `stage_runs` | per-stage status, input/output, timestamps, and error details |
|
|
|
|
Stage data is corpus-scoped. Clear or rebuild from the first invalid stage
|
|
through downstream derived data; do not manually delete unrelated corpora.
|
|
|
|
## Indexing
|
|
|
|
PostgreSQL creates a dimension-specific HNSW cosine index on retrieval items.
|
|
SQLite stores vectors as JSON and uses in-process cosine similarity. The latter
|
|
is useful for tests but does not validate production query plans or PgVector
|
|
behavior.
|
|
|