RAG-DEMO/docs/data-model.md

1.5 KiB

Data model and checkpoint ownership

The initial Alembic revision creates the SQLAlchemy metadata and the PgVector extension. The models live in src/graphrag_platform/models.py.

Pipeline data

Table/model Owner stage Meaning
raw_documents 1 original source payload and provenance
documents 1 normalized text and source identity
chunks 2 cleaned, overlapping retrieval units
knowledge_models 3 configured entity/relationship vocabulary and templates
extractions 4 validated LLM payload per chunk
resolved_entities 5 canonical entities and aliases
resolved_relationships 5 relationships linked to canonical keys
nodes 6 graph nodes
edges 6 graph edges
communities 7 detected groups and summaries
search_items 8 text, metadata, source URL, and embedding

Run tracking

Table/model Purpose
pipeline_runs requested range, status, attempt, caller, correlation, and errors
stage_runs per-stage status, input/output, timestamps, and error details

Stage data is corpus-scoped. Clear or rebuild from the first invalid stage through downstream derived data; do not manually delete unrelated corpora.

Indexing

PostgreSQL creates a dimension-specific HNSW cosine index on retrieval items. SQLite stores vectors as JSON and uses in-process cosine similarity. The latter is useful for tests but does not validate production query plans or PgVector behavior.