RAG-DEMO/docs/module-reference.md

2.2 KiB

Module reference

Platform package

Module Responsibility
config.py typed settings, environment parsing, model definition
models.py SQLAlchemy entities and checkpoint schema
storage.py engine/session, schema checks, persistence, vector index
sources.py strict JSONL and PostgreSQL source readers
llm.py OpenAI-compatible completion and embedding client
runtime.py StageContext construction and initialization
stage_loader.py numbered stage module lookup
pipeline.py compatibility facade for stage methods
orchestrator.py durable runs, claiming, resume, stale recovery, locks
auth.py JWT verification and permission masks
api.py FastAPI app, request models, routes, authorization
worker.py polling worker and one-run execution
cli.py stage and durable-run command line entry points
observability.py structured JSON logging

Numbered stages

Module Main responsibility
01_normalize_content.py validate handoff, extract text, persist raw/normalized documents
02_clean_content.py clean, deduplicate, and chunk
03_define_knowledge_model.py persist entity/relationship vocabulary and templates
04_extract_knowledge.py call LLM, validate extraction, persist per-chunk results
05_reconcile_knowledge.py canonicalize names and resolve cross-chunk relationships
06_build_knowledge_graph.py materialize nodes and edges
07_build_communities.py detect communities and summarize them
08_build_retrieval_index.py embed chunks/graph/community items and build HNSW
09_query_graphrag.py vector retrieval, graph expansion, source resolution
10_generate_answer.py generate answer with source citations

Each numbered module exposes a run function and command-line adapter. Stages 1-5 also expose focused validation/transformation helpers; stage 9 exposes retrieval helpers such as cosine ranking and graph expansion.

Entry points

  • knowledgegraph -> graphrag_platform.cli:main_all
  • knowledgegraph-worker -> graphrag_platform.worker:main
  • uvicorn graphrag_platform.api:app