LLM & RAGRetrieval Latency: ~24.5 msAutomated Tests: 321 passed

NexaRAG — Enterprise Document Intelligence & Hybrid RAG Platform

Hybrid retrieval, context orchestration, and citation-grounded LLM synthesis for complex enterprise documents.

Verified Milestone:321 passing tests · ~24.5 ms average retrieval latency · 7-tier context fusion
Watch Demonstration VideoView Repository
Technologies:PythonFastAPIPostgreSQLpgvectorReactTypeScriptDockerBM25Reciprocal Rank FusionCross-Encoder RerankingNVIDIA NeMo Guardrails
NexaRAG

Problem Statement

Naive RAG architectures suffer from semantic drift, missed alphanumeric identifiers (e.g. part numbers, policy codes), hallucinated citations, and catastrophic context overflow when ingesting complex 100+ page enterprise reports.

Engineering Goals

  • Sub-30ms retrieval latency combining dense semantic search and lexical exact matching.
  • Zero hallucinated citations with verifiable document page and line anchors.
  • Hard multi-tenant isolation and fail-closed prompt-injection defenses.
  • Predictable context assembly staying within a strict 6,000-token allocation.

NexaRAG — System ArchitectureVerified Pipeline

Select any pipeline stage below to inspect verified engineering details and data flow.

8 Stages

Stage: Document Ingestion

Role: Ingestion
Stack: PDF/DOCX/TXT Parsers

Extracts text, preserves headings and bounding boxes, applies token-aware recursive chunking.

Outbound:pgvector Index(Embeddings)BM25 Search(Inverted Index)

Execution Pipeline & Mechanics

01

Intelligent Ingestion & Chunking

Documents are parsed into structure-aware blocks preserving headers, tables, and page metadata, chunked at 512 tokens with 64-token overlap.

02

Dual-Engine Hybrid Retrieval

Parallel execution queries pgvector (HNSW cosine similarity) and an in-memory BM25 index. Candidate ranks are fused via Reciprocal Rank Fusion (RRF).

03

Cross-Encoder Precision Reranking

A distilled MS-MARCO cross-encoder scores query-passage pairs jointly, promoting passages with strong factual alignment.

04

7-Tier Context Fusion (RAG + CAG + MAG)

Balances live retrieval, pre-computed cached facts (CAG), and conversation history (MAG) under a dynamic 6,000-token budget ceiling.

05

Guardrails & Grounded SSE Streaming

NeMo Guardrails validates input/output against prompt injection and secret leakage, streaming tokens with page-specific citation anchors.

Engineering Challenges & Design Decisions

Decision: BM25 + pgvector over standalone vector-only search

Rationale: Pure dense vectors frequently fail on exact codes, invoice IDs, and specific regulation names. Hybrid RRF achieved 28% higher retrieval recall in enterprise documents.

Alternative Rejected: Vector-only embedding retrieval with OpenAI text-embedding-3-small.

Decision: Local cross-encoder reranking vs reranking LLM API

Rationale: Local cross-encoder executes in ~14ms without adding external API costs or network jitter to the hot retrieval loop.

Alternative Rejected: Third-party Cohere Rerank API.

Core Architectural Highlights

NexaRAG utilizes a layered architecture separating ingestion, dual-index retrieval, cross-encoder reranking, context orchestration, and guardrailed generation. State and vectors reside in PostgreSQL 16 with pgvector, while FastAPI handles non-blocking SSE streams to the React client.

•Implemented 6-route dynamic query router classifying inputs into factual, exploratory, code, summary, multi-hop, or conversational modes.
•Built custom SSE (Server-Sent Events) streaming protocol that synchronizes token delivery with citation metadata frames.
•Engineered row-level security (RLS) tenant isolation inside PostgreSQL so users cannot retrieve across organization silos.
•Created end-to-end Docker Compose setup for instant offline reproducibility.

Testing & Empirical Evaluation

321 automated pytest unit, integration, and security tests covering query classification, chunk boundary integrity, RLS tenant leak prevention, and RRF rank order stability.

Result #1
Average retrieval latency: ~24.5 ms across test corpus
Result #2
Pass rate: 100% (321 / 321 automated tests)
Result #3
Citation precision: 96.4% on structured test question bank

Security & Reliability Defenses

NVIDIA NeMo Guardrails 3-tier pipeline intercepts prompt injection and jailbreaks before LLM dispatch.
Automatic regex and entropy scanning for credentials, JWT tokens, and PII.
JWT-based role-based access control (RBAC) with strict tenant namespace partitioning.

Key Engineering Learnings

  • •RRF parameter k=60 proved resilient across variable corpus sizes without hyperparameter fine-tuning per tenant.
  • •Context budget capping at 6,000 tokens prevented downstream model degradation caused by 'lost in the middle' phenomena.

Local Reproduction & Setup Commands

$git clone https://github.com/Livesh-L-28/Nexa_RAG.git
$cd Nexa_RAG
$docker compose up --build
$Access frontend at http://localhost:3000 and FastAPI docs at http://localhost:8000/docs