PageIndex

I want to evaluate and, if it fits, integrate a technique from recent AI research into my project.

## Technique: PageIndex
Tree search framework hits 98.7% on documents where vector search fails

PageIndex: open-source framework that replaces chunking + embedding + cosine search with tree-structured document indexing. Reads the full document once, builds a hierarchical tree (like a smart table of contents), then lets the LLM reason through that tree at query time. 98.7% on FinanceBench vs 61% for the best GPT-4 + vector-RAG baseline. No vector DB, no embeddings, no chunking decisions. Pip-installable, MCP-compatible. Directly applicable to any structured-document RAG pipeline: financial reports, legal contracts, technical manuals.

What the reference demo shows: 4-sub-tab Gradio demo — (1) Live Demo: select doc + enter query → see document tree + retrieved sections; (2) Benchmarks: PageIndex vs GPT-4+VecRAG vs BM25 vs DPR across 3 benchmarks; (3) How It Works: tree traversal algorithm, chunking comparison, when to use each; (4) Implementation Patterns: 3 drop-in patterns (drop-in replacement, hybrid routing, MCP tool).

## Reference implementation (Vector Ridge Labs, POC-045)
- Live demo: https://osoman2-gradio-pocs.hf.space/?poc=POC-045
- Source code: https://huggingface.co/spaces/osoman2/Gradio_pocs/tree/main/page_index
Original source: AlphaSignal Top Repo · 2026-09-30 · 4,598 likes

## My project
<Describe your stack, the problem you want to solve and where this technique would sit. Example: "Python FastAPI RAG service over 50k PDFs, Postgres + pgvector, OpenAI embeddings, p95 latency budget 800 ms.">

## What I need from you
1. Read the reference source code and explain in at most 5 bullets how the technique works and what it would change compared to my current approach.
2. Decide whether it fits my case. If it does not, say why and stop there.
3. Propose the smallest viable integration: files or modules to touch, new dependencies, configuration.
4. Implement it behind a flag or as a separate function so it can be compared side by side with what I have today.
5. Add an evaluation script that measures retrieval quality (recall@k / nDCG on a labelled sample of my queries) and latency, before and after, on my own data.
6. List the risks (latency, cost, failure modes) and what to monitor once it is in production.

Constraints: keep my existing stack and conventions, avoid heavy dependencies unless the gain is clear, and reference the file in the demo source whenever you reuse its logic.