LLM RESEARCH AGENT
A verifiable literature retrieval system using Criteria-Driven RAG and Evidence-to-Generate (E2G) pipelines.
This project was developed as a technical evaluation of Asta, an AI-driven literature search agent. I architected a Python prototype that shifts from traditional retrieve-and-rank methods to a Criteria-Driven RAG pipeline, ensuring every academic claim is grounded in verifiable evidence.
System Architecture: Implementing E2G (Evidence Extraction), TRACE (Reasoning Chains), and AGREE (Self-Verification) to eliminate hallucinations in academic literature retrieval.
Technical Innovations
- Intent Decomposition: Translates natural language queries into 3 structured screening criteria and expanded API search terms.
- Multi-Agent Ensemble Evaluator: A hybrid scoring system that blends Vector Cosine Similarity (40%), LLM-as-a-Judge (40%), and Bibliometric Impact (20%) to rank papers by relevance and authority.
- TRACE Reasoning Chains: Autoregressively constructs Knowledge Graph (KG) triples to map how specific sentences in an abstract support a research claim.
- Faithfulness Self-Audit: An independent LLM auditor performs a final consistency check to ensure the generated report does not exaggerate the original paper’s findings.
Technical Stack
Frameworks: RAG, TRACE, AGREE, E2G
Tools: Python, OpenAI API, Sentence-Transformers, Semantic Scholar API
Methodology: Semantic Search, Intent Translation, Multi-Agent Ensemble