PathoInsight: AI-Powered RAG Assistant for Understanding Pathology Reports

PathoInsight: AI-Powered RAG Assistant for Understanding Pathology Reports

Artificial Intelligence is transforming healthcare by making medical information more accessible than ever before. While pathology reports provide valuable insights into a person's health, they are often difficult for patients to understand due to medical abbreviations, numerical values, and complex terminology.

To solve this problem, I developed PathoInsight, an AI-powered Retrieval-Augmented Generation (RAG) assistant that helps patients understand pathology reports in simple and easy-to-read language.

Instead of relying on generic AI responses, PathoInsight combines document parsing, semantic search, a local medical knowledge base, and Google Gemini to generate contextual, grounded, and patient-friendly explanations.

🩺 The Problem

Millions of patients receive laboratory reports every day. Reports like Complete Blood Count (CBC), Liver Function Test (LFT), Kidney Function Test (KFT), Thyroid Profile, and Lipid Profile contain medical abbreviations and laboratory values that are meaningful to healthcare professionals but confusing for most patients.

A common reaction after receiving a pathology report is searching Google for individual test results. Unfortunately, these searches often provide fragmented information that may not relate to the patient's specific condition.

This communication gap can lead to unnecessary anxiety and confusion.

PathoInsight was built to bridge this gap.

💡 What is PathoInsight?

PathoInsight is a full-stack AI healthcare application capable of reading pathology reports and answering patient questions using Retrieval-Augmented Generation (RAG).

The application combines multiple technologies including intelligent PDF parsing, semantic vector search, a local medical knowledge base, and Google's Gemini AI to generate accurate and easy-to-understand medical explanations.

Instead of simply summarizing reports, PathoInsight understands the patient's uploaded report, retrieves relevant medical information, and generates contextual responses grounded in both local knowledge and live Google Search.

✨ Key Features

• 📄 Hybrid PDF Parsing
• 🧠 Retrieval-Augmented Generation (RAG)
• 🤖 Google Gemini 2.5 Flash Integration
• 📚 Local Medical Knowledge Base
• 🔍 Semantic Search using Vector Embeddings
• 🌐 Google Search Grounding
• ⚡ Real-time Streaming Responses
• 📊 Interactive Pathology Dashboard
• 💬 AI Medical Chat Assistant

🏗️ System Architecture

The following diagram illustrates the overall architecture of PathoInsight and how different components interact with each other.

graph TB

subgraph Browser["Browser (React + Vite)"]
UI["Dashboard + Chat Interface"]
end

subgraph Backend["FastAPI Backend"]
API["API Router"]
PARSER["Hybrid PDF Parser"]
RAG["RAG Engine"]
VECTOR["SQLite Vector Store"]
KB["Medical Knowledge Base"]
end

subgraph Gemini["Google Gemini"]
EMBED["text-embedding-004"]
LLM["Gemini 2.5 Flash"]
SEARCH["Google Search Grounding"]
end

UI --> API

API --> PARSER

PARSER --> RAG

RAG --> VECTOR

RAG --> KB

VECTOR --> EMBED

RAG --> LLM

LLM --> SEARCH

API --> UI

The frontend communicates with the FastAPI backend, which processes uploaded pathology reports. The extracted report is converted into vector embeddings and stored in a lightweight SQLite vector database. During a conversation, the RAG engine retrieves the most relevant medical context before sending it to Google Gemini for generating grounded responses.

📄 Hybrid PDF Processing

One of the most interesting aspects of PathoInsight is its intelligent PDF parsing engine. Rather than using a single OCR solution for every report, the application automatically detects whether the uploaded PDF already contains searchable text.

flowchart LR

A[Upload PDF]

A --> B{Searchable Text?}

B -->|Yes| C[pdfplumber]

B -->|No| D[Gemini Vision OCR]

C --> E[Extract Report]

D --> E

E --> F[Structured Medical Data]

This hybrid approach ensures fast processing for digital reports while maintaining compatibility with scanned pathology reports.

🧠 Understanding Retrieval-Augmented Generation (RAG)

Unlike a traditional AI chatbot that generates responses solely from its training data, PathoInsight uses Retrieval-Augmented Generation (RAG). This approach first retrieves relevant medical information from trusted sources before asking the language model to generate an answer.

The retrieval process combines two important sources of information:

• The patient's uploaded pathology report
• A local medical knowledge base containing reference information for laboratory tests

This ensures that every response is contextual, personalized, and grounded in relevant medical information instead of relying entirely on the AI model's memory.

flowchart LR

REPORT[Uploaded Report]

KB[Medical Knowledge Base]

REPORT --> CHUNK[Split into Chunks]

KB --> CHUNK

CHUNK --> EMBED[text-embedding-004]

EMBED --> DB[(SQLite Vector Store)]

QUESTION[User Question]

QUESTION --> SEARCH[Semantic Search]

DB --> SEARCH

SEARCH --> CONTEXT[Relevant Context]

CONTEXT --> GEMINI[Gemini 2.5 Flash]

GEMINI --> RESPONSE[AI Explanation]

Whenever a user asks a question, the system performs semantic similarity search to retrieve the most relevant medical information before generating the final response.

🔍 Semantic Search

Traditional keyword search only looks for exact words. Semantic search, however, understands the meaning behind a question.

For example, if a patient asks:

Why is my hemoglobin low?

The system understands that this question relates to:

• Hemoglobin
• Anemia
• Red Blood Cells
• CBC Panel
• Reference Range

Instead of matching keywords, PathoInsight retrieves the most semantically similar medical information using vector embeddings generated by Google's text-embedding-004 model.

💬 AI Conversation Workflow

The following sequence diagram illustrates how a conversation flows through the system.

sequenceDiagram

actor User

participant React

participant FastAPI

participant Parser

participant RAG

participant Gemini

User->>React: Upload Report

React->>FastAPI: POST /upload

FastAPI->>Parser: Extract Text

Parser-->>FastAPI: Structured Report

User->>React: Ask Question

React->>FastAPI: POST /chat

FastAPI->>RAG: Retrieve Context

RAG->>Gemini: Prompt + Medical Context

Gemini-->>FastAPI: Stream Tokens

FastAPI-->>React: SSE Response

React-->>User: Display AI Response

🛠 Technology Stack

PathoInsight is built using modern web technologies and AI services to provide a lightweight yet powerful healthcare assistant.

Category	Technology	Purpose
Frontend	React + Vite	Modern interactive user interface
Backend	Python + FastAPI	REST API & AI services
Language Model	Gemini 2.5 Flash	Medical explanation generation
Embeddings	text-embedding-004	Semantic vector generation
Vector Database	SQLite + NumPy	Semantic similarity search
PDF Processing	pdfplumber	Digital PDF extraction
OCR	Gemini Vision	Scanned report extraction
Grounding	Google Search	Current medical references

⚡ Why SQLite Instead of a Dedicated Vector Database?

Many AI applications rely on external vector databases such as Pinecone, ChromaDB, or Weaviate. For PathoInsight, I wanted a lightweight and self-contained solution that could run locally without requiring cloud infrastructure.

SQLite stores vector embeddings while NumPy performs cosine similarity calculations, making the application portable, fast, and easy to deploy.

• ✔ Zero external infrastructure
• ✔ Lightweight deployment
• ✔ Fast semantic search
• ✔ Easy backup and portability

📊 Interactive Dashboard

After processing the uploaded pathology report, PathoInsight presents the extracted information through a clean and organized dashboard. Instead of displaying raw text from the PDF, the application categorizes laboratory values into their respective pathology panels, making the report easier to interpret.

Each test displays its measured value, unit, reference range, and status. Abnormal values are highlighted, allowing users to quickly identify parameters that may require medical attention.

Currently, PathoInsight supports the following pathology panels:

• 🩸 Complete Blood Count (CBC)
• 🧪 Liver Function Test (LFT)
• 💧 Kidney Function Test (KFT)
• 🦋 Thyroid Profile
• ❤️ Lipid Profile

The dashboard acts as the foundation for the AI assistant by converting the pathology report into structured medical data that can be searched and understood by the RAG pipeline.

📁 Project Structure

The project follows a modular architecture where each component has a dedicated responsibility. This separation keeps the application maintainable and makes it easier to extend with new features in the future.

graph TD

ROOT[PathoInsight]

ROOT --> FRONTEND[Frontend]

ROOT --> BACKEND[Backend]

FRONTEND --> APP[React + Vite]

FRONTEND --> COMPONENTS[Components]

BACKEND --> MAIN[main.py]

BACKEND --> PARSER[parser.py]

BACKEND --> RAG[rag_engine.py]

BACKEND --> VECTOR[vector_store.py]

BACKEND --> KB[medical_kb.json]

BACKEND --> CONFIG[config.py]

🚀 Why Hybrid PDF Parsing?

Medical laboratories generate reports in different formats. Some reports are digitally generated PDFs containing selectable text, while others are scanned images without a searchable text layer.

Using only OCR would unnecessarily increase processing time for digital reports. Likewise, relying only on text extraction would fail for scanned reports.

To solve this challenge, PathoInsight automatically detects the report type and chooses the most suitable extraction method:

• 📄 Digital PDFs → pdfplumber
• 🖼 Scanned Reports → Gemini Vision OCR

This hybrid strategy provides both speed and reliability while maintaining compatibility with a wide variety of pathology reports.

💬 Example Questions Users Can Ask

After uploading a report, patients can interact naturally with the AI assistant using everyday language.

• Why is my Hemoglobin low?
• What does an elevated ALT indicate?
• Is my cholesterol level normal?
• Explain my CBC report.
• What causes high Creatinine?
• Should I worry about my Thyroid results?
• Summarize my pathology report.
• Which values are outside the normal range?

Instead of providing generic internet answers, the AI uses the uploaded report together with medical reference information to generate contextual explanations.

⭐ Project Highlights

• 🔬 Hybrid PDF Parsing Engine
• 🧠 Retrieval-Augmented Generation
• 📚 Local Medical Knowledge Base
• 🔍 Semantic Search using Vector Embeddings
• 🤖 Google Gemini 2.5 Flash
• 🌐 Live Google Search Grounding
• ⚡ Streaming AI Responses using Server-Sent Events
• 📊 Interactive Medical Dashboard
• 💻 Modern Full-Stack Architecture
• 🩺 Patient-Friendly Medical Explanations

🚀 Future Scope

Although PathoInsight currently focuses on pathology reports, the architecture has been designed to support additional healthcare applications in the future.

• 📱 Mobile Application
• 🌍 Multi-language Support
• 👨‍⚕️ Doctor Dashboard
• 📈 Historical Report Comparison
• 🎤 Voice-based AI Assistant
• 🏥 Hospital Information System Integration
• 🧬 Additional Laboratory Panels
• 📊 Trend Visualization Across Reports
• ☁ Cloud Synchronization
• 🔐 Secure Patient Authentication

These enhancements would transform PathoInsight from a pathology report assistant into a comprehensive AI-powered healthcare platform.