YouTube Explainer Bot
An AI-powered conversational assistant that ingests YouTube video URLs, extracts transcripts via Supadata API, builds hybrid dense/sparse vector indices, and provides grounded Q&A with Groq LLMs.
- AI
- Video RAG
- Applied AI
Context
Long-form YouTube videos (podcasts, technical lectures, tutorials) contain valuable information, but scrubbing through hours of video footage to find specific answers is slow and inefficient.
YouTube Explainer Bot was engineered as a full-stack RAG application that takes any YouTube video URL, fetches its transcript using Supadata API to avoid cloud IP blocking, builds an in-memory hybrid search index (ChromaDB + BM25), and answers user questions grounded strictly in the video transcript.
The problem
Cloud-hosted RAG applications attempting to fetch YouTube transcripts directly via unofficial libraries frequently suffer IP bans and rate-limiting from YouTube servers.
Furthermore, semantic vector search alone often struggles with specific technical terms, acronyms, or proper names spoken in videos, while keyword matching fails to capture conceptual context.
Product strategy
The strategy was to combine reliable cloud-first transcript extraction via Supadata API with a hybrid retrieval pipeline that fuses dense vector embeddings and BM25 lexical search using Reciprocal Rank Fusion (RRF).
Cloud-resilient extraction
Fetch transcripts and video metadata via Supadata API to eliminate IP blocking on serverless deployment platforms.
Hybrid search fusion
Combine ChromaDB dense semantic search with BM25 keyword matching via Reciprocal Rank Fusion for complete retrieval coverage.
Zero hallucination grounding
Constrain Groq LLM responses to retrieved transcript chunks, complete with timestamped source references.
AuraNode turns that strategy into one native workspace with a provider-neutral backend.
Instant URL transcript ingestion
Paste any public YouTube URL to automatically extract video titles, thumbnails, channel data, and full transcripts.
Hybrid dense & sparse retrieval
Index transcript chunks in ChromaDB with HuggingFace all-MiniLM-L6-v2 embeddings and a rank_bm25 keyword index.
Reciprocal Rank Fusion (RRF)
Merge semantic and exact-keyword search scores dynamically to supply optimal evidence passages to the LLM.
Grounded Groq LLM response
Generate answers powered by Groq's high-speed openai/gpt-oss-120b model with explicit transcript timestamp citations.
System architecture
A Streamlit web interface processes YouTube video URLs, fetching transcripts via Supadata API and executing hybrid RAG (ChromaDB + BM25 with Reciprocal Rank Fusion) using Groq gpt-oss-120b models.
Streamlit UI
Single-page interface accepting YouTube URLs, managing session state, displaying interactive transcripts, and presenting grounded Q&A with source attribution.
- Streamlit
- Python
- Session State
Supadata Loader & Splitter
Fetches full video transcripts and metadata via Supadata API (bypassing cloud IP blocks) and chunks text into 500-character overlapping passages using LangChain.
- Supadata API
- LangChain
- Text Splitter
ChromaDB + BM25 RRF Engine
Combines dense vector similarity (ChromaDB with HuggingFace all-MiniLM-L6-v2) and lexical keyword matching (BM25) fused via Reciprocal Rank Fusion (RRF).
- ChromaDB
- BM25
- Reciprocal Rank Fusion
Groq LLM Engine
Sends top fused transcript passages to Groq (openai/gpt-oss-120b) to synthesize concise, grounded video explanations and answer user queries.
- Groq
- gpt-oss-120b
- LangChain
Supadata API
Bypasses YouTube IP blocking on cloud infrastructure to reliably extract video transcripts, timestamps, titles, and channel metadata.
ChromaDB & BM25 Index
In-memory ChromaDB vector store paired with rank_bm25 for ultra-fast hybrid search over active video transcript chunks.
Streamlit Cloud
Deploys the Python Streamlit web app directly from GitHub with managed API keys and zero server configuration.
Request lifecycle
When a user inputs a YouTube video URL into the Streamlit interface, the application calls the Supadata API to retrieve the video transcript and metadata without encountering cloud IP blocking restrictions. LangChain document splitters chunk the text into 500-character passages with 100-character overlap. Each chunk is embedded locally using HuggingFace all-MiniLM-L6-v2 into an in-memory ChromaDB vector store while simultaneously building a BM25 keyword index. When the user asks a question, a hybrid retriever runs dense vector search and BM25 lexical search in parallel, fusing top results using Reciprocal Rank Fusion (RRF). The fused transcript context is forwarded to Groq (openai/gpt-oss-120b) to generate a grounded answer with timestamped references.
Separation of concerns
The Streamlit interface isolates user interaction, URL input validation, and chat history management from the backend RAG pipeline. Supadata client modules handle external transcript fetching logic, LangChain text splitters manage passage chunking, ChromaDB and BM25 manage hybrid retrieval, and Groq handles LLM reasoning. This modular design keeps transcript processing resilient across video lengths.
Key experiences
The application simplifies video research by transforming any YouTube video into an interactive Q&A knowledge base.
One-click URL input
Paste a YouTube link and watch as the bot fetches transcripts, builds vector indices, and prepares the chat workspace.
Interactive video chat interface
Ask natural language questions about video concepts, key takeaways, code examples, or specific quotes.
Timestamped transcript references
Inspect generated answers with highlighted transcript passages and exact video timestamp markers.
Full transcript explorer
Expand the full video transcript drawer to review raw text alongside AI-generated summaries.
Engineering decisions
Supadata API for reliable cloud ingestion
Replaced fragile local scraper libraries with Supadata API to guarantee 100% uptime on Streamlit Cloud.
ChromaDB & BM25 hybrid search architecture
Combines vector similarity with BM25 keyword matching to capture both conceptual meaning and exact proper nouns.
Groq LLM for real-time video Q&A
Groq high-speed inference engine ensures near-instant answer generation for fluid user conversations.
In-memory session vector indexing
Stores transcript vectors in temporary in-memory ChromaDB indices for instant cleanup when users change videos.
Streamlit Cloud serverless hosting
Deployed directly from GitHub to Streamlit Cloud with environment secrets management.
A cloud-resilient video RAG assistant.
YouTube Explainer Bot turns any YouTube video into an interactive, searchable knowledge base powered by hybrid RAG.
The project is deployed on Streamlit Cloud with an open-source GitHub repository and modular Python backend architecture.
- Supadata API integration bypassing cloud platform IP restrictions
- Hybrid retrieval combining ChromaDB dense search and BM25 lexical matching via RRF
- Groq LLM inference for near-instant video question answering
- Streamlit Cloud live deployment with GitHub continuous deployment
- Open-source codebase available on GitHub