Internal LLM

PythonAWS LambdaClaudeMongoDBLangChainRAG

Architecture Overview

Hover over the highlighted elements in the diagram to reveal additional details.

Chatbot
internal-chatbot.com
What were the top selling products last month?
Example document: { "_id": ....., "table_name": "transactions", "column_name": "transaction date", "embeddings": [0.000232, ......], "embedding_text": "transaction date\nDate and time when the transaction occurred", "context": " Table: transactions Description: Table containing full transaction data Column: transaction date Type: TIMESTAMP WITHOUT TIME ZONE (nullable) Description: Date and time when the transaction occurred " }Example document: { "_id": ....., "table_name": "transactions", "column_name": "transaction date", "embeddings": [0.000232, ......], "embedding_text": "transaction date\nDate and time when the transaction occurred", "context": " Table: transactions Description: Table containing full transaction data Column: transaction date Type: TIMESTAMP WITHOUT TIME ZONE (nullable) Description: Date and time when the transaction occurred " }
Collection: redshiftEmbeddings
AWS 
Redshift
Table 1
Col A1
Col B1
Col C1
...
...
...
Table N
Col AN
Col BN
Col CN
...
...
...
•
•
•
AWS
Lambda
FastAPI App
This lambda function is executed as part of the initial setup. It should be executed only if the relevant Redshift tables/columns change.This lambda function is executed as part of the initial setup. It should be executed only if the relevant Redshift tables/columns change.
Embed Redshift Columns
(setup for RAG)
1) Get all table & column data
3) Embed each column
as a document
APIs
Consists of
Creates
Chats
API
AWS
Bedrock
 Claude
Claude LLM
Amazon Titan
2) Get embedding model
1) Connect model
1. Agent extracts keywords from user query 2. Agent embeds the keywords 3. Agent performs a vector search to find relevant columns.
2) Get relevant context (RAG)
3) Build & execute queries
User query (API call)
The agent has the following components: - LLM: The model which will act as an agent. In this case it's Claude, provided through Bedrock. - System prompt: instructions for the LLM on how to execute it's tasks. - Tools: functions with clear documentation. The LLM explores these functions and decides when to use each. (E.g. GetRelevantContextTool, ExecuteRedshiftQueryTool).
LangChain Agent

Project Details

An enterprise-grade AI agent system I architected to enable employees to interact with company data using natural language queries. The system combines AWS Bedrock (Claude), Retrieval-Augmented Generation (RAG), and intelligent query building to transform conversational questions into executable SQL queries against our AWS Redshift data warehouse.

Key Features

  • Natural language to SQL query conversion using Claude LLM
  • RAG-based context retrieval for accurate query building
  • Real-time query execution against AWS Redshift
  • Conversational histroy allowing back and forth complex data analysis
  • Serverless architecture for cost-effective scaling
  • Enterprise security with proper access controls

Technical Architecture

  • AWS Lambda serverless deployment with FastAPI framework
  • AWS Bedrock integration with Claude LLM for natural language processing
  • Amazon Titan embeddings for vector representation of database schema
  • MongoDB vector storage for schema embeddings and metadata
  • AWS Redshift integration for data warehouse queries
  • Comprehensive schema documentation with column descriptions and constraints
  • LangChain agent framework for orchestrating the query pipeline

RAG Implementation

  • Database schema embedded as contextual documents in MongoDB
  • Each document contains column metadata: name, description, type, possible values, table context
  • Intelligent keyword extraction from user queries
  • Vector similarity search to retrieve relevant schema context
  • Dynamic SQL query generation based on retrieved context
  • Query validation and execution with result formatting