langchain-prod-checklist
Execute LangChain production deployment checklist. Use when preparing for production launch, validating deployment readiness, or auditing existing production LangChain applications. Trigger with phrases like "langchain production", "langchain prod ready", "deploy langchain", "langchain launch checklist", "production checklist".
Installation
Details
Usage
After installing, this skill will be available to your AI coding assistant.
Verify installation:
skills listSkill Instructions
name: langchain-prod-checklist description: | Execute LangChain production deployment checklist. Use when preparing for production launch, validating deployment readiness, or auditing existing production LangChain applications. Trigger with phrases like "langchain production", "langchain prod ready", "deploy langchain", "langchain launch checklist", "production checklist". allowed-tools: Read, Write, Edit, Bash(python:*) version: 1.0.0 license: MIT author: Jeremy Longshore jeremy@intentsolutions.io
LangChain Production Checklist
Overview
Comprehensive checklist for deploying LangChain applications to production with reliability, security, and performance.
Prerequisites
- LangChain application developed and tested
- Infrastructure provisioned
- CI/CD pipeline configured
Production Checklist
1. Configuration & Secrets
- All API keys in secrets manager (not env vars in code)
- Environment-specific configurations separated
- Fallback values for non-critical settings
- Configuration validation on startup
from pydantic_settings import BaseSettings
from pydantic import Field, SecretStr
class Settings(BaseSettings):
"""Validated configuration."""
openai_api_key: SecretStr = Field(..., env="OPENAI_API_KEY")
model_name: str = "gpt-4o-mini"
max_retries: int = Field(default=3, ge=1, le=10)
timeout_seconds: int = Field(default=30, ge=5, le=120)
class Config:
env_file = ".env"
settings = Settings() # Validates on import
2. Error Handling & Resilience
- Retry logic with exponential backoff
- Fallback models configured
- Circuit breaker for cascading failures
- Graceful degradation strategy
from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic
primary = ChatOpenAI(model="gpt-4o-mini", max_retries=3)
fallback = ChatAnthropic(model="claude-3-5-sonnet-20241022")
robust_llm = primary.with_fallbacks([fallback])
3. Observability
- Structured logging configured
- Metrics collection enabled
- Distributed tracing (LangSmith or OpenTelemetry)
- Alerting rules defined
import os
# LangSmith tracing
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"] = settings.langsmith_api_key
os.environ["LANGCHAIN_PROJECT"] = "production"
# Prometheus metrics
from prometheus_client import Counter, Histogram
llm_requests = Counter("langchain_llm_requests_total", "Total LLM requests")
llm_latency = Histogram("langchain_llm_latency_seconds", "LLM latency")
4. Performance
- Caching configured for repeated queries
- Connection pooling enabled
- Timeout limits set
- Batch processing for bulk operations
from langchain_core.globals import set_llm_cache
from langchain_community.cache import RedisCache
import redis
# Production caching with Redis
redis_client = redis.Redis.from_url(os.environ["REDIS_URL"])
set_llm_cache(RedisCache(redis_client))
5. Security
- Input validation implemented
- Output sanitization enabled
- Rate limiting per user/IP
- Audit logging for all LLM calls
from langchain_core.runnables import RunnableLambda
def validate_input(input_data: dict) -> dict:
"""Validate and sanitize input."""
user_input = input_data.get("input", "")
if len(user_input) > 10000:
raise ValueError("Input too long")
return input_data
secure_chain = RunnableLambda(validate_input) | prompt | llm
6. Testing
- Unit tests for all chains
- Integration tests with mock LLMs
- Load tests completed
- Chaos engineering (failure injection)
# pytest.ini
[pytest]
markers =
unit: Unit tests
integration: Integration tests
load: Load tests
7. Deployment
- Health check endpoint
- Graceful shutdown handling
- Rolling deployment strategy
- Rollback procedure documented
from fastapi import FastAPI
from contextlib import asynccontextmanager
@asynccontextmanager
async def lifespan(app: FastAPI):
# Startup
print("Warming up LLM connections...")
yield
# Shutdown
print("Cleaning up...")
app = FastAPI(lifespan=lifespan)
@app.get("/health")
async def health_check():
return {"status": "healthy", "model": settings.model_name}
8. Cost Management
- Token counting implemented
- Usage alerts configured
- Cost allocation by tenant/feature
- Budget limits enforced
import tiktoken
def estimate_cost(text: str, model: str = "gpt-4o-mini") -> float:
"""Estimate API cost for text."""
encoding = tiktoken.encoding_for_model(model)
tokens = len(encoding.encode(text))
# Approximate pricing (check current rates)
cost_per_1k = {"gpt-4o-mini": 0.00015, "gpt-4o": 0.005}
return (tokens / 1000) * cost_per_1k.get(model, 0.001)
Deployment Validation Script
#!/usr/bin/env python3
"""Pre-deployment validation script."""
def run_checks():
checks = []
# Check 1: API key configured
try:
settings = Settings()
checks.append(("API Key", "PASS"))
except Exception as e:
checks.append(("API Key", f"FAIL: {e}"))
# Check 2: LLM connectivity
try:
llm = ChatOpenAI(model="gpt-4o-mini")
llm.invoke("test")
checks.append(("LLM Connection", "PASS"))
except Exception as e:
checks.append(("LLM Connection", f"FAIL: {e}"))
# Check 3: Cache connectivity
try:
redis_client.ping()
checks.append(("Cache (Redis)", "PASS"))
except Exception as e:
checks.append(("Cache (Redis)", f"FAIL: {e}"))
for name, status in checks:
print(f"[{status}] {name}")
return all("PASS" in status for _, status in checks)
if __name__ == "__main__":
exit(0 if run_checks() else 1)
Resources
Next Steps
After launch, use langchain-observability for monitoring.
More by jeremylongshore
View allRabbitmq Queue Setup - Auto-activating skill for Backend Development. Triggers on: rabbitmq queue setup, rabbitmq queue setup Part of the Backend Development skill category.
evaluating-machine-learning-models: This skill allows Claude to evaluate machine learning models using a comprehensive suite of metrics. It should be used when the user requests model performance analysis, validation, or testing. Claude can use this skill to assess model accuracy, precision, recall, F1-score, and other relevant metrics. Trigger this skill when the user mentions "evaluate model", "model performance", "testing metrics", "validation results", or requests a comprehensive "model evaluation".
building-neural-networks: This skill allows Claude to construct and configure neural network architectures using the neural-network-builder plugin. It should be used when the user requests the creation of a new neural network, modification of an existing one, or assistance with defining the layers, parameters, and training process. The skill is triggered by requests involving terms like "build a neural network," "define network architecture," "configure layers," or specific mentions of neural network types (e.g., "CNN," "RNN," "transformer").
Oauth Callback Handler - Auto-activating skill for API Integration. Triggers on: oauth callback handler, oauth callback handler Part of the API Integration skill category.