retellai-load-scale
Implement Retell AI load testing, auto-scaling, and capacity planning strategies. Use when running performance tests, configuring horizontal scaling, or planning capacity for Retell AI integrations. Trigger with phrases like "retellai load test", "retellai scale", "retellai performance test", "retellai capacity", "retellai k6", "retellai benchmark".
Installation
Details
Usage
After installing, this skill will be available to your AI coding assistant.
Verify installation:
skills listSkill Instructions
name: retellai-load-scale description: | Implement Retell AI load testing, auto-scaling, and capacity planning strategies. Use when running performance tests, configuring horizontal scaling, or planning capacity for Retell AI integrations. Trigger with phrases like "retellai load test", "retellai scale", "retellai performance test", "retellai capacity", "retellai k6", "retellai benchmark". allowed-tools: Read, Write, Edit, Bash(k6:), Bash(kubectl:) version: 1.0.0 license: MIT author: Jeremy Longshore jeremy@intentsolutions.io
Retell AI Load & Scale
Overview
Load testing, scaling strategies, and capacity planning for Retell AI integrations.
Prerequisites
- k6 load testing tool installed
- Kubernetes cluster with HPA configured
- Prometheus for metrics collection
- Test environment API keys
Load Testing with k6
Basic Load Test
// retellai-load-test.js
import http from 'k6/http';
import { check, sleep } from 'k6';
export const options = {
stages: [
{ duration: '2m', target: 10 }, // Ramp up
{ duration: '5m', target: 10 }, // Steady state
{ duration: '2m', target: 50 }, // Ramp to peak
{ duration: '5m', target: 50 }, // Stress test
{ duration: '2m', target: 0 }, // Ramp down
],
thresholds: {
http_req_duration: ['p(95)<500'],
http_req_failed: ['rate<0.01'],
},
};
export default function () {
const response = http.post(
'https://api.retellai.com/v1/resource',
JSON.stringify({ test: true }),
{
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${__ENV.RETELLAI_API_KEY}`,
},
}
);
check(response, {
'status is 200': (r) => r.status === 200,
'latency < 500ms': (r) => r.timings.duration < 500,
});
sleep(1);
}
Run Load Test
# Install k6
brew install k6 # macOS
# or: sudo apt install k6 # Linux
# Run test
k6 run --env RETELLAI_API_KEY=${RETELLAI_API_KEY} retellai-load-test.js
# Run with output to InfluxDB
k6 run --out influxdb=http://localhost:8086/k6 retellai-load-test.js
Scaling Patterns
Horizontal Scaling
# kubernetes HPA
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: retellai-integration-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: retellai-integration
minReplicas: 2
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Pods
pods:
metric:
name: retellai_queue_depth
target:
type: AverageValue
averageValue: 100
Connection Pooling
import { Pool } from 'generic-pool';
const retellaiPool = Pool.create({
create: async () => {
return new RetellAIClient({
apiKey: process.env.RETELLAI_API_KEY!,
});
},
destroy: async (client) => {
await client.close();
},
max: 20,
min: 5,
idleTimeoutMillis: 30000,
});
async function withRetell AIClient<T>(
fn: (client: RetellAIClient) => Promise<T>
): Promise<T> {
const client = await retellaiPool.acquire();
try {
return await fn(client);
} finally {
retellaiPool.release(client);
}
}
Capacity Planning
Metrics to Monitor
| Metric | Warning | Critical |
|---|---|---|
| CPU Utilization | > 70% | > 85% |
| Memory Usage | > 75% | > 90% |
| Request Queue Depth | > 100 | > 500 |
| Error Rate | > 1% | > 5% |
| P95 Latency | > 1000ms | > 3000ms |
Capacity Calculation
interface CapacityEstimate {
currentRPS: number;
maxRPS: number;
headroom: number;
scaleRecommendation: string;
}
function estimateRetell AICapacity(
metrics: SystemMetrics
): CapacityEstimate {
const currentRPS = metrics.requestsPerSecond;
const avgLatency = metrics.p50Latency;
const cpuUtilization = metrics.cpuPercent;
// Estimate max RPS based on current performance
const maxRPS = currentRPS / (cpuUtilization / 100) * 0.7; // 70% target
const headroom = ((maxRPS - currentRPS) / currentRPS) * 100;
return {
currentRPS,
maxRPS: Math.floor(maxRPS),
headroom: Math.round(headroom),
scaleRecommendation: headroom < 30
? 'Scale up soon'
: headroom < 50
? 'Monitor closely'
: 'Adequate capacity',
};
}
Benchmark Results Template
## Retell AI Performance Benchmark
**Date:** YYYY-MM-DD
**Environment:** [staging/production]
**SDK Version:** X.Y.Z
### Test Configuration
- Duration: 10 minutes
- Ramp: 10 → 100 → 10 VUs
- Target endpoint: /v1/resource
### Results
| Metric | Value |
|--------|-------|
| Total Requests | 50,000 |
| Success Rate | 99.9% |
| P50 Latency | 120ms |
| P95 Latency | 350ms |
| P99 Latency | 800ms |
| Max RPS Achieved | 150 |
### Observations
- [Key finding 1]
- [Key finding 2]
### Recommendations
- [Scaling recommendation]
Instructions
Step 1: Create Load Test Script
Write k6 test script with appropriate thresholds.
Step 2: Configure Auto-Scaling
Set up HPA with CPU and custom metrics.
Step 3: Run Load Test
Execute test and collect metrics.
Step 4: Analyze and Document
Record results in benchmark template.
Output
- Load test script created
- HPA configured
- Benchmark results documented
- Capacity recommendations defined
Error Handling
| Issue | Cause | Solution |
|---|---|---|
| k6 timeout | Rate limited | Reduce RPS |
| HPA not scaling | Wrong metrics | Verify metric name |
| Connection refused | Pool exhausted | Increase pool size |
| Inconsistent results | Warm-up needed | Add ramp-up phase |
Examples
Quick k6 Test
k6 run --vus 10 --duration 30s retellai-load-test.js
Check Current Capacity
const metrics = await getSystemMetrics();
const capacity = estimateRetell AICapacity(metrics);
console.log('Headroom:', capacity.headroom + '%');
console.log('Recommendation:', capacity.scaleRecommendation);
Scale HPA Manually
kubectl scale deployment retellai-integration --replicas=5
kubectl get hpa retellai-integration-hpa
Resources
Next Steps
For reliability patterns, see retellai-reliability-patterns.
More by jeremylongshore
View allRabbitmq Queue Setup - Auto-activating skill for Backend Development. Triggers on: rabbitmq queue setup, rabbitmq queue setup Part of the Backend Development skill category.
evaluating-machine-learning-models: This skill allows Claude to evaluate machine learning models using a comprehensive suite of metrics. It should be used when the user requests model performance analysis, validation, or testing. Claude can use this skill to assess model accuracy, precision, recall, F1-score, and other relevant metrics. Trigger this skill when the user mentions "evaluate model", "model performance", "testing metrics", "validation results", or requests a comprehensive "model evaluation".
building-neural-networks: This skill allows Claude to construct and configure neural network architectures using the neural-network-builder plugin. It should be used when the user requests the creation of a new neural network, modification of an existing one, or assistance with defining the layers, parameters, and training process. The skill is triggered by requests involving terms like "build a neural network," "define network architecture," "configure layers," or specific mentions of neural network types (e.g., "CNN," "RNN," "transformer").
Oauth Callback Handler - Auto-activating skill for API Integration. Triggers on: oauth callback handler, oauth callback handler Part of the API Integration skill category.