jeremylongshore

firecrawl-advanced-troubleshooting

@jeremylongshore/firecrawl-advanced-troubleshooting
jeremylongshore
1,004
123 forks
Updated 1/18/2026
View on GitHub

Apply FireCrawl advanced debugging techniques for hard-to-diagnose issues. Use when standard troubleshooting fails, investigating complex race conditions, or preparing evidence bundles for FireCrawl support escalation. Trigger with phrases like "firecrawl hard bug", "firecrawl mystery error", "firecrawl impossible to debug", "difficult firecrawl issue", "firecrawl deep debug".

Installation

$skills install @jeremylongshore/firecrawl-advanced-troubleshooting
Claude Code
Cursor
Copilot
Codex
Antigravity

Details

Pathplugins/saas-packs/firecrawl-pack/skills/firecrawl-advanced-troubleshooting/SKILL.md
Branchmain
Scoped Name@jeremylongshore/firecrawl-advanced-troubleshooting

Usage

After installing, this skill will be available to your AI coding assistant.

Verify installation:

skills list

Skill Instructions


name: firecrawl-advanced-troubleshooting description: | Apply FireCrawl advanced debugging techniques for hard-to-diagnose issues. Use when standard troubleshooting fails, investigating complex race conditions, or preparing evidence bundles for FireCrawl support escalation. Trigger with phrases like "firecrawl hard bug", "firecrawl mystery error", "firecrawl impossible to debug", "difficult firecrawl issue", "firecrawl deep debug". allowed-tools: Read, Grep, Bash(kubectl:), Bash(curl:), Bash(tcpdump:*) version: 1.0.0 license: MIT author: Jeremy Longshore jeremy@intentsolutions.io

FireCrawl Advanced Troubleshooting

Overview

Deep debugging techniques for complex FireCrawl issues that resist standard troubleshooting.

Prerequisites

  • Access to production logs and metrics
  • kubectl access to clusters
  • Network capture tools available
  • Understanding of distributed tracing

Evidence Collection Framework

Comprehensive Debug Bundle

#!/bin/bash
# advanced-firecrawl-debug.sh

BUNDLE="firecrawl-advanced-debug-$(date +%Y%m%d-%H%M%S)"
mkdir -p "$BUNDLE"/{logs,metrics,network,config,traces}

# 1. Extended logs (1 hour window)
kubectl logs -l app=firecrawl-integration --since=1h > "$BUNDLE/logs/pods.log"
journalctl -u firecrawl-service --since "1 hour ago" > "$BUNDLE/logs/system.log"

# 2. Metrics dump
curl -s localhost:9090/api/v1/query?query=firecrawl_requests_total > "$BUNDLE/metrics/requests.json"
curl -s localhost:9090/api/v1/query?query=firecrawl_errors_total > "$BUNDLE/metrics/errors.json"

# 3. Network capture (30 seconds)
timeout 30 tcpdump -i any port 443 -w "$BUNDLE/network/capture.pcap" &

# 4. Distributed traces
curl -s localhost:16686/api/traces?service=firecrawl > "$BUNDLE/traces/jaeger.json"

# 5. Configuration state
kubectl get cm firecrawl-config -o yaml > "$BUNDLE/config/configmap.yaml"
kubectl get secret firecrawl-secrets -o yaml > "$BUNDLE/config/secrets-redacted.yaml"

tar -czf "$BUNDLE.tar.gz" "$BUNDLE"
echo "Advanced debug bundle: $BUNDLE.tar.gz"

Systematic Isolation

Layer-by-Layer Testing

// Test each layer independently
async function diagnoseFireCrawlIssue(): Promise<DiagnosisReport> {
  const results: DiagnosisResult[] = [];

  // Layer 1: Network connectivity
  results.push(await testNetworkConnectivity());

  // Layer 2: DNS resolution
  results.push(await testDNSResolution('api.firecrawl.com'));

  // Layer 3: TLS handshake
  results.push(await testTLSHandshake('api.firecrawl.com'));

  // Layer 4: Authentication
  results.push(await testAuthentication());

  // Layer 5: API response
  results.push(await testAPIResponse());

  // Layer 6: Response parsing
  results.push(await testResponseParsing());

  return { results, firstFailure: results.find(r => !r.success) };
}

Minimal Reproduction

// Strip down to absolute minimum
async function minimalRepro(): Promise<void> {
  // 1. Fresh client, no customization
  const client = new FireCrawlClient({
    apiKey: process.env.FIRECRAWL_API_KEY!,
  });

  // 2. Simplest possible call
  try {
    const result = await client.ping();
    console.log('Ping successful:', result);
  } catch (error) {
    console.error('Ping failed:', {
      message: error.message,
      code: error.code,
      stack: error.stack,
    });
  }
}

Timing Analysis

class TimingAnalyzer {
  private timings: Map<string, number[]> = new Map();

  async measure<T>(label: string, fn: () => Promise<T>): Promise<T> {
    const start = performance.now();
    try {
      return await fn();
    } finally {
      const duration = performance.now() - start;
      const existing = this.timings.get(label) || [];
      existing.push(duration);
      this.timings.set(label, existing);
    }
  }

  report(): TimingReport {
    const report: TimingReport = {};
    for (const [label, times] of this.timings) {
      report[label] = {
        count: times.length,
        min: Math.min(...times),
        max: Math.max(...times),
        avg: times.reduce((a, b) => a + b, 0) / times.length,
        p95: this.percentile(times, 95),
      };
    }
    return report;
  }
}

Memory and Resource Analysis

// Detect memory leaks in FireCrawl client usage
const heapUsed: number[] = [];

setInterval(() => {
  const usage = process.memoryUsage();
  heapUsed.push(usage.heapUsed);

  // Alert on sustained growth
  if (heapUsed.length > 60) { // 1 hour at 1/min
    const trend = heapUsed[59] - heapUsed[0];
    if (trend > 100 * 1024 * 1024) { // 100MB growth
      console.warn('Potential memory leak in firecrawl integration');
    }
  }
}, 60000);

Race Condition Detection

// Detect concurrent access issues
class FireCrawlConcurrencyChecker {
  private inProgress: Set<string> = new Set();

  async execute<T>(key: string, fn: () => Promise<T>): Promise<T> {
    if (this.inProgress.has(key)) {
      console.warn(`Concurrent access detected for ${key}`);
    }

    this.inProgress.add(key);
    try {
      return await fn();
    } finally {
      this.inProgress.delete(key);
    }
  }
}

Support Escalation Template

## FireCrawl Support Escalation

**Severity:** P[1-4]
**Request ID:** [from error response]
**Timestamp:** [ISO 8601]

### Issue Summary
[One paragraph description]

### Steps to Reproduce
1. [Step 1]
2. [Step 2]

### Expected vs Actual
- Expected: [behavior]
- Actual: [behavior]

### Evidence Attached
- [ ] Debug bundle (firecrawl-advanced-debug-*.tar.gz)
- [ ] Minimal reproduction code
- [ ] Timing analysis
- [ ] Network capture (if relevant)

### Workarounds Attempted
1. [Workaround 1] - Result: [outcome]
2. [Workaround 2] - Result: [outcome]

Instructions

Step 1: Collect Evidence Bundle

Run the comprehensive debug script to gather all relevant data.

Step 2: Systematic Isolation

Test each layer independently to identify the failure point.

Step 3: Create Minimal Reproduction

Strip down to the simplest failing case.

Step 4: Escalate with Evidence

Use the support template with all collected evidence.

Output

  • Comprehensive debug bundle collected
  • Failure layer identified
  • Minimal reproduction created
  • Support escalation submitted

Error Handling

IssueCauseSolution
Can't reproduceRace conditionAdd timing analysis
Intermittent failureTiming-dependentIncrease sample size
No useful logsMissing instrumentationAdd debug logging
Memory growthResource leakUse heap profiling

Examples

Quick Layer Test

# Test each layer in sequence
curl -v https://api.firecrawl.com/health 2>&1 | grep -E "(Connected|TLS|HTTP)"

Resources

Next Steps

For load testing, see firecrawl-load-scale.

More by jeremylongshore

View all
rabbitmq-queue-setup
1,004

Rabbitmq Queue Setup - Auto-activating skill for Backend Development. Triggers on: rabbitmq queue setup, rabbitmq queue setup Part of the Backend Development skill category.

model-evaluation-suite
1,004

evaluating-machine-learning-models: This skill allows Claude to evaluate machine learning models using a comprehensive suite of metrics. It should be used when the user requests model performance analysis, validation, or testing. Claude can use this skill to assess model accuracy, precision, recall, F1-score, and other relevant metrics. Trigger this skill when the user mentions "evaluate model", "model performance", "testing metrics", "validation results", or requests a comprehensive "model evaluation".

neural-network-builder
1,004

building-neural-networks: This skill allows Claude to construct and configure neural network architectures using the neural-network-builder plugin. It should be used when the user requests the creation of a new neural network, modification of an existing one, or assistance with defining the layers, parameters, and training process. The skill is triggered by requests involving terms like "build a neural network," "define network architecture," "configure layers," or specific mentions of neural network types (e.g., "CNN," "RNN," "transformer").

oauth-callback-handler
1,004

Oauth Callback Handler - Auto-activating skill for API Integration. Triggers on: oauth callback handler, oauth callback handler Part of the API Integration skill category.