Multi-Agent Platform Stability for Insurance Claims: A Scenario-Based Benchmark of Three Open-Weight Models

Why Multi-Agent Stability Matters for Insurance Claims Processing As of May 22, 2026, insurance carriers are accelerating the adoption of multi-agent systems for claims triage, fraud detection, and settlement negotiation. These systems promise faster throughput and reduced operational costs, but production deployments reveal a persistent challenge: stability under sustained, high-volume workloads. Unlike traditional rule-based automation, multi-agent systems rely on LLM reasoning, inter-agent coordination, and retrieval-augmented generation (RAG) to handle nuanced claims—making them vulnerable to coordination deadlocks, latency spikes, and embedding drift after model updates. For operations leaders evaluating multi-agent platform stability in insurance claims, the question isn't whether AI can process a single claim, but whether the system can maintain consistent performance over an enti

re 8-hour shift while handling 500 concurrent claims. This article provides a vendor-neutral, scenario-based evaluation framework using three open-weight models (Qwen 3.7 Max, Composer 2.5, and Gemma 3) on three leading multi-agent platforms: LUMOS, AutoGen, and LangGraph. We benchmark task completion rate, latency variance, and RAG citation accuracy drift, and supply a decision matrix to align automation levels with regulatory risk appetite. Benchmarking Methodology: Simulated Claims Pipeline with 500 Concurrent Claims To simulate a realistic property-casualty claims environment, we built a pipeline mirroring a mid-size carrier's daily intake: first notice of loss, policy validation, damage assessment, fraud scoring, and settlement offer generation. Each platform—LUMOS (latest open-source release on GitHub), AutoGen (v0.4.2, Microsoft's multi-agent conversation framework), and LangGraph

(v3.1, LangChain's graph-based agent orchestration)—was configured with identical agent roles: triage agent, policy checker, fraud analyst, settlement negotiator, and supervisor agent. All agents shared a common RAG store of 10,000 policy documents and historical claim records (embedding model: e5-large-v2 for parity). We tested each platform with three open-weight LLMs as the backbone: Qwen 3.7 Max (Alibaba Cloud, released November 2025) – 72B parameter model optimized for long-context reasoning. Composer 2.5 (Together AI, released January 2026) – 48B mixture-of-experts model with strong instruction following. Gemma 3 (Google, released March 2026) – 27B model with improved multilingual capabilities. All models were deployed on a consistent infrastructure: 4x NVIDIA H100 GPUs per instance, 512 GB RAM, with identical request queuing via Redis. The simulation injected 500 concurrent claim

s at random intervals over an 8-hour shift, with each claim requiring 3–7 agent interactions. We recorded metrics from three consecutive runs per platform-model combination. Key Stability Metrics: Task Completion, Latency Variance, and Embedding Drift We focused on three metrics crucial for multi-agent platform stability in insurance claims: 1. Task Completion Rate – Percentage of claims that reached a final settlement or escalation decision within 8 hours, excluding any claims dropped due to agent timeout, coordination failure, or unrecoverable error. 2. Latency Variance – Standard deviation of end-to-end claim processing time (from intake to decision) across all completed claims. Low variance indicates predictable performance even under load. 3. Embedding Drift – Measured as the drop in RAG citation accuracy (exact policy clause recall) after a live model weight update (simulated by ho

t-swapping the embedding model to a newer version mid-run). This tests how well the multi-agent system adapts to embedding changes without manual recalibration. Each metric was recorded per platform-model pair, and averages across the three runs were computed. Results are from a controlled simulation; real production environments with custom data, network latency, and concurrency may vary. Results: How LUM