ReplyPilot

Research method

Analysis Methodology

ReplyPilot separates deterministic data handling and risk controls from model-generated grouping and writing. This page documents the current analysis and benchmark method.

Updated July 23, 2026

Input and normalization

CSV and connected-source records are mapped into a common support-message structure. The workflow preserves source identifiers, customer text, subject, channel, status, tags, and source time where available. Workspace-scoped deduplication prevents repeated imports from becoming new demand.

  • Customer-authored messages are the demand unit
  • Agent replies and private notes are excluded where source metadata allows
  • Stable external identifiers support deduplication
  • Original CSV files are stored separately from normalized records

Analysis stages

A deterministic local pass identifies known risk language and baseline topics. The AI report then uses two stages: topic clustering followed by report composition. High-risk deterministic results cannot be downgraded by the generative stage.

Current model benchmark

On July 23, 2026, ReplyPilot tested Cloudflare Workers AI Qwen3 30B A3B FP8 on a synthetic 60-message support CSV. The two-stage request produced 7 themes and 13 action assets, used 5,837 total tokens, completed in approximately 16.7 seconds, and had an estimated model inference cost of about $0.00070. This is a technical benchmark, not a customer outcome.

  • Model: @cf/qwen/qwen3-30b-a3b-fp8
  • Dataset: synthetic support messages
  • Maximum analyzed rows per request: 100
  • Cost excludes reviewer time and other infrastructure

Operational validation

A useful report must survive persistence, review, export, and later retrieval. ReplyPilot tracks import completion, report completion, asset status, draft approval, export, write-back, and second-import events. Customer value and willingness to pay still require real pilot evidence.