Research method
Analysis Methodology
ReplyPilot separates deterministic data handling and risk controls from model-generated grouping and writing. This page documents the current analysis and benchmark method.
Updated July 23, 2026
Input and normalization
CSV and connected-source records are mapped into a common support-message structure. The workflow preserves source identifiers, customer text, subject, channel, status, tags, and source time where available. Workspace-scoped deduplication prevents repeated imports from becoming new demand.
- Customer-authored messages are the demand unit
- Agent replies and private notes are excluded where source metadata allows
- Stable external identifiers support deduplication
- Original CSV files are stored separately from normalized records
Analysis stages
A deterministic local pass identifies known risk language and baseline topics. The AI report then uses two stages: topic clustering followed by report composition. High-risk deterministic results cannot be downgraded by the generative stage.
Current model benchmark
On July 23, 2026, ReplyPilot tested Cloudflare Workers AI Qwen3 30B A3B FP8 on a synthetic 60-message support CSV. The two-stage request produced 7 themes and 13 action assets, used 5,837 total tokens, completed in approximately 16.7 seconds, and had an estimated model inference cost of about $0.00070. This is a technical benchmark, not a customer outcome.
- Model: @cf/qwen/qwen3-30b-a3b-fp8
- Dataset: synthetic support messages
- Maximum analyzed rows per request: 100
- Cost excludes reviewer time and other infrastructure
Operational validation
A useful report must survive persistence, review, export, and later retrieval. ReplyPilot tracks import completion, report completion, asset status, draft approval, export, write-back, and second-import events. Customer value and willingness to pay still require real pilot evidence.