The short answer

Summarizing long email threads is one of the generative AI use cases that can deliver measurable value without granting the model authority to make risky decisions. German SaaS company Epilot embedded this capability in its platform for energy companies. According to an AWS customer case study, users save 87% of the time spent handling email threads, while the service produces about 55,000 summaries per month.

Those figures are reported by the cloud provider together with its customer, so they should not be transferred directly into another company's business case. The implementation sequence is still useful: user interviews first, a two-month MVP, human evaluation of models and prompts, and production only after that. For a small or medium-sized business, the main lesson is not the choice of a particular cloud but the narrow scope and testable architecture.

What work was automated

Epilot develops an XRM platform for energy suppliers, municipal utilities, grid operators and solution providers. Sales and service employees receive long email threads: the newest message may be short, but making a decision requires reconstructing the context, commitments, promised dates and expected next action.

Instead of building a universal “email agent,” the company selected one limited function: at an employee's request, read a thread and return a few key points. This reduces risk for several reasons:

  • the original messages remain available for verification;
  • the output first helps a person understand the situation instead of changing records or sending a customer response;
  • quality can be evaluated on real threads against understandable criteria;
  • a flawed summary is easier to detect and correct than an incorrectly executed transaction.

According to the AWS case study, experimentation began in December 2023. The team conducted user research and delivered an MVP in about two months. Before production, product specialists and engineers compared models and instructions through human-based evaluation: people rated outputs from different email threads against defined success criteria. The team ultimately selected Claude Sonnet through Amazon Bedrock.

AWS and Epilot report that roughly 80% of users say the feature makes their work easier and that time savings reach 87%. The wording matters: this is a reported result for one product and audience, not an independently replicated industry benchmark.

Architecture: why email and the model need a queue between them

In the published solution, an API receives a summary request and places the job in Amazon SQS. A queue event triggers AWS Lambda, which orchestrates the model call in Bedrock. This asynchronous design separates the user interface from inference: a spike in requests does not need to create the same number of simultaneous direct connections to the model.

A private or local deployment may use different components, but the roles remain the same:

1. **Connector:** fetches the selected thread from the email system or CRM.
2. **Normalizer:** removes repeated quotations, signatures and service headers while preserving authors, timestamps and message order.
3. **Queue:** accepts jobs and limits concurrency.
4. **Worker:** assembles the permitted context, calls the model and requests a structured output.
5. **Validator:** checks required fields, length, message references and refusal signals.
6. **Employee interface:** displays the summary beside the original conversation.
7. **Telemetry:** records latency, cost, model version, user rating and whether a correction was required.

A queue does not guarantee exactly-once execution. AWS documentation for Lambda with SQS explicitly warns that an event can be processed more than once. The worker therefore needs to be idempotent: replaying the same job must not create a duplicate summary, duplicate a CRM record or repeat an action. A practical idempotency key can combine the thread identifier, its latest version and the operation type.

What a useful summary should contain

A free-form paragraph may look polished, but it is difficult to validate or integrate into a process. A pilot should request a predictable structure:

  • the topic of the request;
  • confirmed facts and commitments;
  • unresolved questions;
  • promised dates together with the message author;
  • the next expected action;
  • a confidence signal or reason for refusal;
  • references to the message numbers supporting each important fact.

The model should not guess a missing date, payment status or customer intent. When messages conflict, a useful output identifies the contradiction and asks the employee to open specific messages. A summary without traceability can accelerate reading but makes verification harder, which is a material weakness in sensitive workflows.

Data and security

An email thread may contain personal data, commercial terms, contracts and attachments. Before a pilot, create a data map: which mailboxes are connected, which message categories are prohibited, where requests and responses are stored, who can see logs, how long caches live, and whether a provider uses content for training.

The minimum control set includes:

  • a separate service account with access only to the required folders;
  • encryption in transit and at rest;
  • masking unnecessary identifiers before the model call;
  • separate content logs and technical telemetry;
  • defined retention for prompts and outputs;
  • no automatic external sending in the first stage;
  • version records for the model, instruction template and preprocessing rules.

The NIST Generative AI Profile recommends managing data provenance, evaluating outputs, documenting limitations and applying human oversight in proportion to the consequences of error. For email, that produces a simple rule: “read and advise” can be automated before “update the customer record” or “promise a deadline.”

A local model makes sense when messages cannot leave a protected environment, workload is sufficiently stable, or complete control of logs is required. Locality alone does not solve access control, deletion or summary quality. A cloud API may deliver a pilot faster, while a local stack offers more control; the choice should follow data classification and workload measurement rather than a slogan.

How to test quality before production

Collect 150–300 anonymized threads representing real request types: short and long conversations, attachments, forwarding, subject changes, conflicting dates and multiple participants. For every thread, create a reference as a set of required facts and forbidden errors, not merely as one “perfectly worded” summary.

Measure at least five dimensions:

  • coverage of required facts;
  • rate of fabricated or distorted claims;
  • correctness of the next action;
  • employee time spent checking and editing;
  • rate of correct refusals or uncertainty flags.

An average score can hide rare but costly failures. Track missed deadlines, incorrect amounts, customer identity mix-ups and commitments attributed to the wrong participant separately. Acceptance thresholds should be stricter when summaries feed calculations or legally significant communication.

Shadow mode is useful: the system produces summaries, but employees continue their normal process and compare the results. The next stage can introduce an “accept summary” control, followed by a draft action. Automatic data updates should be treated as a separate capability and keep a human confirmation step, as Epilot does with its suggested actions feature.

Pilot economics

The relevant unit is not the price of one model call but the cost of one verified summary. A simple monthly formula is:

**savings = threads × minutes before automation − threads × verification minutes after automation**, multiplied by labor cost per hour, minus infrastructure, development, maintenance and error correction.

Assume a team handles 4,000 threads per month, spends six minutes on each and needs two minutes to verify a summary. The theoretical capacity released is about 267 hours. This is a model calculation: it must be reduced by unusable summaries, retries, quality control and operating effort.

Three business metrics are enough for a go/no-go decision: median time saved, share of summaries requiring no material edit and cost per accepted summary. If employees still read every thread from beginning to end, the feature may be convenient, but the economic benefit has not been demonstrated.

A practical next step

Choose one mailbox or request type where errors are reversible and threads are genuinely long. Spend two weeks building the corpus and acceptance criteria, then run a four-week shadow pilot. Do not enable automatic sending or CRM updates until critical-fact omissions have been measured and idempotency is in place.

The most useful lesson from the Epilot case is not simply that “a model reads email 87% faster.” It is the sequence: a narrow task, asynchronous architecture, human evaluation on real data, transparent limitations, and only then broader authority. That sequence is what turns a compelling demo into a manageable business tool.