AI-assisted data processing services

How a data processing engagement runs

1. 01

   Define outputs

   Fields, formats and allowed values agreed.

   Output: Output specification

2. 02

   Pilot

   A sample with known answers processed and compared.

   Output: Error report by type

3. 03

   Full run

   Remaining data processed with validation; failures routed for review.

   Output: Processed data and review queue

4. 04

   Verification

   Random sample of accepted results checked; final report delivered.

   Output: Verified results

Insist on a pilot

Before processing all your data, the provider should run a pilot on a sample where the correct answers are known, ideally prepared by your team. The pilot shows the error rate and, more usefully, the types of error: misread numbers, wrong categories, missing fields. You can then decide whether the accuracy is good enough, what needs human review and whether the full run is worth doing.

A provider unwilling to run a pilot is asking you to trust results you cannot check.

What you should receive

Questions to ask a provider

Warning signs

• Accuracy promised in advance

  It can only be measured on your data.

• No link back to sources

  You cannot verify or correct results.

• Unclear data handling

  You need to know where your data goes and when it is deleted.

Data handling during and after the work

Agree in writing where data will be stored during processing, which AI provider will process it and under what terms, who has access and when copies will be deleted. For one-off jobs, a confirmation of deletion at the end is reasonable. For ongoing processing, retention of logs should match your own policies. Minimising the fields sent to the AI step reduces risk throughout.

From one-off to ongoing

A successful one-off clean-up often leads to a question: can new data be processed the same way as it arrives? The method developed for the clean-up, with its prompts, rules and validation, usually becomes the basis of an ongoing workflow, which then needs continuous sampling to make sure accuracy holds as new kinds of input appear.

Preparing the pilot sample

The pilot sample should represent the full range of your data, not just the tidiest examples: different layouts, older records, partial entries and the formats that cause your team the most trouble. Someone who knows the data should note the correct answer for each sample item. This sample becomes the yardstick for the whole job and is worth the time to get right.

If personal data is involved, mask or remove details that the processing does not need before sharing the sample.

Agreeing acceptable accuracy

Different fields need different accuracy. A misread invoice total matters more than a slightly wrong product colour name. Agreeing per field what level of accuracy is acceptable, and which fields always need human confirmation, sets clear expectations and shapes how the review queue is designed.

A usable review queue

Items needing review should be easy to work through: the source document or record shown beside the suggested result, the reason it was flagged and a quick way to accept or correct it. A spreadsheet with links can be enough for one-off jobs; ongoing processing benefits from a simple review screen in a tool your team already uses.