Evaluating a business assistant before extending its use

Identify the information used, the steps of the work and the checks required. This description serves as the point of comparison with the assisted way of working.

Define what the assistant should produce and what remains the user’s responsibility.

Use authorised data within an access framework agreed with the relevant owners. Choose usual situations as well as incomplete or ambiguous cases.

The aim is to understand the limits of the output, not just to get a satisfactory answer on one example.

Record the time spent preparing the request, reviewing the answer and correcting it. Also gather the difficulties users encounter.

Generation speed alone is not enough to judge the assistant’s value.

Compare the observations with the initial expectations. You can adjust the scope, rework some elements or extend the use when the results justify it.

Document the limits identified and how users should deal with them. This information supports the continuation of the project.