AI Pilot to Production with Forward Deployed Engineering
Moving an AI pilot to production means making it work inside an actual business process, with defined quality, access controls and operational ownership. Forward deployed engineering keeps the same engineer close to the users and the implementation, helping the team resolve the decisions that a successful demonstration can leave unanswered.
A pilot can show that a model understands a document or drafts a useful reply. Production has a wider test. The system must receive the right inputs, handle ordinary exceptions and fit the way people work. When something fails, the team needs to know who takes over and what happens to unfinished work.
That gap is visible in broader adoption data. McKinsey’s 2025 survey found that 88% of respondents reported regular AI use in at least one business function, while approximately one-third said their organisations had begun scaling AI programmes. The survey covered 1,993 participants across 105 nations. Those are self-reported adoption measures, rather than a failure rate for individual pilots.
Why does a convincing pilot stall before production
The pilot often leaves critical assumptions untested. Its sample documents may be cleaner than daily inputs. A developer may have used broad access that ordinary users do not have. A helpful output may still require copying between systems and checking several facts manually.
Responsibility can also become fragmented. The business team recognises the value, engineering has a full roadmap and security receives an incomplete design. Each group waits for an answer from another. The next step is difficult to assign because it involves several groups at once.
An FDE can coordinate and implement that work, provided the client gives the engineer a business counterpart and access to decision makers. The engineer should maintain an explicit list of unresolved assumptions, the evidence needed to settle them and the person who can make each decision.
What should the first production scope include
Choose a complete but narrow unit of work. For example, a hypothetical support assistant might prepare a draft response for one category of ticket using an approved set of documents. A support colleague checks the draft before sending it. The initial scope excludes refunds, account changes and unsupported ticket categories.
That boundary makes evaluation and operation more manageable. The team can observe whether staff receive useful drafts, how long review takes and which cases require the existing process. The workflow is complete enough to assess, while the consequences remain controlled.
Document the conditions under which the system should decline or escalate. Missing customer identity, conflicting source documents or an unavailable integration are normal operational conditions. A system that recognises these conditions and routes the work correctly can be more useful than one that always produces an answer.
How can you structure the first 90 days
The following is a planning example for a bounded workflow with an available process owner. Access delays, complex integrations or formal approvals can extend the schedule. Treat each period as a set of evidence to produce, rather than a promise that every initiative will be live by a particular day.
| Period | Main work | Evidence for the next decision |
|---|---|---|
| Days 1 to 15 | Observe the workflow and confirm access | Agreed scope, baseline and dependency owners |
| Days 16 to 35 | Build a narrow implementation and evaluation set | Results on representative and failure cases |
| Days 36 to 60 | Integrate and test with a restricted user group | Measured review effort and operational behaviour |
| Days 61 to 90 | Improve reliability and agree ongoing ownership | Release decision, support plan and outcome review |
The first decision may be to stop or change direction. If essential data cannot be accessed, or the workflow has too little activity to justify the integration cost, discovering that early is a useful result. Preserve the evidence so that the organisation can revisit the decision if conditions change.
How do you evaluate more than a good demonstration
Create test cases from actual work, with approved handling of any sensitive data. Include common requests, ambiguous inputs and cases where the correct behaviour is to ask for help. A domain expert should be able to explain what makes each result acceptable.
In its January 2026 guidance on agent evaluation, Anthropic suggests that 20 to 50 simple tasks drawn from real failures can be a useful starting point. That is an early development practice, not a sufficient sample for proving a rare incident rate. Larger releases and higher-consequence actions require evaluation suited to their risks.
Separate development examples from cases reserved for assessment. Otherwise, repeated tuning can make results look stronger without showing that the system handles new work. For systems with variable outputs, repeat selected cases and inspect the pattern of failures.
Evaluate the whole workflow. If a draft is accurate but takes longer to check than writing an answer manually, the business goal has not been met. If the system fails safely but routes nearly every request to a person, the team needs to understand that workload before expanding access.
Which integration decisions matter before release
Map the path from input to recorded outcome. Identify where the application obtains a user’s identity, which data it can retrieve and how any change is written back to a business system. Existing application rules should continue to govern authorisation and valid state changes.
Choose the simplest implementation that meets the requirement. Anthropic’s guidance on building effective agents distinguishes predefined workflows from systems that let a model decide its next steps, and recommends adding complexity only when it improves results. For a bounded process, ordinary code around a model call may be sufficient.
Decide how retries work before the system can create records or send messages. A timeout does not necessarily mean an action failed. Retrying without checking can duplicate work. Use the business system’s supported mechanisms to identify prior actions and reconcile uncertain outcomes.
Keep operational visibility proportionate to the task. The team needs enough information to investigate failures, while logs should avoid unnecessary sensitive content. Record the relevant configuration and model version so that changes can be assessed against earlier behaviour.
Who needs to accept the production release
The process owner accepts the change to daily work and confirms that the evidence supports the intended use. Engineering accepts the implementation and maintenance arrangement. The relevant security and data owners review the access and information flows within their remit.
Agree these responsibilities before the release discussion. Late disagreement often arises because people are being asked to approve something different from what they thought the pilot covered. A short release record should state the permitted use, known limits, remaining issues and the person authorised to stop the workflow.
The FDE prepares the implementation and evidence, resolves issues across teams and keeps the decision visible. That coordination does not replace the client’s approval responsibilities.
What should happen after the first release
Reserve time to observe use. Look at the eligible work that actually reaches the system, the cases people correct and the reasons they return to the manual process. Speak with users who stopped using it as well as those who adopted it quickly.
Compare like-for-like work with the baseline. Changes in volume, case difficulty or staffing can affect the result. If you cannot isolate the AI contribution confidently, report that uncertainty and collect better evidence before attributing the entire improvement to the system.
At Rollout IT, we recommend assessing the next investment against this evidence. A useful first workflow may justify another integration or a larger user group. It may also reveal that a small configuration change produces more value than extending the AI component.
To scope an FDE engagement, bring the pilot, a sample of ordinary work and the proposed business outcome. These make the production gap concrete enough to discuss.
Frequently asked questions
Can every AI pilot reach production in 90 days
No. The schedule above is a planning example for a bounded workflow. Data access, procurement, integration complexity and release requirements can change the timeline. Agree milestones after checking the main dependencies.
What does production ready mean for an AI workflow
It means the workflow meets agreed acceptance criteria and has appropriate access controls, operational visibility and support. Users know its limits, failures have a defined handling path and an accountable owner can decide whether it should continue running.
Is a human review step enough to make a pilot safe
No. Review must be supported by understandable outputs, sufficient context and realistic reviewer capacity. Authorisation, data handling and system-side validation still need to work independently of the model’s response.
When should an organisation stop an AI pilot
Stop or rescope when the expected benefit does not justify the total cost, essential dependencies cannot be resolved, or the workflow cannot meet its acceptance criteria. Record what was learned so that the same assumptions do not generate another unsupported pilot.
Sources
McKinsey | The State of AI in 2025 | 2025 survey of 1993 respondents
Anthropic | Demystifying Evals for AI Agents | 9 January 2026