Start with one decision
A founder can now produce an impressive AI demonstration in an afternoon. A useful operating workflow takes a different kind of preparation: somebody has to decide which job the system performs, which evidence it receives and what happens when it cannot answer. Those decisions are often more valuable than another round of prompt tuning.
Consider a proposed assistant that drafts replies to customer enquiries. Its job might be to summarize the enquiry and suggest an answer from approved product documentation. That is a bounded assignment. Allowing it to promise delivery dates, negotiate prices and change customer records creates several additional jobs, each with its own permissions and failure costs. Write those responsibilities separately before making them automatic.
Use the framework as a set of questions
NIST’s voluntary AI Risk Management Framework organizes risk work into Govern, Map, Measure and Manage. Its generative-AI profile adds context for the risks associated with these systems. These are frameworks for evaluating a deployment, rather than certifications that a particular product works.
A small team can translate the framework into a one-page pilot brief. Identify the owner, describe the task and affected people, choose measures that expose failure, and agree on the response when those measures deteriorate. This is our proposed operating interpretation. It should be adapted to the consequences of the actual task, rather than treated as a universal compliance checklist.
Measure the work that reaches a customer
Build a small evaluation set from permitted, representative material. Include straightforward cases, incomplete requests and questions whose answer does not exist in the source documents. Keep some cases separate from prompt development so that repeated tuning does not simply teach the system the test.
Measure at least three things independently: whether the answer is supported, whether it is useful, and how much checking a person must still perform. A fluent answer can score well on usefulness while being unsuitable to send. Count an unsupported promise as a failure even when the rest of the response reads beautifully. Record the original request, source version and final human decision so the team can investigate disagreements.
Design the uncertain answer
Define what the assistant should do when information is missing. It might ask one focused question, prepare a partial draft or send the case to a colleague. The escalation needs an owner and a response window. A button labelled “human review” does little if there is nobody responsible for that queue.
Keep access proportional to the task. A draft-writing assistant may need approved reference material without needing permission to issue refunds. If the pilot later gains another action, review that change as a new capability. Make it possible to pause the automated action while leaving the underlying customer service process available.
Run a review that can change the decision
At the end of the pilot, compare the total effort with the current workflow, including review, corrections and support. Ask whether the system improved the job for the people doing it. Record who benefits, who absorbs the extra checking and which cases should remain with a person.
The decision can be to expand, narrow or stop. All three are useful results when they follow evidence. A clear job description makes those decisions possible: it gives the team something more concrete to evaluate than whether the demonstration felt intelligent.
Sources & further reading
NIST AI Risk Management Framework ↗Voluntary AI risk framework and generative-AI profile.NIST AI RMF core ↗Govern, Map, Measure and Manage functions.An original, AI-assisted GSC launch analysis, researched against the primary sources below on 7 September 2026. Practical frameworks are editorial proposals, not reported company results. This is not an interview.
Request a correction →
