Seven questions to ask any “agentic AI” planning vendor

In the last two years, nearly every supply chain planning vendor has added “agents” to its product page. Some are genuinely new architectures. Others are a chatbot bolted onto a decade-old APS. From a demo, they can look surprisingly similar.

Here are seven questions that separate agents that can safely change your plan from agents that can only talk about it. They’re the questions we’d want a customer to ask us — and the answers you should expect from anyone.

1. Where do the numbers come from?

This is the single most important question. When the agent proposes moving 3,000 cases to Line 2 on Thursday, what produced that number?

If the honest answer is “the language model,” be careful. Language models generate plausible text; they don’t check capacity constraints. A good answer sounds like: “The agent formulates the problem; an optimization engine computes the quantities and dates under all constraints; the language model only explains the result.”

2. How does the agent know what’s feasible in my plant?

Feasibility lives in details: changeover times that depend on the previous product, molds that can only be on one line at a time, operators certified for specific equipment, ingredients that expire. Ask how those rules are represented. If the answer is “we retrieve your documents” or “the model learns from history,” the agent is guessing.

Look for an explicit model of your operation — an ontology or equivalent — where rules and constraints are first-class, inspectable and editable without code.

3. Can it prove the plan is good, not just feasible?

A feasible plan isn’t the same as a good one. Ask whether the system can tell you how far its plan is from the best possible — a measured optimality gap — and which constraints are binding. If it can’t, you’ll never know whether you’re leaving capacity or margin on the table.

4. What happens when something changes at 2 a.m.?

Agents earn their keep after the plan is published. Ask the vendor to walk through a real disruption: a machine breakdown or a short delivery. Does the agent detect it on its own, or does someone have to ask? Does it re-solve only the affected scope, or rerun everything and leave a planner to diff the results? Does it explain what changed and why?

5. What exactly can the agent do without asking?

Autonomy should be a dial you control, not a switch. Look for configurable thresholds — by cost impact, orders affected or customer tier — inside which agents act and outside which they ask. You should be able to start in recommend-only mode and widen authority over time. And every automated action should be reversible.

6. Can I audit a decision after the fact?

Six weeks from now, someone will ask why a key customer’s order was pushed. You need to be able to replay the event the agent sensed, the reasoning path it followed, the optimization run it triggered, the options it rejected and who approved the result. If the audit trail is “the chat history,” that isn’t enough for an enterprise.

7. How long until it’s running on my data?

Gartner puts the average supply chain planning implementation at around 13 months. Agentic architectures should be faster, not slower — because an ontology with industry templates replaces most custom code. Ask for a concrete plan: which use case first, which data, what’s validated by when. A credible answer is weeks to first value on a scoped use case, not a year-long program before anything works.

A quick scorecard

We built Convect so we could answer all seven well: planning agents that act on a Contour ontology and call an optimization engine for every number, with guardrails and an audit trail built in. If you’d like to put us through these questions on your own scenario, book a demo.