Skip to content
FR

Evaluate Costs and Benefits

Step 7 of 22

AB-100: AI Agent ROI, TCO, and Model Routing

AI agent ROI and total cost of ownership turn a promising use case into a measurable business decision. This AB-100 lesson covers the skills measured under Plan AI-powered business solutions (25–30%) that relate to costs and benefits: building a business case for an agent, estimating total cost of ownership (TCO), choosing metrics to track return on investment (ROI), and controlling consumption cost with model selection and model routing.

Learning objectives

After this lesson you can:

  • Define ROI criteria and success metrics for an AI agent before it is built.
  • List the cost components that make up the TCO of an agent solution.
  • Identify the benefit categories (hard savings, productivity, revenue, risk reduction) and measure them against a baseline.
  • Explain how consumption-based billing (for example Copilot Credits in Copilot Studio) affects cost estimates.
  • Describe how Microsoft Foundry model router selects a model and when to use Balanced, Cost or Quality mode.
  • Apply build versus buy reasoning to costs, not only to features.

1. Why a business case comes first

An agent is a product with running costs, not a one-time project. Before building, the architect needs a business case that states the outcome, the baseline (how the process performs today), the target (what must improve), the total cost and the time to value. Without an agreed baseline, attributing benefits to the agent is difficult. Keep assumptions explicit and use a comparable pilot or control group where practical.

A usable business case answers five questions:

  1. Which process or decision is improved, and for whom?
  2. What does it cost today (time, errors, delays, licenses)?
  3. What will the agent cost to build, run and govern?
  4. Which measurable benefit is expected, and by when?
  5. What are the risks and the assumptions behind the numbers?

2. ROI criteria and metrics

ROI compares the benefit gained with the cost incurred over a period. In practice the architect defines criteria and metrics up front so results can be monitored after deployment.

Benefit category Example metrics Notes
Hard savings Reduced licensing, outsourcing or rework cost; fewer tickets escalated to paid support Easiest to prove; appears in budgets
Productivity Time saved per task, handling time, cases per agent-hour Only real if the saved time is redirected to other work
Revenue and experience Conversion, retention, customer satisfaction, resolution rate Needs a control group or before/after comparison
Risk and quality Fewer errors, better compliance, faster audits Often expressed as avoided cost

Good practice:

  • Pick a small number of metrics tied to the business goal, not to technical activity (number of prompts is not a benefit).
  • Measure cost per task or cost per resolved case, which links benefit and consumption.
  • Capture the baseline before the pilot and compare the same population afterward.
  • Separate adoption metrics (active users, usage frequency) from outcome metrics. Low adoption is a common reason ROI is not reached.
  • Treat time savings carefully: 10 minutes saved for 1,000 people is only value if it changes output or cost.

Exam tip: if a scenario says the sponsor wants proof the agent is worthwhile, the answer usually involves defining measurable success criteria and a baseline, then tracking them with monitoring and analytics, rather than choosing a bigger model.

3. Total cost of ownership

TCO covers everything required to build and run the solution over its life, not only the model bill.

Cost area Examples
Licensing and capacity Microsoft 365 Copilot or Dynamics 365 licenses, Copilot Studio capacity, Power Platform capacity
Consumption Copilot Credits, model tokens in Foundry, tool and flow executions, search/indexing, storage
Build Design, development, integration, data preparation, security review
Data Cleaning and structuring grounding data, connectors, ongoing sync
Operations Monitoring, evaluation, incident handling, updates when models or prompts change
Governance and compliance Policy, audits, responsible AI reviews, access management
People Training, change management, Center of Excellence, support

Points the exam likes to test:

  • Consumption costs scale with usage. A pilot with ten users can look cheap while the same design at ten thousand users is not. Estimate at production volume and add a growth margin.
  • Data preparation is often the largest hidden cost. Poor grounding data lowers quality and drives rework.
  • Cheapest per call is not cheapest per outcome. A weak model that needs retries or human correction can cost more per resolved task.
  • Include non-production environments and evaluation runs in the estimate.

4. Consumption billing in Copilot Studio

Copilot Studio bills agent activity through a pay-as-you-go meter or prepaid Copilot Credit packs. Different features consume credits at different rates. Microsoft's published billing rates distinguish, for example, classic answers, generative answers, agent actions, tenant graph grounding, agent flow actions, and text and generative AI tools billed per 1,000 tokens at basic, standard and premium levels. Verify the billing unit for each feature rather than assuming every feature is measured per message or every AI tool uses the same meter.

These published rates apply to the standard harness described on the billing page. Some authenticated employee-facing usage is included for Microsoft 365 Copilot licensed users under specific conditions; triggers, usage limits, and exclusions still matter. Identify the applicable scenario before applying a rate.

What matters for design:

  • For metered scenarios, generative answers, actions, and grounding can carry different credit rates from classic answers; one interaction can incur several feature charges.
  • Using a premium-tier model for a simple classification step costs more than needed; match the tier to the task.
  • Rates and meters change over time, so the architect checks the current billing page and uses the billing rates and management documentation and the Power Platform admin center to monitor consumption and configure supported limits. Budget alerts alone do not stop execution.
  • Billing rates exclude models you bring yourself (for example Foundry models); those are billed separately on the Azure side. Costs of a hybrid design therefore appear in more than one place.

Cost controls available to a solution architect: budgets and alerts, capacity limits per environment, restricting who can create or publish agents, retiring unused agents, and reviewing consumption reports regularly.

5. Model selection and model router

In Microsoft Foundry, cost and quality depend heavily on which model answers which request. Not every prompt needs the largest reasoning model.

Model router is a purpose-built machine-learning routing model exposed through a single deployment. For each prompt it chooses, in real time, a suitable underlying model from a set of models, using the configured mode and eligible model pool to trade off predicted quality, cost, and latency. Actual results require workload evaluation. Simple requests can go to smaller, cheaper models; complex ones can go to larger or reasoning models.

Routing modes:

Mode Behavior Choose it when
Balanced (default) Balances quality and cost across eligible models A baseline configuration for mixed workloads
Cost Favors cheaper eligible models, accepting quality trade-offs Cost-sensitive workloads after quality testing; not a spending cap
Quality Prioritizes predicted response quality over cost within the eligible pool Tasks whose evaluation justifies the extra spend; no compliance guarantee

Constrain the pool to approved models where required and verify residency and feature support. If the workload must use the same specific model for every request, use a direct deployment rather than relying on routing. A high-quality prediction does not grant authorization or replace human review. For the implementation trade-offs, revisit AI-103 model and Foundry service selection.

Other ways to lower model cost without losing the outcome:

  • Right-size per step in a multi-agent flow: a small model for triage and extraction, a stronger one for final reasoning.
  • Shorten prompts and context: retrieve only relevant passages instead of whole documents.
  • Cache repeated answers where freshness allows, scoped to the caller's access rights and relevant data version. Do not reuse one user's confidential answer for another user.
  • Limit output length and unnecessary tool calls.
  • Evaluate before switching: compare candidate models on a test set that reflects the real task, then decide using quality and cost together.
  • Plan for model lifecycle: models are retired and replaced, so budget for re-evaluation.

Model router is a trade-off tool, not a guarantee: routing adds a dependency on the router's choices, so keep monitoring quality and cost per task after deployment.

6. Build versus buy, on cost

Extend the build, buy, and extend strategy with cost reasoning:

Option Typical cost profile Watch for
Prebuilt agent or feature Per-user or capacity license; low build effort Paying for seats that are not used; limited customization
Extend a prebuilt solution License plus small build and data work Extension limits and maintenance when the product updates
Low-code (Copilot Studio) Build effort moderate; consumption grows with usage Credit consumption at scale; governance of makers
Pro-code (Foundry) Highest build and operations effort; fine control over models and hosting Engineering and monitoring staff, separate Azure billing

A good default is to start with the least custom option that meets the requirements, then move up only where requirements or unit economics justify it. Revisit the choice when volume changes: a feature that is cheapest as a prebuilt license at low volume can become expensive at scale, and the reverse.

7. Worked example

A support organization handles 40,000 cases per month. Average handling time is 12 minutes and 25% of cases are repeat questions already answered in the knowledge base.

  • Baseline: 12 minutes per case, customer satisfaction 78%, escalation rate 18%.
  • Target: automate resolution of simple repeat questions, reduce average handling time for the rest through agent assistance.
  • Metrics: cost per resolved case, containment rate, handling time, satisfaction, escalation rate.
  • Costs: Customer Service licenses, Copilot Studio consumption for the front-end agent, knowledge-base cleanup, monitoring and evaluation effort, change management.
  • Model choice: a Foundry deployment with model router in Balanced mode for the knowledge answers; a separately evaluated path for sensitive refund recommendations, constrained to approved models and followed by the required policy and human approval checks. Quality mode alone does not make a refund decision safe or compliant.
  • Decision: run a pilot in one region, compare against the baseline, and extend only if cost per resolved case and satisfaction meet the target.

Worked calculation: first-year ROI and simple payback

The following numbers are illustrative assumptions, not product prices or promised savings. Assume 60% of the 10,000 repeat-question cases are safely resolved without a human case, and the remaining cases retain their original handling time.

  1. Time released: 6,000 cases × 12 minutes ÷ 60 = 1,200 hours per month. Do not also count those same minutes as an improvement on the remaining cases.
  2. Realizable benefit: at an assumed loaded value of $30 per hour, the maximum capacity value is $36,000. If the business can put only 50% of that capacity to productive use, model $18,000 per month. This is capacity value; call it cash savings only if expenditure actually falls.
  3. Costs: assume a $24,000 initial build and $9,000 monthly incremental operating cost, including licences, model and tool usage, data refresh, monitoring, and human review. Use the grounding-data requirements to bound the cleanup and refresh work.
  4. First-year ROI: with twelve months at that steady level, benefits are $216,000 and costs are $24,000 + 12 × $9,000 = $132,000. ROI = (benefits − costs) ÷ costs = 63.6%. Report the calculation period and all assumptions.
  5. Simple payback: $24,000 ÷ ($18,000 − $9,000) = about 2.7 months after steady operation begins. This simplified measure excludes ramp-up and discounting. If the realizable monthly benefit is at or below $9,000, there is no positive monthly surplus to repay the build under these assumptions.

The pilot must test containment quality, actual usage charges, and whether released time becomes useful capacity. A smaller model bill alone does not establish a positive ROI.

Key takeaways

  • Define a baseline, target and measurable metrics before building; track cost per task, not just usage.
  • TCO includes licensing, consumption, build, data, operations, governance and people. Estimate at production volume.
  • Copilot Studio consumption is metered (Copilot Credits) and varies by feature; check current rates and monitor usage.
  • Model router chooses among eligible models per prompt; evaluate Balanced, Cost, or Quality against measured cost and quality requirements.
  • Match model tier to the task, shorten context and evaluate on a representative test set.
  • Re-evaluate build versus buy as volume and requirements change.

Sources