AI Agent Evaluation: Metrics, Validation, and End-to-End Testing

AI agent evaluation combines quality grading with verifiable business outcomes. For AB-100, this lesson covers Manage the testing of AI-powered business solutions within Deploy AI-powered business solutions (40–45% in the course outline before the announced November 16, 2026 update). You will select evaluation metrics, define model validation criteria, and test complete workflows across Dynamics 365 applications.

Learning objectives

After this lesson you can:

• Recommend a process and metrics to test agents.
• Create validation criteria for custom AI models.
• Validate that Copilot prompts follow effective best practices.
• Design end-to-end test scenarios for AI solutions that span several Dynamics 365 apps.
• Build a strategy for creating test cases by using Copilot.

1. A testing process for agents

Traditional software testing includes exact assertions, invariants, and integration checks. Agent responses can be valid in many forms, so add reviewed quality rubrics while retaining exact checks for permissions, identifiers, and committed business effects.

Test in a dedicated environment with representative but non-sensitive data, and release through the ALM process, so a result can be tied to one version of the solution.

Test layers

• Component tests: one topic, tool, connector, flow or prompt in isolation.
• Agent tests: the agent answers single questions and runs multi-turn conversations.
• Integration tests: the agent calls real or simulated systems (Dataverse, Dynamics 365, Microsoft Graph, custom APIs).
• End-to-end tests: a complete business process from the user's request to the final business outcome.
• User acceptance testing (UAT): business users validate the solution against their own expectations.
• Non-functional tests: performance and load, security, accessibility, responsible-AI checks.

2. Metrics to test agents

Choose metrics from four families and set a target for each.

Principles:

• Use more than one metric. A fast agent that is wrong, or accurate but too expensive, fails the business goal.
• Define thresholds and which metrics are blocking. A safety failure should block a release even if the average score is high.
• Report distributions and segments (by intent, language, user group), not only averages.
• Because outputs vary, run important cases several times or use many cases per scenario rather than relying on one run.

Test methods in Copilot Studio evaluations

Copilot Studio agent evaluation runs a test set of test cases against the agent and scores the results with test methods. For the documented standard-harness experience, a single-response test set holds up to 100 cases and a conversational set up to 20, with up to 12 messages per conversation case. Confirm current limits and method availability for the chosen test-set type. Available methods include general quality (the default, needing no expected answer), content safety, compare meaning, tool use, keyword match, text similarity, exact match and custom graders. Choose the method that matches the question being asked:

A Tool use pass establishes use of the expected capability, not correct parameters, authorization, or transaction completion. A keyword check cannot validate an arbitrary identifier pattern; use a programmatic format check or an appropriate custom method. General quality is an LLM judgment, not an oracle of factual correctness. When refusal is the intended outcome, use an explicit custom refusal rubric because the general-quality abstention criterion can penalize it.

Foundry evaluations offer built-in and custom evaluators for supported agents and models. Calibrate model-based graders against expert-reviewed examples and record their configuration with the results.

3. Validation criteria for custom AI models

When the solution uses a custom model (for example a custom prediction model in AI Builder or Azure Machine Learning, or a fine-tuned model), validation criteria must be defined before training results are known, otherwise the team tunes the criteria to the result.

Criteria to define:

1. Business success measure tied to value, such as reduced churn or fewer manual reviews.
2. Technical performance appropriate to the task: accuracy, precision, recall and F1 for classification; error measures for regression; quality scores for generative output.
3. Data quality and representativeness: training, validation and hold-out test data kept separate; no leakage; coverage of segments, languages and edge cases.
4. Fairness and bias: compare performance across groups, with defined tolerances.
5. Robustness: behavior on noisy, rare or adversarial inputs.
6. Explainability and transparency: can users and auditors understand why a result was produced?
7. Safety, privacy and compliance: sensitive data handling, regional and regulatory rules, human oversight where required.
8. Operational fitness: latency, throughput, cost, and monitoring for drift after deployment.

Choose the metric for the business risk. If missing a fraud case is costly, prioritize recall; if false alerts overload staff, prioritize precision. Compare the custom model with a baseline (a prebuilt model or the current manual process) and require it to be better by an agreed margin before adopting it. Record the criteria, results and approvals as governance evidence, and re-validate when data, model or policy changes.

4. Validating Copilot prompt best practices

A prompt in a Copilot, an agent instruction or a prompt tool is part of the solution and must be tested like code. Validate that prompts follow effective practices:

• Clear goal and role: what the task is and who the assistant acts as.
• Context: relevant business context and grounding data, not assumptions.
• Specific instructions and constraints: scope, tone, what to do when information is missing, when to refuse or escalate.
• Output format: structure, length, language, fields to return.
• Examples when the format or style is hard to describe.
• One task at a time, with complex work broken into steps.
• Safe handling: no secrets in prompts, protection against injected instructions in retrieved content, and use of the platform’s content filters. Prompt wording and filters do not replace authorization and validation in the services the agent calls.

Validation approach: write the prompt, run it against a representative set of inputs, compare outputs with expected behavior, change one element at a time, and keep versions so a regression can be traced. Include negative cases (missing data, ambiguous requests, attempts to override instructions). Store prompts under version control or in a solution so they move through environments with the rest of the ALM process.

5. End-to-end scenarios across Dynamics 365 apps

Agents rarely live in one app. A quote-to-cash process may start in Dynamics 365 Sales, check stock in Supply Chain Management, create a case in Customer Service and post an invoice in Finance. An end-to-end scenario validates the entire chain, including data handoffs and agent actions.

Design guidance:

1. Start from the business process, not from individual apps. Identify actors, systems, triggers and the expected business outcome.
2. Define the data: customers, products, price lists and permissions that make the scenario realistic and repeatable. Use test data, never production personal data.
3. Cover paths: the normal flow, alternate flows (partial delivery, discount approval), failure flows (system unavailable, invalid data) and human-in-the-loop approvals.
4. Test security: each persona uses its own security roles; verify denied as well as permitted access under the actual connection identity. A signed-in conversation does not prove that every tool runs as that user.
5. Verify outcomes in each system: records created or updated, statuses, amounts, notifications and audit trail, not only the agent's final message.
6. Include integration points: connectors, Dataverse, Power Automate flows, MCP or API tools, and the behavior when one of them is slow or fails.
7. Measure non-functional behavior: end-to-end latency, throughput at expected volume, and cost.
8. Automate regressions where possible so each release reruns the main scenarios, and keep UAT for judgment-heavy scenarios.

The environment matters: use a test environment that mirrors production configuration (same solutions, security roles and integrations) and refresh data between runs so results are reproducible.

6. A strategy for creating test cases by using Copilot

Copilot can speed up test creation, but it does not replace test design. A sound strategy:

1. Provide the sources: requirements, user stories, process descriptions, the agent's instructions and knowledge. Test cases generated from vague input are generic.
2. Ask for structure: scenario, preconditions, input, expected result, and the metric or grader to use.
3. Request coverage: happy path, boundaries, invalid input, ambiguous requests, multiple languages, security and out-of-scope attempts.
4. Generate in batches by topic or requirement, then deduplicate and prune.
5. Human review: subject-matter experts confirm that expected results are correct. A model can invent an expected answer; an unreviewed test set can encode wrong behavior.
6. Add representative cases: use approved, de-identified questions from telemetry and feedback to reflect real usage, while retaining independently designed cases for unknown information and denied requests.
7. Keep traceability: link each test case to a requirement so coverage gaps are visible.
8. Maintain the set: add a case for every defect found, retire obsolete ones, and version the set with the solution.

Copilot Studio can generate test cases from the agent's description and instructions, from knowledge sources or topics, or you can import them from a file. Treat generated sets as a draft: use them to start, then enrich with expert and production-derived cases. Do not let the same model both write the cases and be the only judge, and keep a human-reviewed sample of graded results to check that automated graders agree with people.

Evaluation identity and sensitive test data

Copilot Studio evaluations use the currently signed-in user’s connections. To exercise a different user’s rights, use the supported sign-in/profile procedure and verify which identity the tool actually uses. A successful maker evaluation is not evidence that a restricted technician can perform the same action.

Generated cases can contain sensitive material accessible to the connected account, and makers with access to the agent can see its linked test sets. Inspect and minimize generated content before sharing it. Generation from existing knowledge is useful for covered topics but does not discover every missing-knowledge scenario; design those cases explicitly.

7. Worked example

A manufacturer deploys an agent that helps service technicians. It reads work orders in Dynamics 365 Field Service, searches a repair knowledge base, and can order parts through Supply Chain Management.

Test design:

• Metrics and thresholds: groundedness and correctness at or above the agreed threshold on the knowledge set; tool-selection accuracy for part ordering; zero tolerance for ordering the wrong part number in the test set; p95 latency under the target; safety checks must all pass.
• Test set: generated with Copilot from the repair manuals and work-order schema, reviewed by two senior technicians, enriched with anonymized real questions. Multi-turn cases cover clarifying questions.
• Prompt validation: the instruction states role, scope, format of the repair steps and a rule to escalate when safety procedures are involved; negative tests try to override the safety rule.
• Custom model: a failure-prediction model is validated against a held-out dataset, with recall prioritized, compared with the current manual rule, and checked across machine families.
• End-to-end scenario: from work-order creation to part order to stock reservation to invoice, with a technician persona lacking purchasing rights, confirming that approval is routed to a manager.
• Gate and regression: the set runs in the test environment on every change; failures block promotion through ALM; UAT with technicians precedes go-live.

Step-by-step acceptance example

Consider an illustrative parts-order request: technician T17 needs two units of part P-204 for work order W-806. The technician may read the work order but cannot approve a purchase. The expected result is an approval request, with no purchase committed before the manager approves.

1. Create the work order, stock state, user roles, and test connections in the non-production environment. Record the agent, model, prompt, and knowledge versions so the run is reproducible.
2. Ask the agent to order the part. Check the intended capability selection with a Tool use evaluation, then inspect the recorded tool arguments: work-order ID, part ID, quantity, and requesting identity. A correct tool with quantity 20 instead of 2 is still a failure.
3. Verify the approval record and absence of an unauthorized purchase in the backend. Grade the response for clarity: it should say approval is pending, rather than claim that the order is complete.
4. Approve through the manager’s authorized test account and verify the resulting order and stock reservation. Repeat the delivery of the action request and confirm that the business service prevents a duplicate order.
5. Repeat with no stock, a rejected approval, a tool timeout, an inaccessible work order, and malicious instructions embedded in a repair document. Check the expected error or escalation path, not just whether an answer was produced.
6. Add multi-turn variants in which the technician changes the quantity or clarifies the part number. Grade the final committed outcome against the latest authorized request and preserve the trace of what was approved.

The purpose is to validate a complete business contract. A fluent explanation can coexist with a failed write, an excessive permission, or a duplicate transaction. Those failures should block release even if an LLM grader likes the response.

Mini case: an excellent grader score with a faulty test oracle

An AI-generated expected answer says that every warranty lasts 24 months. The approved policy grants 12 months for one product family. A candidate matches the generated reference and receives a high Compare meaning score. Has it passed the business requirement?

Solution: the reference is wrong. Correct the expected result with the policy owner, add the product-family distinction, and rerun baseline and candidate against the corrected set. Keep a record of the test-set change so scores from different oracles are not presented as directly comparable. Automated agreement with a false reference is not business accuracy.

Key terms

• Test oracle: the reviewed rule or expected result used to decide whether behavior is correct.
• Groundedness: support in the supplied evidence, which still needs freshness and accuracy checks.
• Regression set: versioned cases used to detect changes in previously supported behavior.
• Hold-out data: examples excluded from model training and tuning, reserved for final assessment.
• Release gate: a defined decision rule that prevents promotion when required criteria fail.

Use agent telemetry and feedback to add representative failures to the set. Keep the gate within ALM for agents, grounding data, and models, and use the agent security and governance lesson when defining forbidden operations.

Exam tips

• The process starts with business-driven criteria and thresholds, then test sets, graders, a release gate and regression after each change.
• Pick the grader that matches the question: Tool use for expected capability invocation, separate assertions for arguments and effects, Compare meaning for semantic agreement, and Content safety for configured harmful-output categories.
• Define custom-model validation criteria up front and include fairness, robustness, data separation and a baseline comparison, not only accuracy.
• Prompt best practices: clear role and goal, context, constraints, format, examples, and versioned testing with negative cases.
• End-to-end tests follow the business process across apps and verify outcomes in each system, with the right security roles and test data.
• Copilot-generated test cases need requirements as input, human review of expected results and traceability to requirements.

Sources

• Microsoft Learn, study guide for exam AB-100: https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/ab-100 (opens in a new tab)
• Microsoft Learn, Manage Testing AI-Powered Business Solutions (training module): https://learn.microsoft.com/en-us/training/modules/manage-testing-ai-powered-business-solutions/ (opens in a new tab)
• Microsoft Learn, Copilot Studio agent evaluation, choose evaluation methods: https://learn.microsoft.com/en-us/microsoft-copilot-studio/analytics-agent-evaluation-overview (opens in a new tab)
• Microsoft Learn, create a single response test set: https://learn.microsoft.com/en-us/microsoft-copilot-studio/analytics-agent-evaluation-create (opens in a new tab)
• Microsoft Learn, create a conversational test set: https://learn.microsoft.com/en-us/microsoft-copilot-studio/analytics-agent-evaluation-multi-turn (opens in a new tab)
• Microsoft Learn, common evaluation approaches (Copilot Studio): https://learn.microsoft.com/en-us/microsoft-copilot-studio/guidance/architecture/common-evaluation-approaches (opens in a new tab)
• Microsoft Learn, plan a testing strategy for your Dynamics 365 solution: https://learn.microsoft.com/en-us/training/modules/test-strategy/ (opens in a new tab)