Microsoft Foundry Setup for AI-103: Deployments, Agents and CI/CD

Introduction

Setting up Microsoft Foundry for AI-103 means connecting infrastructure, model deployments, versioned agents and a repeatable delivery pipeline. This lesson continues the domain Plan and manage an Azure AI solution (25–30% of AI-103). After choosing models and Foundry services for AI-103, you must provision the Azure infrastructure, deploy models and agents, and automate the whole process. The exam tests whether you can pick the right resource layout, the right deployment option and a safe CI/CD approach.

After this lesson you will be able to design the Azure resources for an AI app, deploy a model and an agent, connect an application to a Foundry project without secrets, and describe a CI/CD pipeline that promotes Foundry assets from development to production.

Learning objectives

• Describe the relationship between a Foundry resource, projects, connections and model deployments.
• Choose regions, deployment types and capacity settings for a workload.
• Provision infrastructure with the portal, Azure CLI, Bicep and the Foundry SDK.
• Deploy models and agents and call them from Python with keyless authentication.
• Design a CI/CD workflow that builds, evaluates, deploys and promotes AI assets across environments.

1. Azure infrastructure for AI apps and agents

1.1 Resource layout

A typical production landing zone for an AI app contains:

Plan one Foundry project per application boundary when its agent assets and access policies differ; a project alone does not create independent model capacity, billing or network isolation. Plan several projects under one resource when teams share model deployments and network design but need separate agents and files. Put dev, test and prod in separate resource groups, or separate subscriptions for strong isolation.

1.2 Region and capacity planning

Model availability, deployment types and quota differ by region. For pay-per-token deployments, TPM/RPM quota is scoped by subscription, region, model and deployment type. Provisioned deployments use PTUs, with a shared per-subscription/region/deployment-type quota across models; quota approval does not guarantee that the requested model capacity is available. Decide the region from: data residency, model availability, latency to users and quota headroom. A frequent architecture is a primary region plus a secondary deployment for failover, fronted by retry logic or an API gateway.

Worked example 1: sizing a landing zone

Scenario. Northwind Health is launching a patient FAQ agent for the EU. Requirements: processing inside the EU, no public internet exposure of the model endpoint, separate dev and prod, the agent grounded on documents in an index, and no keys stored in app settings.

Solution, step by step.

1. Create two resource groups (rg-faq-dev, rg-faq-prod) in EU regions that offer the needed models.
2. In each, create a Foundry resource and project, an Azure AI Search service and a storage account.
3. Choose Data Zone (EU) deployments for chat and embedding models so data processing stays within the EU.
4. Add private endpoints for the Foundry resource, Search and Storage; disable public network access in prod. Review Azure private endpoints and network access controls to separate private connectivity from firewall exceptions.
5. Run the app on Azure Container Apps in a virtual network with a system-assigned managed identity; grant it the required roles (for example, Foundry Agent Consumer for an app that only invokes a published agent endpoint; Foundry User for authoring/data operations where needed; Search Index Data Reader at the supported index/service scope for retrieval).
6. Send sanitized traces to an appropriately located Application Insights resource. Verify residency for Search, Storage, telemetry, agent state, safety processing and external tools separately; a model Data Zone selection is not a blanket residency guarantee for the complete application.

Why other choices are worse. Global Standard may process data outside the EU. Using an API key in app settings can leak and cannot be tied to an identity. A single shared resource group makes it easy to let test changes affect production.

2. Deployment options

2.1 Deploying a model

You deploy a model from the catalog by choosing: model and version, deployment name, deployment type, capacity (TPM allocation units for eligible Standard deployments, PTUs for provisioned deployments), content filter policy and, for some models, an automatic version-update policy. In the serverless-deployment examples here, the application calls the deployment name. Foundry Instant access (preview) is a separate model-by-name mode; do not generalize that mode to deployed Azure OpenAI endpoints.

Using the Azure CLI, a deployment looks like this:

az cognitiveservices account deployment create \
  --name my-foundry-resource --resource-group rg-faq-prod \
  --deployment-name chat-main \
  --model-name gpt-4.1 --model-version "2025-04-14" --model-format OpenAI \
  --sku-name DataZoneStandard --sku-capacity 100

The command assumes an existing compatible Foundry resource, sufficient quota, deployment permissions and model availability in the resource region. The gpt-4.1 version shown is a dated example: Microsoft lists it as deprecated with retirement on April 14, 2027. Select a supported version from the live catalog and plan migration; do not copy this model choice as an evergreen recommendation. The idea to retain is that capacity, SKU and filter policy are properties of the deployment.

2.2 Choosing capacity and throttling behaviour

When a deployment returns HTTP 429 (rate limit), the client should honour the Retry-After header and use exponential backoff with jitter. First inspect the error body and rate-limit headers: 429 can indicate allocated TPM/RPM exhaustion, temporary backend pressure or a reduced effective rate limit. Honor retry-after-ms in milliseconds when returned. Add a bounded retry budget and avoid repeating non-idempotent agent tool actions. Extra deployments in the same quota pool do not create quota; rebalance allocations or request quota for genuine exhaustion, and evaluate PTU for sustained capacity needs. Any fallback must preserve residency, safety and tool permissions.

2.3 Deploying agents

A prompt agent combines a deployed model, instructions and tool definitions in a versioned project asset. Creating a version is distinct from routing/publishing an endpoint and granting a runtime permission to invoke it. Hosted agents also deploy code/runtime infrastructure; not every agent is just a configuration object. Keep version identifiers, tool schemas and instructions under change control.

This development example uses the Foundry projects 2.x API (azure-ai-projects>=2.3.0 and azure-identity), a compatible prompt-agent model deployed as chat-main, and an authorized developer identity. Lock tested package versions in your application. It creates a new agent version on each execution; normal application requests should invoke an approved version/endpoint rather than recreate the definition.

import os
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition

with DefaultAzureCredential() as credential:
    with AIProjectClient(
        endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
        credential=credential,
        allow_preview=True,
    ) as project:
        agent = project.agents.create_version(
            agent_name="faq-assistant",
            definition=PromptAgentDefinition(
                model="chat-main",
                instructions="Explain appointment scheduling. Do not invent clinic policies.",
            ),
        )
        print(agent.name, agent.version)
        with project.get_openai_client(agent_name=agent.name) as client:
            reply = client.responses.create(
                input="What information should I have ready before rescheduling?"
            )
            print(reply.output_text)

Copy the project endpoint in the format https://<resource>.services.ai.azure.com/api/projects/<project> into the environment variable. The agent-bound endpoint path requires preview opt-in in the current SDK reference; check the API/SDK combination and endpoint routing before promotion. This minimal agent has no grounding tool and is not the complete clinical FAQ design above.

DefaultAzureCredential is an ordered credential chain, not a guarantee that Azure always selects managed identity. Verify which principal authenticated; in production use an explicitly configured managed-identity credential or restrict the chain and remove unintended environment credentials. A key cannot authenticate current Foundry Agent Service operations.

Worked example 2: fixing an authentication failure

Scenario. The app works on a developer laptop but returns an authorization 403 in Azure Container Apps after network reachability and token audience have been verified.

Reasoning.

1. Locally, DefaultAzureCredential used the developer's Azure CLI login, who has roles on the project.
2. Inspect the principal actually used in Azure. In this scenario it is the container app managed identity, which has no invocation assignment.
3. Fix: enable the managed identity and assign the least-privileged role on the Foundry project or resource scope (Foundry Agent Consumer for endpoint-only invocation, or the appropriate Foundry User/data role for the operation). Role propagation can take minutes.

Wrong fixes. Pasting an API key does not authenticate the current Agent Service and does not fix missing Entra authorization. Granting Owner is excessive.

3. CI/CD integration for Foundry projects

3.1 What to automate

• Infrastructure: Bicep or Terraform templates for resource groups, Foundry resource and projects, deployments, Search, networking and role assignments. Pipelines run what-if/plan for review before applying.
• Application: build container, run unit tests, deploy to Container Apps or App Service.
• AI assets: agent definitions, prompts, tool schemas, evaluation datasets and index definitions, all stored in Git.
• Evaluation gates: run automated evaluations against a fixed test set and fail the pipeline if quality or safety metrics drop below thresholds.
• Promotion: deploy the same artifacts to test and then prod with environment-specific parameters (endpoint, capacity, filter policy), using approvals for production.

3.2 A reference workflow

# Application repository example; paths and evaluation CLI are project-owned.
name: deploy-ai-app
on: { push: { branches: [main] } }
permissions: { id-token: write, contents: read }
jobs:
  test-and-eval:
    environment: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: azure/login@v3
        with:
          client-id: ${{ vars.AZURE_CLIENT_ID }}
          tenant-id: ${{ vars.AZURE_TENANT_ID }}
          subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }}
      - run: python3 -m pip install -r requirements.txt
      - run: python3 -m pytest
      - run: python3 eval/run_eval.py --dataset eval/golden.jsonl --min-groundedness 4.0
  deploy-test:
    needs: test-and-eval
    environment: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: azure/login@v3
        with:
          client-id: ${{ vars.AZURE_CLIENT_ID }}
          tenant-id: ${{ vars.AZURE_TENANT_ID }}
          subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }}
      - run: az deployment group create -g rg-faq-test -f infra/main.bicep -p env=test
  deploy-prod:
    needs: deploy-test
    environment: production
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: azure/login@v3
        with:
          client-id: ${{ vars.AZURE_CLIENT_ID }}
          tenant-id: ${{ vars.AZURE_TENANT_ID }}
          subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }}
      - run: az deployment group create -g rg-faq-prod -f infra/main.bicep -p env=prod

Each job starts on a fresh runner: checkout and Azure login are repeated deliberately. Precreate the GitHub environments, environment-specific variables and Azure federated credentials for the exact repository/environment subjects; configure production protection rules separately. OIDC permission alone grants no Azure RBAC. The runner must reach private endpoints (use appropriately networked runners for private-only environments). The repository must supply requirements.txt, tests, the evaluation script/dataset and infra/main.bicep; the score 4.0 is an application gate, not an official pass mark. This excerpt deploys infrastructure only: add versioned application/agent promotion and preserve evaluation evidence before releasing the same approved artifacts.

Key points: the pipeline authenticates through workload identity federation (OIDC), so no client secret is stored; evaluation runs before promotion; production deployment needs an approval.

3.3 Handling model version changes

Models are retired and updated. Decide whether a deployment follows automatic upgrades or is pinned to a version. In production, pin versions, subscribe to retirement notices, test a new version in a parallel deployment with your evaluation set, then switch by changing the deployment name or configuration. Pinning controls normal upgrades while that version is supported; it cannot prevent retirement, mandatory migration or all service-side changes. Plan upgrades before the published retirement date and retain a tested, supported fallback.

Worked example 3: promoting a prompt change safely

Scenario. A developer edits the agent's instructions to be friendlier. Two days later users report wrong answers about refund periods.

Better process.

1. Keep instructions in Git; open a pull request.
2. The pipeline runs the evaluation set (which includes refund-period questions) using relevance and groundedness evaluators.
3. The score on refund questions drops, so the gate fails, and the change is blocked.
4. After the fix, the pipeline deploys to test, then to prod after approval.
5. Monitoring in production compares live metrics to the baseline to detect regressions the test set missed. Continue with Foundry quotas, monitoring and production security to choose the operational signals.

3.4 Infrastructure as code in practice

A minimal Bicep fragment for a model deployment shows that capacity and SKU live in the template, so reviewers see them in a pull request:

param foundryName string
param modelName string
param modelVersion string
param capacity int = 100
param deploymentSku string = 'DataZoneStandard'
param filterPolicyName string = 'Microsoft.Default'

resource foundry 'Microsoft.CognitiveServices/accounts@2024-10-01' existing = {
  name: foundryName
}
resource chat 'Microsoft.CognitiveServices/accounts/deployments@2024-10-01' = {
  parent: foundry
  name: 'chat-main'
  sku: { name: deploymentSku, capacity: capacity }
  properties: {
    model: { format: 'OpenAI', name: modelName, version: modelVersion }
    raiPolicyName: filterPolicyName
  }
}

Parameterize capacity, sku.name and raiPolicyName per environment: size capacity to each environment while retaining the required safety baseline. Supply a supported model/version and an existing filter policy; TPM conversion is model/SKU-specific, so capacity 100 is not universally 100 TPM. Run az deployment group what-if in pull requests so the reviewer sees exactly which resources change. A shared parent does not serialize sibling deployments. Use an explicit dependency chain or a Bicep loop with @batchSize(1) when account operations must run sequentially, because concurrent deployment creation on the same account can fail.

3.5 Network and identity design checkpoints

Worked example 3b: diagnosing "works on the portal, fails from the app"

Scenario. After private endpoints are enabled, the Foundry portal playground still works for developers, but the container app returns connection timeouts.

Reasoning. The portal call comes from the developer's network path, which is allowed. The app uses Azure-provided DNS in a VNet that is not linked to the private DNS zone, so the Foundry hostname resolves publicly and cannot use the private path. Distinguish DNS failure, a route timeout and a firewall 403 using the observed response; the portal succeeding does not prove the application uses the same path. The fix is to link the private DNS zone to the app's virtual network (and confirm the app has a route to the private endpoint subnet). Changing the model or key would not help, and neither would re-enabling public access, which defeats the requirement.

4. Comparison tables

Common pitfalls

• Using a catalog model name where the selected API expects a deployment name; Instant Access preview follows a different model-by-name contract.
• Forgetting role assignments for the managed identity (or assigning them at too broad a scope).
• Deploying to a region without quota or model availability and discovering it at deployment time.
• Letting prod pull the latest model version automatically.
• Treating the evaluation set as optional in the pipeline.
• Using a single environment for dev and prod.

Exam tips

• "No secrets in code or configuration" → managed identity + DefaultAzureCredential, and OIDC for pipelines.
• "Processing must stay in a geography" → Data Zone deployment; "single region" → regional Standard.
• "Prevent a regression from reaching production" → automated evaluation gate plus approval.
• "429 errors" → retry with backoff first, then capacity changes.
• Pay attention to where a role is assigned: the identity that runs the code needs it, not the person who built it.

Mini case study

Scenario. Adatum builds a claims-assistant agent. Dev, test and prod must be separated; model inference must stay within the US data zone, with separate residency controls for connected services; weekly prompt updates are expected; auditors want proof that every change was tested; the security team forbids API keys.

Solution. Three resource groups (or subscriptions) provisioned from the same Bicep template with environment parameters; Data Zone (US) deployments pinned to specific model versions; managed identities with least-privilege roles for the app and the pipeline's OIDC identity; agent instructions and tool schemas in Git; an evaluation job gating promotion; GitHub environment approvals for production; deployment logs and evaluation reports kept as audit evidence; Application Insights for runtime telemetry.

Commentary. Each requirement maps to a specific mechanism: model-processing residency → an eligible Data Zone deployment; weekly changes + audit → Git and automated evaluation; no keys → managed identity and OIDC; controlled normal upgrades → explicit versions plus a retirement migration plan. Using API keys in pipeline variables would violate policy, and auto-updating model versions require service-version change records and evaluation evidence as well as Git history, because behavior can change without a code change.

Key terms

• Foundry resource / project: shared host and workspace for models, agents and files.
• Deployment name: identifier your code uses to call a model.
• TPM / RPM: tokens-per-minute and requests-per-minute quota.
• Managed identity: Azure-managed credential for a resource.
• OIDC federation: pipeline authentication without stored secrets.
• Evaluation gate: pipeline step failing the build if quality or safety falls under a threshold.
• Infrastructure as code: declarative templates (Bicep, Terraform).

Summary checklist

• I can lay out resource groups, Foundry resource, projects, Search, Storage and networking.
• I can pick region and deployment type from residency, quota and latency needs.
• I can deploy a model and call it by deployment name with keyless authentication.
• I can explain what deploying an agent means and keep its definition in Git.
• I can design a pipeline with OIDC, evaluation gates, approvals and pinned model versions.
• I can troubleshoot 401/403 (roles) and 429 (throttling).

Sources

• AI-103 study guide (opens in a new tab)

• Microsoft Foundry documentation (opens in a new tab)

• Azure OpenAI deployment types (opens in a new tab)

• Workload identity federation for GitHub Actions (opens in a new tab)

• Foundry resource and project architecture (opens in a new tab)

• Foundry RBAC roles and scopes (opens in a new tab)

• Create a prompt agent with the current SDK (opens in a new tab)

• Model retirement schedule (opens in a new tab)

• Azure Login action (opens in a new tab) and Checkout action (opens in a new tab)