Skip to content
FR

Design High-Performing and Elastic Compute Solutions

Step 13 of 28

AWS Compute Selection and Auto Scaling for SAA-C03

Where this lesson fits

This lesson covers Domain 3: Design High-Performing Architectures, Task 3.2: Design high-performing and elastic compute solutions. Domain 3 represents 24% of the scored content of SAA-C03. AWS compute selection matches workload duration, resource needs and scaling signals. Questions in this task give you a workload profile — a steady web tier, a bursty API, a nightly batch of 40,000 independent jobs, a memory-hungry analytics engine — and ask which compute service, which instance family, and which scaling policy and metric deliver the required performance elastically.

Purchasing options (Spot, Savings Plans, Reserved Instances) belong to the cost domain and are covered in EC2 rightsizing, purchasing, and elastic workloads. Here the question is always: does this compute choice keep up with demand, and does it scale on the right signal?

Learning objectives

After this lesson you will be able to:

  • Choose among Amazon EC2, AWS Lambda, AWS Fargate, Amazon ECS/EKS, AWS Batch, and Amazon EMR for a stated workload.
  • Read an EC2 instance type name and match instance families to CPU, memory, storage, and accelerator profiles.
  • Explain burstable (T family) behaviour, CPU credits, and the unlimited mode trap.
  • Configure Auto Scaling groups with the right policy type (target tracking, step, scheduled, predictive) and the right CloudWatch metric.
  • Scale a queue-driven worker fleet on backlog per instance rather than CPU.
  • Use warm pools, lifecycle hooks, instance refresh, and Application Auto Scaling for non-EC2 resources.
  • Recognize Lambda's concurrency model, cold-start mitigations, and quotas.

1. Choosing the compute model

Service You manage Scaling unit Best fit Watch out for
Amazon EC2 OS, runtime, patching, capacity Instance (via Auto Scaling group) Long-running servers, licensed software, specific kernels or GPUs, lift-and-shift You own scaling configuration and patching
AWS Lambda Only code and configuration Concurrent execution per request/event Event-driven work, APIs, glue, spiky and unpredictable traffic 15-minute maximum duration, 10,240 MB memory for standard functions, cold starts, no persistent local state
AWS Fargate (with ECS or EKS) Container image plus ECS task definition or Kubernetes pod specification Task or pod Containers without node management, steady or variable microservices Per-task vCPU/memory combinations; no daemonset-style host access
Amazon ECS / EKS on EC2 Container orchestration on your nodes Task/pod plus node capacity Container workloads needing GPUs, special networking, or maximum control Two scaling layers (tasks and nodes) must both be configured
AWS Batch Job definitions and queues Job (array jobs for thousands) Batch computing: simulations, rendering, genomics; managed compute environments on EC2, Fargate, or Spot Container jobs can be independent or use dependencies; size quotas and available capacity
Amazon EMR Cluster configuration Cluster nodes / serverless capacity Hadoop, Spark, Hive, Presto analytics at scale Cluster lifecycle; consider EMR Serverless for intermittent jobs

Decision shortcuts the exam rewards:

  • Short, event-triggered, unpredictable → Lambda.
  • Containers, no servers to manage → Fargate.
  • Thousands of independent compute jobs with dependencies and retries → AWS Batch.
  • Spark/Hadoop analytics → EMR (or EMR Serverless).
  • Needs a specific OS, GPU driver, licence, or runs for hours continuously → EC2 in an Auto Scaling group.

2. Reading EC2 instance types

An instance type such as m7g.2xlarge decomposes into family (m = general purpose), generation (7), capability letters (g = AWS Graviton, i = Intel, a = AMD, d = local NVMe, n = extra network bandwidth, e = extra memory or storage), and size (2xlarge).

Family Optimized for Typical workloads
T (t3, t4g) Burstable, baseline CPU with credits Low-traffic web servers, dev/test, microservices with idle periods
M (m6i, m7g) Balanced CPU and memory Application servers, mid-size databases, backends
C (c6i, c7g) Compute (high CPU per GB) Batch processing, ad serving, HPC front ends, game servers, media transcoding
R, X, z Memory In-memory caches, large relational databases, SAP HANA (X), high-frequency trading (z)
I, D, H Local storage throughput/IOPS NoSQL databases, data warehouses, distributed file systems
P, G, Trn, Inf Accelerators (GPU, Trainium, Inferentia) Model training, graphics, inference

AWS Graviton instances (the g suffix) run Arm64 and can improve price-performance for workloads that can be recompiled or that run on managed runtimes; AWS continues to iterate on them, with Graviton5-based instance families announced in 2026 and supported by managed services such as Lambda Managed Instances. The exam asks about Graviton as the answer to "improve price-performance for a compute-bound Linux workload we can rebuild".

2.1 Burstable instances and CPU credits

T-family instances earn CPU credits while below their baseline utilization and spend them when above it. When credits are exhausted in standard mode, the instance is throttled to its baseline (for example 20% of a vCPU), which shows up as mysterious, sustained slowness. Two correct answers exist depending on the stem:

  • The workload is genuinely steady and CPU-bound → move to an M or C family instance.
  • The workload is mostly idle with short spikes → enable unlimited mode (it bursts beyond credits for an additional charge) or keep standard mode if cost predictability matters.

Exam tip: "performance degrades after a few hours of load on a t3.medium" is a CPU-credit question, not a disk or network question.

3. Auto Scaling: policies and metrics

An Auto Scaling group (ASG) holds a launch template, a minimum, maximum, and desired capacity, and a set of subnets across AZs. Its scaling behaviour comes from policies:

Policy How it works Use when
Target tracking Keeps a metric at a target value (for example average CPU 50%, or ALBRequestCountPerTarget at 1,000) The default answer for most workloads: simple and self-correcting
Step scaling Adds or removes a defined number of instances per alarm breach size You need different reactions to moderate versus severe breaches
Simple scaling One adjustment per alarm, with a cooldown Legacy; rarely the best answer
Scheduled scaling Changes capacity at known times Predictable calendar patterns: market open, nightly batch, Monday payroll
Predictive scaling Machine learning forecasts load from history and pre-provisions capacity Recurring daily or weekly cycles where reactive scaling is too slow

Choosing the metric matters as much as the policy:

  • A web tier behind an ALB scales well on ALBRequestCountPerTarget or CPU.
  • A queue-driven worker fleet can scale on backlog per instance — a custom CloudWatch metric computed from ApproximateNumberOfMessagesVisible divided by the running instance count — because CPU can stay low while messages pile up.
  • An I/O-bound fleet may need a proportional custom metric such as work per instance. Use job latency for diagnosis or a suitable alarm-driven step policy; latency alone is not generally a proportional target-tracking metric.

3.1 Making scaling fast enough

Technique Effect
Warm pools Keep bootstrapped instances stopped, running or hibernated. Stopped instances retain disk setup, not RAM; use supported hibernation to preserve an in-memory cache
Lifecycle hooks Pause an instance at launch or terminate to run configuration or draining logic
Instance refresh Roll out a new launch template version (new AMI) gradually with a minimum healthy percentage
Health check grace period Delay optional health-check replacement after entering service; configure instance warmup separately for scaling metrics
Detailed / high-resolution CloudWatch metrics 1-minute (or finer) data points let alarms fire sooner than the 5-minute default
Golden AMIs Bake dependencies into the image instead of installing them at boot
Mixed instance types and multiple AZs More capacity pools to draw from, fewer InsufficientInstanceCapacity failures

Managed services are improving here too: Amazon ECS added opt-in 20-second CPU/memory metrics for supported target-tracking policies in 2026; enable the metric resolution and select the matching policy metric — a useful illustration of the same principle, that scaling speed is bounded by how fast you observe demand.

3.2 Scaling beyond EC2

Application Auto Scaling extends the same target-tracking model to ECS services, DynamoDB tables and indexes, Aurora Replicas, Lambda provisioned concurrency, SageMaker endpoints, and more. Lambda scales concurrency automatically; DynamoDB on-demand scales automatically within table/account limits and warm-throughput constraints; sudden growth beyond twice the previous peak can still throttle. When a question says "scale the container service", the answer is an ECS service auto scaling policy on a metric such as CPU, memory, or request count per target — and, when nodes are on EC2, a capacity provider so node capacity follows task demand.

4. AWS Lambda performance model

  • Memory is the performance dial. CPU allocation scales with configured memory; benchmark network and storage paths as well (128 MB to 10,240 MB for standard functions, with roughly 6 vCPU at the top). Doubling memory can halve duration and sometimes cost the same.
  • Timeout for an ordinary invocation is at most 15 minutes. Eligible Lambda Managed Instances asynchronous/event-source invocations support up to 90 minutes (excluding Amazon MQ/DocumentDB mappings). Durable executions can checkpoint and wait across invocations; an existing long container job often fits Fargate or Batch better.
  • Ephemeral storage in /tmp is configurable from 512 MB to 10 GB.
  • Concurrency: the account has a default concurrent-execution quota (commonly 1,000, raisable; new accounts may start lower). Reserved concurrency both reserves and caps a function's share, without keeping environments warm. Provisioned concurrency pre-initializes allocated environments, but traffic spilling beyond that capacity can cold-start. SnapStart restores initialized snapshots for supported runtimes and cannot be combined with provisioned concurrency.
  • Cold starts depend on runtime, package and initialization work. Hyperplane ENIs remove the old per-environment VPC attachment penalty, but initial ENI provisioning and dependencies still affect readiness; measure initialization rather than assuming VPC attachment explains every spike.
  • Lambda Managed Instances offers EC2 instance selection and configurable memory-to-vCPU ratios while AWS manages infrastructure. Graviton5 options were announced in September 2026. Check the selected compute model, invocation type, runtime and Region; do not apply ordinary-function quotas universally or assume an EC2 image runs unchanged as a Lambda handler.

Exam tip: "ordinary function times out on an uninterrupted 40-minute video job" → evaluate Fargate, Batch or MediaConvert; an eligible Managed Instances function has a different timeout. "Occasional 2-second latency on the first request" → provisioned concurrency or SnapStart.

5. Worked example: scaling a queue-driven image pipeline

Scenario. PixelForge processes user-uploaded images. An SQS queue receives upload events. A fleet of EC2 instances (c6i.large) pulls messages and runs a CPU-heavy transform taking 40 seconds per image. Traffic is flat for most of the day and multiplies by 20 for three hours each evening. Today the ASG scales on average CPU at 60% and customers complain of hours-long delays in the evening.

Step 1 — diagnose the delay. Average CPU near 100% exceeds the 60% target and does trigger scale-out when the policy can act. CPU does not directly quantify the waiting queue, so it may react too slowly to a 20-fold burst. Inspect scaling activities, maximum capacity, instance quotas and warm-up before blaming the metric. A saturated metric is not an inactive policy.

Step 2 — pick the right metric. Publish a custom CloudWatch metric backlog per instance = ApproximateNumberOfMessagesVisible / GroupInServiceInstances. Decide the acceptable latency: if a message must start within 5 minutes and one instance clears 1 message per 40 seconds (about 7.5 per 5 minutes), the target backlog per instance is roughly 7.

Step 3 — choose the policy. Use target tracking with target 7 as a starting point. This approximates 7 × 40 = 280 seconds of waiting per single-slot worker, not a strict latency guarantee. Handle zero in-service instances in metric math and keep a minimum worker or an explicit scale-from-zero trigger.

Step 4 — make scale-out fast enough. Bake the transform dependencies into a golden AMI, set a health check grace period that matches bootstrap time, and if boot still takes minutes, add a warm pool of stopped, pre-initialized instances. Because the evening peak is a known calendar pattern, add scheduled scaling (or predictive scaling) to raise the minimum capacity shortly before it starts, so the reactive policy only handles deviations.

Step 5 — protect correctness while scaling. Set SQS visibility timeout above processing plus deletion time and extend it for variable jobs. A termination lifecycle hook lets workers stop polling and drain, but must send heartbeats/completion and has a deadline; it cannot extend a Spot interruption. Use idempotent processing, checkpoints and optional scale-in protection for busy instances.

Why other answers are worse. Moving to a bigger instance size raises throughput by a fixed factor and still cannot absorb a 20x spike. Switching to Lambda is attractive but each transform is CPU-heavy and sustained; it can work within the 15-minute limit, yet the question's existing fleet with a corrected metric is the minimal change (Lambda or Fargate would be the right answer if the stem emphasized avoiding server management). If maximum capacity is the blocker, raising it can allow the existing CPU policy to add instances. If slow startup or backlog recovery is the blocker, a higher maximum alone does not resolve that delay.

6. Comparison: scaling signals at a glance

Workload Scale on Service pattern
Web/API behind ALB ALBRequestCountPerTarget or CPU; p95 latency as an alarm ASG target tracking, or Fargate service auto scaling
Queue workers Backlog per instance ASG target tracking on a custom metric
Streaming consumers ( Kinesis) Shard count and iterator age Lambda event source mapping, or KCL workers with enhanced fan-out
Batch jobs Job queue depth AWS Batch managed compute environment
Spark analytics Cluster utilization / step queue EMR managed scaling or EMR Serverless
Cron-like calendar peaks Time Scheduled scaling, or predictive scaling for recurring cycles

7. Common pitfalls

  • Assuming CPU above its target cannot scale out; it can, but backlog per instance measures queue recovery needs more directly.
  • Leaving the ASG health check at EC2 only, so application-level failures are never replaced (see Chapter 5).
  • Forgetting that both layers must scale for ECS/EKS on EC2: tasks and node capacity (capacity providers or Karpenter/managed node groups).
  • Ignoring CPU credits on T instances for steady workloads.
  • Exceeding an ordinary Lambda invocation's 15-minute limit or expecting persistent local state between invocations.
  • Not reserving concurrency, so one noisy function exhausts unreserved concurrency in the same account and Region.
  • Placing an ASG in one subnet/AZ, which removes both resilience and capacity pools.
  • Relying on default 5-minute metrics when a spike needs reaction in seconds.
  • Using cluster placement groups for a fleet that needs capacity elasticity: they constrain placement to one AZ and limited hardware.
  • Treating predictive scaling as a replacement for reactive policies; AWS recommends combining it with target tracking.

8. Exam tips

  • "No servers to manage" + containers → Fargate; + event-driven code → Lambda.
  • "Tens of thousands of independent jobs, with retries and priorities" → AWS Batch with array jobs.
  • "Spark/Hive/Presto" → EMR; intermittent and unpredictable → EMR Serverless.
  • "Compute-bound Linux workload, improve price-performance, can rebuild" → Graviton.
  • "Memory-intensive in-memory database" → R/X family; "high local IOPS" → I family.
  • "Queue grows while CPU stays low" → scale on backlog per instance.
  • "Known daily peak" → scheduled or predictive scaling; "unknown spikes" → target tracking.
  • "Instances take 8 minutes to boot and scale-out is too slow" → warm pool plus golden AMI.
  • "First invocation latency matters" → provisioned concurrency or SnapStart.

9. Mini case study (with solution)

Scenario. LedgerOne runs three workloads:

  1. A customer-facing REST API in Java behind an ALB. Traffic is flat overnight, rises sharply at 08:00, and p95 latency must stay under 300 ms. Instances take about 6 minutes to boot and warm their JVM cache.
  2. A monthly risk-simulation batch of 50,000 independent Monte Carlo tasks, each 3 to 12 minutes long and containerized, which must finish within 6 hours.
  3. A nightly Spark job over 4 TB in Amazon S3 that runs for 90 minutes and is idle the rest of the day.

Each workload is currently a fixed fleet of oversized EC2 instances, and the team reports slow mornings, a batch that takes 20 hours, and an idle Spark cluster.

Questions. Propose compute and scaling for each workload, with the metric or mechanism used.

Solution.

  1. API tier: keep EC2 in an ASG across three AZs (or move to Fargate if the team wants to stop managing hosts) with target tracking on ALBRequestCountPerTarget and a p95 latency alarm for visibility. Because boot and JVM warm-up take 6 minutes, use a golden AMI and a warm pool that preserves only the state you need: stopped instances retain disk bootstrap; supported hibernated instances can retain the warmed JVM heap. Test resume and readiness, and use scheduled or predictive scaling to raise the minimum capacity before 08:00. Use ELB health checks, appropriate grace and default instance warmup; a grace period is not a substitute for a readiness endpoint.
  2. Monte Carlo batch: AWS Batch with five array jobs of 10,000 tasks (10,000 is the non-adjustable maximum per array job), orchestrated with a workflow or submission service. Use compute-optimized C-family containers across instance types/AZs, with retry/checkpoint handling for Spot interruptions. At the conservative 12-minute duration, 50,000 tasks require 600,000 task-minutes: a six-hour window needs at least 1,667 simultaneous task slots before startup, retries and safety margin. Confirm each task's vCPU/memory request, EC2/Batch quotas and actual capacity; this calculation is a planning lower bound, not a deadline guarantee.
  3. Spark job: replace the always-on cluster with a transient EMR cluster created for the job and terminated at completion, with EMR managed scaling, or with EMR Serverless with configured capacity limits, pre-initialized capacity and auto-stop settings. Keep durable input/output and required state in S3 or external stores; reading S3 alone does not make every application stateless.

Why not alternatives. T instances in standard mode can throttle after credits are exhausted; unlimited mode can instead incur surplus-credit charges. Scaling the API on CPU alone misses latency caused by downstream waits. Implementing the Monte Carlo batch as 50,000 Lambda invocations is plausible for short tasks, but 12-minute tasks leave little headroom under the 15-minute limit and Batch gives queueing, priorities, and retries natively. Keeping a permanent EMR cluster wastes more than 22 hours of capacity every day.

10. Glossary

  • Launch template: versioned instance configuration used by an Auto Scaling group.
  • Target tracking: scaling policy that holds a CloudWatch metric at a target value.
  • Predictive scaling: forecast-based provisioning ahead of recurring demand.
  • Backlog per instance: queue depth divided by in-service instances, a useful proportional metric for queue workers.
  • Warm pool: bootstrapped instances kept stopped, running or hibernated for faster scale-out; RAM survives only supported hibernation/running state.
  • Lifecycle hook: pause point in launch or termination for custom actions.
  • Instance refresh: progressive replacement of instances with a new launch template version.
  • CPU credit: burstable-instance accounting unit that allows CPU above baseline.
  • Reserved concurrency: Lambda setting that caps or guarantees concurrent executions.
  • Provisioned concurrency: pre-initialized Lambda environments for allocated capacity; spillover can still cold-start.
  • Array job: AWS Batch job that fans out into many identical, independently indexed tasks.
  • Capacity provider: ECS mechanism that scales node capacity to match task demand.

11. Summary checklist

  • I can choose between EC2, Lambda, Fargate, ECS/EKS, Batch, and EMR from a workload description.
  • I can map instance families (T, M, C, R/X/z, I/D/H, P/G/Inf/Trn) and Graviton to workload profiles.
  • I can explain CPU credits and the two correct remedies for throttled T instances.
  • I can pick target tracking, step, scheduled, or predictive scaling and justify the choice.
  • I can define backlog per instance and explain why backlog better reflects waiting work than CPU alone.
  • I know warm pools, lifecycle hooks, instance refresh, grace periods, and golden AMIs.
  • I know Lambda's duration, memory, storage, and concurrency model, and how to remove cold starts.
  • I know that ECS/EKS on EC2 requires scaling both tasks and nodes.

For the queue and orchestration choices behind these scaling signals, review SQS, events, containers, and serverless architecture. Combine capacity planning with Multi-AZ availability and recovery design.

Sources