Skip to content
FR

Design Highly Available and Fault-Tolerant Architectures

Step 9 of 28

AWS High Availability and Disaster Recovery: Multi-AZ, RPO, and RTO

Where this lesson fits

This lesson covers Domain 2: Design Resilient Architectures, Task 2.2: Design highly available and fault-tolerant architectures. Together with Task 2.1 (previous lesson), Domain 2 represents 26% of the scored content of SAA-C03. AWS high availability and disaster recovery designs begin with a failure domain and measurable recovery objectives. Questions here give you a business requirement ("the application must survive the loss of a data center", "recover within 15 minutes with at most 5 minutes of data loss") and expect you to pick the cheapest architecture that meets it.

Learning objectives

After this lesson you will be able to:

  • Distinguish high availability, fault tolerance and disaster recovery.
  • Use Availability Zones and Regions correctly in a design.
  • Translate RPO and RTO into one of the four DR strategies: backup and restore, pilot light, warm standby, multi-site active-active.
  • Configure load balancing, Auto Scaling health checks and Amazon Route 53 routing and health checks for failover.
  • Choose the right Multi-AZ and cross-Region options for Amazon RDS, Amazon Aurora, Amazon DynamoDB, Amazon S3, Amazon EFS and Amazon EBS.
  • Recognize the services dedicated to recovery: AWS Backup and AWS Elastic Disaster Recovery.

Architecture map

Solid arrows show normal requests and local replication. Dashed cross-Region replication and tested promotion support DR; application recovery must meet the chosen RTO/RPO.

1. Vocabulary: three different goals

Term Meaning Example
High availability (HA) The system keeps serving with minimal interruption when a component fails Web servers in two AZs behind an ALB; a failover takes seconds to minutes
Fault tolerance (FT) The system continues with no interruption or data loss when a component fails Independent active replicas designed for a specified failure mode; managed redundancy alone does not guarantee uninterrupted calls or zero data loss in every failure
Disaster recovery (DR) Restoring service after a large-scale event (Region outage, data corruption, ransomware) Pilot light in a second Region

Fault tolerance usually costs more than high availability. The exam rewards the least costly design that meets the stated requirement. If a question says "tolerate the failure of an AZ" a Multi-AZ design is enough; do not choose a multi-Region active-active design unless it mentions Region failure.

1.1 Availability math

Availability is usually expressed as a percentage of uptime. 99.9% allows about 8.8 hours of downtime per year, 99.95% about 4.4 hours, 99.99% about 52 minutes. Components in series multiply their availability (two 99.9% parts give about 99.8%); redundant components in parallel raise it dramatically: two independent 99.9% copies give 1 minus (0.001 times 0.001), or 99.9999%. These simplified formulas assume independent failures, working failover and sufficient surviving capacity; shared dependencies can invalidate that assumption.

2. AWS global infrastructure and failure domains

  • A Region is a geographic area with multiple, isolated AZs. Check service-specific data residency and replication behavior; Regional isolation does not by itself cover every global service or optional cross-Region feature.
  • An Availability Zone is one or more data centers with independent power, cooling and networking, connected by low-latency links (single-digit milliseconds). Synchronous replication between AZs is practical.
  • Edge locations serve CloudFront and Route 53; Local Zones and Wavelength extend latency-sensitive workloads.

Failure domains, from smallest to largest: instance, rack/host, AZ, Region. Design to survive the failure domain your requirement names. AZ names are mapped per account (us-east-1a in your account may be a different physical AZ than in another), so use AZ IDs (use1-az1) when coordinating across accounts.

Rule: a production workload spans at least two AZs; the classic answer places the ALB, the Auto Scaling group and the database standby in different AZs.

3. Compute-tier resilience

3.1 Elastic Load Balancing

Load balancer Layer Notes
Application Load Balancer (ALB) 7 Path/host routing, containers, Lambda targets, WebSockets; cross-zone load balancing is on by default
Network Load Balancer (NLB) 4 Millions of requests per second, static IP per AZ (Elastic IPs), TLS passthrough; cross-zone is off by default (and incurs inter-AZ data charges when on)
Gateway Load Balancer 3 Inserts third-party virtual appliances (firewalls)

Load balancers use health checks to avoid unhealthy targets while healthy capacity remains. If every target is unhealthy, ALB/NLB can fail open and route to unhealthy targets; a health check is not an unconditional traffic block. They are themselves deployed across the AZs you enable. Enable at least two AZs.

3.2 Auto Scaling groups

An Auto Scaling group (ASG) spanning several AZs keeps the desired capacity: when an instance fails a health check, the group terminates and replaces it, and when an AZ becomes impaired the group rebalances capacity in the remaining AZs. Use ELB health checks (not only EC2 status checks) so the application-level failure also triggers replacement. A group of size 1 with a launch template is the cheapest "self-healing" pattern for a single instance; you may also use EC2 Auto Recovery for a host failure when the instance has no replacement requirement.

Stateless design makes this work: sessions in ElastiCache or DynamoDB, uploads in S3 or EFS, configuration in Parameter Store.

3.3 Placement groups

A spread placement group places instances on distinct hardware (at most 7 running instances per AZ per group) to limit correlated failures for a small number of critical instances. A cluster group favors low latency but concentrates risk. Partition groups isolate large distributed systems such as Hadoop or Kafka.

4. Data-tier resilience

Service Built-in resilience What you configure
Amazon S3 Objects stored across at least 3 AZs (except One Zone-IA/Express One Zone), designed for 11 nines of durability Versioning; Cross-Region Replication for Region-level DR; Multi-Region Access Points
Amazon EBS Replicated within one AZ Snapshots (stored in S3, restorable in any AZ), copy snapshots to other Regions, Data Lifecycle Manager or AWS Backup
Amazon EFS Regional file system across multiple AZs (One Zone option available) Mount targets in every AZ; replication to another Region; AWS Backup
Amazon RDS (Multi-AZ instance) Synchronous standby in another AZ, automatic failover typically 60 to 120 seconds; the standby is not readable Enable Multi-AZ; it is for availability, not read scaling
Amazon RDS Multi-AZ DB cluster Two readable standbys in different AZs, failover typically about 35 seconds MySQL and PostgreSQL only
Amazon Aurora Storage replicated six ways across three AZs; up to 15 Aurora Replicas that double as failover targets; failover usually under 30 seconds Place replicas in several AZs; Aurora Global Database for cross-Region
Amazon DynamoDB Replicated across three AZs automatically Global tables (multi-Region, multi-active) for Region-level resilience; PITR configurable from 1 to 35 days
Amazon ElastiCache Replication groups with Multi-AZ automatic failover Enable Multi-AZ for Redis/Valkey

Important distinctions:

  • Multi-AZ is not a read scaling feature (for the classic RDS instance deployment). Read replicas scale reads and can be cross-Region, but promoting one is a manual action (or part of your runbook) and replication is asynchronous.
  • Aurora Global Database replicates storage across Regions with typical lag under one second, and supports managed cross-Region failover with an RTO on the order of minutes. Replication lag and application recovery determine actual RPO/RTO. A planned switchover synchronizes data for zero RPO; an unplanned failover can lose unreplicated writes.
  • DynamoDB global tables support multi-Region eventual consistency (MREC, asynchronous replication with last-writer-wins conflict resolution) and multi-Region strong consistency (MRSC, synchronous coordination with zero RPO for acknowledged writes). Select MRSC only where supported Region groups and feature restrictions fit; it is not a toggle that makes every existing MREC design strongly consistent.
  • Retained, usable backups protect against logical corruption and deletion (a standby also receives valid destructive writes). Resilience needs both replication and backup.

5. Disaster recovery strategies

Two numbers drive every DR decision:

  • RPO (Recovery Point Objective): the maximum acceptable amount of data loss, measured in time (a 5-minute RPO means you can lose at most 5 minutes of writes).
  • RTO (Recovery Time Objective): the maximum acceptable downtime before service is restored.

Lower RPO and RTO cost more. AWS describes four strategies on a spectrum:

Strategy What runs in the recovery Region RPO and RTO design considerations Relative cost
Backup and restore Nothing; backups (and infrastructure as code templates) are copied RPO follows usable backup freshness; RTO follows provisioning and restore $
Pilot light Data replicated continuously; core resources (database) running, application servers off or at zero Low replication RPO; RTO includes launching the app tier $
Warm standby A scaled-down but fully functional copy runs continuously Low replication RPO; RTO includes promotion and scaling $$
Multi-site active-active Multiple Regions serve traffic; remaining Regions must absorb failed-site load Near-zero targets depend on data replication and conflict handling $$$$

How to choose: read the RPO/RTO, then pick the cheapest strategy that meets them. "Within hours" implies backup and restore. "Within 30 minutes with minimal cost" suggests pilot light. "Within minutes; can scale up at failover" is warm standby. "Seconds, minimal interruption" suggests active-active. These are design targets, not service guarantees: include detection, promotion, scaling, DNS, reconnection, quotas and recovery testing. Continuous/PITR backups can also provide a finer RPO than periodic snapshots when the service and Region-copy path support it.

5.1 Supporting services

  • AWS Backup: centralized backup plans, cross-Region and cross-account copies where supported, and Vault Lock; compliance-mode lock becomes immutable after its grace period expires; the standard answer for a policy-driven, low-effort backup strategy.
  • AWS Elastic Disaster Recovery (DRS): continuous block-level replication of servers (on premises or other clouds, or EC2) to a low-cost staging area in AWS, enabling recovery in minutes; the managed pilot-light mechanism for servers.
  • AWS CloudFormation / infrastructure as code: recreate the stack in the recovery Region quickly and reproducibly. AMIs and snapshots must be copied to that Region in advance.
  • Amazon Application Recovery Controller (ARC): routing controls and readiness checks for coordinated, safe failovers of multi-Region applications.
  • AWS Fault Injection Service: run controlled experiments (terminate instances, impair AZ-level network) to verify resilience.
  • Testing: an untested DR plan is not a plan. Schedule game days.

6. Failover at the DNS and network edge

6.1 Amazon Route 53

Route 53 is a highly available DNS service with a 100% availability SLA for the data plane. Routing policies:

Policy Use
Simple One resource, no health checks
Weighted Split traffic by percentage (blue/green, canary)
Latency Send users to the Region with the lowest latency
Failover Active-passive: primary record answers while its health check passes, otherwise the secondary
Geolocation Route by user location (compliance, localization)
Geoproximity Route by geographic distance with an adjustable bias
Multivalue answer Return up to 8 healthy records
IP-based Route by client CIDR

Health checks can monitor endpoints, other health checks (calculated) or CloudWatch alarms (useful for private resources). Alias records can point the zone apex at supported AWS targets such as ALB, CloudFront, S3 websites or Global Accelerator. Alias queries to supported AWS resources have no Route 53 query charge. Health evaluation is target-specific: use "Evaluate target health" for ALB/NLB, but it cannot be true for a CloudFront alias. A direct ALB/NLB alias inherits the target TTL (60 seconds); for non-alias failover records, configure a low TTL. Resolver/client caches and existing connections can still delay switchover.

6.2 AWS Global Accelerator and CloudFront

  • Global Accelerator gives two static anycast IP addresses and routes over the AWS backbone to the closest healthy endpoint; traffic can shift to healthy endpoints without waiting for DNS caches to expire. Health detection, endpoint readiness and client reconnection determine the observed recovery time; existing TCP sessions are not transparently moved. Choose it for non-HTTP or fast-failover requirements or when clients hard-code IP addresses.
  • CloudFront origin groups provide automatic failover to a secondary origin (for example a second S3 bucket in another Region) when the primary returns configured error codes or qualifying connection/time-out errors. Origin-group failover applies only to GET, HEAD and eligible cached OPTIONS, not POST or PUT; it does not replicate application data.

7. Worked example: matching a design to RPO and RTO

Scenario. TicketWave sells concert tickets using a three-tier application in us-east-1: ALB, Auto Scaling group of EC2 instances, and Amazon Aurora MySQL. The business requires that:

  1. Loss of one AZ must recover within a 60-second application RTO, tolerating brief database errors with retries. This is an availability objective to test, not a guaranteed service failover time.
  2. If the entire Region is unavailable, service must resume in us-west-2 within 15 minutes with at most 1 minute of data loss.
  3. Costs should be as low as possible while meeting the above.

Step 1: AZ requirement. Spread the ALB and ASG over three AZs; place an Aurora Replica in another AZ. Aurora failover can cause transient read/write errors; it often takes under 30 seconds, but this is not a guaranteed maximum. Configure bounded retries with backoff, reconnect through the writer endpoint and verify DNS/connection behavior. Test the complete application against the 60-second RTO; increase the budget or revise the design if failover plus recovery exceeds it. This is HA, not a promise of uninterrupted database calls.

Step 2: Region requirement. Hourly snapshots cannot satisfy a 1-minute RPO. The database needs continuous cross-Region replication: Aurora Global Database, with typical replication lag under a second.

Step 3: RTO of 15 minutes. Pilot light requires launching the application tier during the incident; warm standby keeps a minimum fleet running and scales up. With a 15-minute RTO, a warm standby (a small ASG and ALB already running in us-west-2 pointed at the secondary Aurora cluster) is the safe choice; pilot light could also meet 15 minutes if the AMIs, launch templates and automation are tested, so check the statement for other hints.

Step 4: Failover mechanism. Route 53 failover routing with a health check on the primary ALB (low TTL), or Global Accelerator. Use an orchestrated runbook to promote the secondary Aurora cluster, scale the ASG, verify writable application health, then release traffic. DNS failover alone cannot promote the database and must not send writes to an unready standby.

Why other choices are worse. Multi-site active-active would need a suitable data model and failover capacity; Aurora Global Database itself still has one writer Region, even with write forwarding. Copying snapshots every hour cannot meet a 1-minute RPO. A cross-Region RDS read replica has no managed global failover and its promotion is manual. Placing everything in one AZ with Auto Recovery does not satisfy requirement 1.

8. Worked example: why the database failed over slowly

A company reports that a Multi-AZ RDS for PostgreSQL instance caused a 90-second outage during a failover and that their reporting queries cannot use the standby. The architect explains that the classic Multi-AZ instance deployment keeps a synchronous standby that is not readable and promotes it by flipping the DNS CNAME of the endpoint (60 to 120 seconds). To reduce failover to roughly 35 seconds and use the standbys for reads, migrate to a Multi-AZ DB cluster (two readable standbys) or to Aurora with replicas, and make sure applications do not cache DNS beyond the TTL and reconnect with retries. RDS Proxy can retain eligible client connections and reconnect to the new database, reducing disruption. In-flight transactions, pinned sessions and errors still require tested application retry and reconnection behavior; proxying does not guarantee transaction continuity.

9. Common pitfalls

  • Assuming Multi-AZ RDS gives read scaling. Use read replicas or Aurora Replicas for reads.
  • Confusing replication with backup. Destructive writes and logical errors can propagate to replicas; keep point-in-time recovery and backups.
  • Forgetting that EBS volumes and instance store live in a single AZ; use snapshots or EFS for cross-AZ data.
  • Selecting a warm standby or active-active design when a question only requires AZ resilience, or periodic snapshots when the RPO requires much fresher recovery points.
  • Leaving a high DNS TTL on failover records.
  • Enabling only one AZ on a load balancer or placing the ASG in one subnet.
  • Forgetting that NLB cross-zone load balancing is off by default, so uneven traffic can overload one AZ.
  • Not preparing AMIs, snapshot copies, secrets and KMS permissions in the recovery Region. Use an appropriate destination-Region key for encrypted copies, or explicitly create a related multi-Region replica key where supported. An ordinary single-Region KMS key cannot simply be copied or converted to multi-Region.
  • Assuming a Savings Plan or Regional Reserved Instance guarantees capacity in the DR Region. Only On-Demand Capacity Reservations (or zonal Reserved Instances) reserve capacity; without one, a Regional outage may find the recovery Region short of instances.

10. Exam tips

  • Always map the stem to RPO and RTO first. A small data-loss budget requires sufficiently fresh replication or supported continuous/PITR backups; check the actual recovery points available in the destination Region.
  • "Survive AZ failure": Multi-AZ. "Survive Region failure": cross-Region replication plus Route 53 or Global Accelerator.
  • "Lowest cost DR with an RTO of hours": backup and restore. "Minimal running resources, quick recovery": pilot light.
  • "On-premises servers must be recoverable on AWS quickly and cheaply": AWS Elastic Disaster Recovery.
  • "Same DNS name, route to healthy Region automatically": Route 53 failover with health checks. "Static IPs and rapid failover": Global Accelerator.
  • "Centralized backup compliance across accounts": AWS Backup (with Organizations).
  • "Multi-Region NoSQL with writes in all Regions": DynamoDB global tables.

11. Mini case study (with solution)

Scenario. HarborBank runs an internal loan-approval system: web tier on EC2, a PostgreSQL database on a single self-managed EC2 instance with a 2 TB EBS volume, and documents on an NFS server. Regulators require the service to survive the loss of a data center with less than 5 minutes of downtime, and to be recoverable within 4 hours with no more than 15 minutes of data loss if the whole Region becomes unavailable. Management wants to limit cost and operational effort.

Questions. Propose the architecture for (a) AZ resilience, (b) Region recovery, and (c) the documents.

Solution.

a) Move the database to Amazon RDS for PostgreSQL in a Multi-AZ deployment (or Aurora PostgreSQL with replicas). Run the web tier in an Auto Scaling group across three AZs behind an ALB with ELB health checks. RDS failover is typically 60–120 seconds, but large transactions or recovery work can extend it. Verify that failure detection, database recovery and application reconnection together meet the five-minute target.

b) A 4-hour RTO with a 15-minute RPO fits pilot light: create a cross-Region read replica (asynchronous) or Aurora Global Database in the recovery Region, copy AMIs and keep CloudFormation templates there, with the web tier scaled to zero. In a disaster: promote the replica, run the stack, scale up and change Route 53. Backup and restore with hourly snapshots would violate the 15-minute RPO; compare supported continuous backup replication if its restore time meets the four-hour RTO; active-active is unnecessary for a 4-hour RTO.

c) Replace the NFS server with Amazon EFS (Regional, Multi-AZ) and enable EFS Replication to the recovery Region, or store documents in S3 with Cross-Region Replication. EFS replication targets a 15-minute RPO for most file systems after initial synchronization, but large or frequently changing datasets can lag longer. Monitor TimeSinceLastSync, test failover permissions/mount targets and verify that all data tiers meet the target. Ordinary S3 CRR has no fixed 15-minute deadline; S3 RTC has a conditional replication SLA, not a guarantee for every object. Use retained AWS Backup copies where supported for the compliance trail.

Why not alternatives. One larger EC2 instance with Auto Recovery stays in one AZ. Keeping the self-managed database requires custom replication and failover scripting that RDS provides as a feature. Route 53 geolocation routing is not a failover mechanism.

12. Glossary

  • High availability: minimizing downtime through redundancy.
  • Fault tolerance: continuing operation without interruption through component failure.
  • RPO: maximum acceptable data loss in time.
  • RTO: maximum acceptable time to restore service.
  • Pilot light: minimal always-on core (data) in the recovery Region.
  • Warm standby: scaled-down but running full environment in the recovery Region.
  • Active-active: multiple Regions serving production traffic at once.
  • Failover routing: Route 53 policy for active-passive DNS answers.
  • Aurora Global Database: cross-Region Aurora with storage-level replication.
  • Global tables: multi-Region, multi-active DynamoDB tables.
  • AWS Elastic Disaster Recovery: continuous replication and orchestrated recovery of servers.

13. Summary checklist

  • I can tell HA, FT and DR apart and match each to a requirement.
  • I can compute downtime from availability percentages and combine components in series and parallel.
  • I can choose among backup and restore, pilot light, warm standby and active-active from the stated RPO and RTO.
  • I know Multi-AZ options for RDS, Aurora, ElastiCache, EFS and the limits of EBS.
  • I can configure ALB/NLB health checks, ASG multi-AZ and Route 53 failover with low TTL.
  • I know when to use Global Accelerator, CloudFront origin groups, AWS Backup and Elastic Disaster Recovery.

To protect recovery copies, review AWS encryption and data-protection controls. Check zonal and Regional storage choices alongside network routing, load balancing, and edge failover.

Sources