The architecture utilizes multiple Availability Zones (AZs) within AWS regions to achieve high availability. By spreading resources across AZs, the system can continue functioning even if one AZ becomes unavailable. Load balancers are implemented to distribute traffic across instances and AZs, ensuring redundancy and fault tolerance. 

All microservices were rehosted on the ECS cluster(Blue/Green) and internally exposed

through internally facing NLB. APIs are publicly accessible through an API gateway via a

custom domain. The databases used are RDS Postgres instance and Redis. 

The architecture incorporates auto-scaling mechanisms that dynamically adjust resources to meet demand, thereby optimizing performance and availability. Resilience is enhanced through the integration of self-healing components. 

In Amazon ECS, unhealthy tasks/container are automatically detected and replaced, with restarts occurring across the AZs within the AWS Region. AWS services like Auto Scaling Groups monitors container health and proactively replace or redistribute resources in the event of failures. This automated recovery process minimizes downtime and ensures uninterrupted availability without the need for manual intervention. 

To enhance fault isolation, the solution employs a multi-account strategy. This approach facilitates logical separation of resources and enforces strict access controls to prevent widespread disruptions. A Management Account is created using AWS Organizations, while separate accounts are established for production, non-production, logging, audit, and security purposes. 

For data resilience and redundancy, Amazon RDS offers built-in replication mechanisms that replicate data across multiple AZs. Additionally, AWS Backup is periodically executed to ensure data protection and recovery capabilities.

AWS CloudWatch, a centralized monitoring and management service, played a crucial role in monitoring the health and performance of Pando's resources. The logs from ECS were directed to CloudWatch log groups, allowing Pando to visualize metrics and monitor the application's health through CloudWatch dashboards. Alerts and notifications were set up to promptly notify stakeholders of any downtime experienced, ensuring proactive issue resolution. By utilizing CloudWatch metrics, Pando could monitor other managed services like RDS and Load Balancers, providing a comprehensive view of their infrastructure's operational condition.

To enhance awareness and notification capabilities, Pando utilized Amazon EventBridge to create health notifications for AWS services. An EventBridge rule was created and integrated with Amazon SNS for service notifications. Stakeholders received SNS email subscriptions to stay informed about any critical service updates or incidents. This approach ensured that Pando had real-time visibility into the availability and health of their AWS resources.

Comprinno implemented a highly available DNS solution using Amazon Route53, a scalable and reliable cloud domain name system. Route 53 played a vital role in directing and supporting health checks, continuously monitoring the availability of resources and automatically routing traffic to healthy endpoints. Additionally, Route53 provided DNS failover capabilities, redirecting traffic to alternate resources in the event of a failure, and ensuring uninterrupted DNS resolution.

Infrastructure was automated using Terraform.  This approach allowed for consistent and auditable deployments, ensuring that infrastructure changes were managed effectively. The Terraform code was stored in GitHub, enabling version control and collaboration among the development team.

AWS Config and AWS CloudTrail were implemented to provide detailed views of the configuration of AWS resources and track changes over time. AWS Config allowed Pando to assess resource configuration compliance against desired configurations and detect any unauthorized changes. AWS CloudTrail served as a comprehensive audit trail of all actions taken within Pando's AWS account, enabling monitoring and validation of changes as part of their change management process. 

CI/CD pipeline was built for automated deployments. Deployment is triggered whenever code is committed to GitHub repository. AWS CodeBuild was used for building the docker image and then the image was pushed to AWS ECR (Elastic Container Registry). The built-in capability of ECR to scan docker images for known vulnerabilities was leveraged and the pipeline proceeds to deployment only when no Critical OR High severity vulnerabilities are reported by ECR.

A notification alert was setup using AWS SNS to report developers about the failed pipeline.