ABOUT YOMA FLEET
Yoma SH is a large enterprise in Myanmar with activities on many branches such as financial services, transportation services, hospitality and energy sector, among others. Within Yoma, there are multiple business units, one of which is Yoma Fleet. The business activities of Yoma Fleet focus on automobile sales, rentals and carsharing services. To support these services, Yoma Fleet has created multiple web services and websites running on AWS and using multiple AWS services.
GET IN TOUCH
Yoma SH is hosting a large number of workloads in Amazon Web Services (AWS), and continuously expanding their AWS usage. As is common for many companies that move to the cloud, Yoma SH had set up a single AWS account to host the enterprise’s internal workloads, and different business units’ servers and services are all located in that single account. In addition, Yoma SH does provide hosting services for other companies and entities in Myanmar, as well as some testing services, these are hosted in separate AWS accounts. To manage the accounts, Yoma SH has an AWS Organization, under which the internal services account and other accounts belong.
Due to the business growth and the expansion of complexity in the services, the approach of hosting servers and services from different business units slowly became inconvenient. Yoma Fleet in particular experienced issues due to sharing the same account with other business units. Some of the challenges:
- Cost allocation between entities and business units
- Lack of separation between different teams and departments workloads
- Not able to give access to just the team’s own servers and resources, without also giving access to other teams’ resources
- Monitoring, difficulties to identify and filter the resources that belong to a specific entity
- Sharing the same service quotas and resource limits between all teams
- Inability to separate some AWS services in the same account (i.e. QuickSight)
- Lack of logical isolation between unrelated resources
After careful internal review within Yoma SH, a decision was made to solve these challenges by moving all Yoma Fleet workloads to a new AWS account created under the organization. To prepare for this move, Yoma Fleet own business, development, and DevOps teams listed out all their resources and determined which resources the team could be moved between accounts without support. It was realized that an AWS Advanced Partner, Renova Cloud was needed to support this migration.
Renova Cloud provides Infrastructure Managed Services on AWS for Yoma SH; familiar with the account, the team was able to quickly determine the scope and required tasks to complete the Yoma Fleet migration. Even though all the servers and resources were already hosted in AWS, the task of moving them to another account is still complex, because no automated tooling for this exists, and it is a live production system. Some additional complications included that the servers should be moved as-is, without installation of agents, and some of the S3 buckets hosted static websites, requiring the new bucket have the same name as the old. On the other hand, the task was simplified due to the fact that both the source region and the target region were the same (Singapore region). Essentially, the scope in this case was a cross-account migration within the same AWS region.
Renova Cloud analyzed the Yoma Fleet migration scope and provided the plan, including special attention to the priorities highlighted by the customer. An adjusted architecture on AWS was presented along with the schedule and required involvement from the customer’s side. Through discussion with the Yoma Fleet team, it was also determined that certain resources would be re-created in the new account by the DevOps team using different configurations, therefore those resources wouldn’t need to be migrated at the same time.
At a high level, the main AWS services used by Yoma Fleet include a number of EC2 instances hosting web apps, backend APIs, databases and additional services, a number of RDS instances hosting databases including MySQL and PostgreSQL, both in Single-AZ and in Multi-AZ configuration, SQS queues providing application integration, and S3 buckets fronted by CloudFront distributions for file storage, serving static assets and hosting a few static websites. Some of the supporting services include Redis clusters in ElastiCache for caching, Elasticsearch Service domain for data analytics, QuickSight for data analytics visualization, Route 53 and ACM (Certificate Manager) for managing domain DNS and certificates, and CloudWatch for monitoring, alarms, and event rules.
The migration approach after discussing with the customer is divided between persistent data stores and pure configuration. Persistent data stores include databases, S3 buckets, and EC2 instances used to host data or that have custom modifications. These data stores were required to be migrated at the same time and preserving consistency. Some other resources can be treated essentially as pure configuration, and these are not so much migrated that the configuration is recorded and re-created in the destination account. For example, SQS and ElastiCache for Redis are not used to store persistent data in this environment, these could just be re-created in the new account.
On the below image, the high-level view of migration architecture for the cross-account move:
EC2 instances backup & restore cross-account:
S3 buckets cross-account replication:
At a high-level, the following tasks were taken to complete the migration:
1. Details of all resources to be migrated written down to be tracked
2. VPC configuration in destination account including subnets, routing, and VPC Peering
3. CloudFormation stack created to configure SQS, ElastiCache, and other resources that don’t store persistent data; the stack tested to verify resource creation
4. S3 buckets cross-account replication configured
5. CloudFront distributions to point to the new S3 buckets
6. AMIs of EC2 instances and RDS snapshots transferred for a test run
7. In the case of encrypted volumes, custom keys setup and shared cross-account
8. DMS (Database Migration Service) configured for a few selected RDS instances
9. Maintenance window overnight scheduled together with Yoma Fleet team
10. During the maintenance window, EC2 and RDS migrated via AMI, snapshot copy, and DMS
11. KMS custom keys used to encrypt volumes
12. S3 replication completed, for static websites old bucket deleted and replicated to a new bucket with the same name in the destination account
13. At the end of the maintenance window, config CloudFront, ACM, Route 53 for select domains
14. Share new configuration and endpoints with the customer, support to modify endpoints in applications, and test
15. Support setup of CloudWatch metrics, alarms, events; monitor results over several days
A special task completed separately after the rest of the migration was to move the QuickSight resources to the new AWS account. This required a considerable amount of custom scripting with the AWS SDK and command-line tools but was possible due to AWS supporting all QuickSight APIs in the CLI and SDK since mid-2020. Since the process is complex, it is detailed in another blog post.
Having migrated to their own separate AWS account, Yoma Fleet is able to control their own resources without the interference of other teams and business units, while the other business units receive the same benefit in the source AWS account. This frees up Yoma Fleet’s ability to easier reason about their own servers and services without confusion on other resources. Unintended conflicts between Yoma Fleet services and those of other business units’ can also be avoided when not sharing the same AWS account.
On Yoma SH organizational level, the clear separation of accounts simplifies the billing as Yoma Fleet entity AWS usage can now be regarded separately and there are no potential disputes about the allocation of the costs. The organization’s governance is strengthened by setting up the logical boundaries that satisfy everyone.