Architectural design patterns for multi-tenancy on AWS

In the first part of this two-part blog series, we discussed the factors to be considered when designing a typical multi-tenant SaaS application. In this final part, we will look at the various architectural design patterns for multi-tenancy on AWS and assess their pros and cons.

The solution can range from a fully isolated tenant deployment to a completely shared tenancy SaaS model, assuming the application itself also supports and incorporates the multi-tenancy components. An ideal architecture depends on its potential impact on the interconnected components, with a multi-dimensional view on tenancy. To tackle the bull by its horns, AWS provides several cloud-enabled solutions, which in effect, can be utilized as the following SaaS deployment models:

1. Multi-tenancy isolation through the Governance Model

In this model, all the tenants can be attached to a master (root) organization account (OU), which manages all the accounts associated with it. Organizational accounts can provide separations of concerns on account level isolation. The AWS Organizations service helps in defining central configurations, resource sharing across accounts in the organization and account & group creation. Its policies can be automated to reflect business needs by using governance. Billing for all accounts can be simplified by setting up a single payment method for all the owner AWS accounts and cost reports can be ascertained for each account separately.

Figure 1 - Multi-tenancy using AWS Organizations

Pros:

• Resources can be shared across accounts with effective utilization of services.
• Consolidated billing allows the organization to have a full overview of the tenant’s expenses.
• OU level-isolation is the maximum level of isolation available. Thus, it should be used only for premium level customers by a SaaS provider.

Cons:

• As each root account will be bound by its OU limits, this model may not be very scalable.
• It is more difficult to set up and maintain as there can be shared services in place, and an impact on a shared service may affect all the tenants. However, individual services being used by each account will not be impacted.
• This model could become very expensive for economies of scale, including the applicable limits at the root level for maximum OUs.

2. Tenant isolation at the AWS account level

In this model, all the tenants will have their own specific AWS accounts, which will be isolated to an extent. In essence, as a multi-tenant SaaS solution, this can be treated as a managed solution on AWS. This model’s advantages are account-level tenant isolation ranging from resource isolation and billing to metrics and security. A marketplace solution that can be deployed on each AWS account is a good example of this model.

Figure 2 - AWS as a managed service for Multi-tenancy

Pros:

• Account-level isolation provides a complete separation of concerns, which is ideal for security.
• Due to the isolation, customization of the solution and its configuration is deployment-specific for the tenant.
• Data is completely isolated for each tenant and hence, it is a highly secure deployment.
• Billing is generated for each tenant individually, and tenants do not have any access to the each other’s billing information.

Cons:

• Volume discounts and resource sharing are not utilized effectively for each tenant, leading to higher costs.
• As the number of tenants increase, it becomes difficult to manage operations for each separate AWS account and this may become a management overhead.

3. Tenant isolation at the Amazon VPC layer

In this model, all tenant solution deployments are within the same AWS account. However, the level of separation is at the VPC layer. For every tenant deployment, there is a separate VPC, which creates a logical separation between tenants. It is easier to manage as the entire solution is deployed within a single account (rather than in a multi-account setup). Further, appropriate isolation via a VPC provides better economies of scale and improved utilization of Amazon EC2-reserved instances, as all pricing constructs are applicable on the same AWS account. The limitation of this model is that in consolidated billing, separate account billing by VPC cannot be ascertained.

Figure 3 - Multi-tenancy separation at the VPC boundary

Pros:

• Simplified management & deployment of the setup since it is within a single account and within a single VPC.
• This model allows for improved utilization of AWS-reserved instances as all reservations are within the same AWS account.

Cons:

• For each region, Amazon VPC limits will have to be monitored at the tenant level.
• There would be a complex interaction in case the VPCs require connectivity with an on-premise network. Therefore, managing such connectivity might become a challenge.

4. Tenant isolation at the Amazon VPC subnet layer

In this model, there is a single AWS account and a single VPC for all tenant deployments. The isolation is prevalent at the level of subnets. Each tenant owns a separate application or solution version, without any sharing across tenants. The advantage of this model is no requirement of VPC peering and simplification of VPN and AWS Direct Connect connectivity to a single on-premises site. This model’s limitation is that its NACL’s (Network Access Control Lists) and security groups need to be managed very carefully.

Figure 4 - Multi-tenancy at the subnet layer

Pros:

• All tenants can have their own public and private subnet, providing appropriate security for the resources deployed within the subnets.
• No requirement of creating VPC peering for intercommunications, thus, making it easier to manage resources within the VPC.
• Being a single account multi-subnet deployment, costs would be reduced by using shared services, wherever possible.

Cons:

• Tenant isolation at the subnet level is tedious to manage as there are many Network Access Control Lists (NACLs) and security groups.
• Amending a VPC setting such as DHCP option set affects all tenants equally, regardless of their isolated deployment.
• An on-premises connectivity requirement may become complex due to the large number of VPC’s that need to be connected.

5. Tenant isolation at the container layer

Containerization platforms such as Docker or Kubernetes use containers to isolate the application and data within its context.

By design, different workloads can share the same cluster, with some level of logical or physical isolation between them. In this model, all the deployments for tenant solution are in the same AWS account, but the level of separation is at the namespace level.

We can achieve tenant isolation for each tenant application by using namespaces. The SELinux runtime environment provides a separate namespace for every tenant deployment. This ensures a logical separation between tenants. The stack consists of Kubernetes operators as the runtime that manages the application and an application custom resource instance and a database custom resource instance.

AWS provides containerization through several services such as AWS ECS (Elastic Container Service), EKS (Elastic Kubernetes Service), and AWS Fargate. In the case of Kubernetes, multi-tenancy can be achieved by leveraging namespaces and Platform-as-Code approach to build a multi-tenant SaaS solution on a Kubernetes cluster. In the realm of physical isolation, multi-tenant PaaS solutions can provide the orchestration power of Kubernetes and the runtime isolation through namespaces to build a secure multi-tenant platform. Rancher, a multi-tenant Kubernetes as a service solution, is a good example of such a service. Rancher can be used to deploy multiple containers separated by namespaces, with groups of applications and database combined in a database-per-service design pattern.

Figure 5 - Multi-tenancy at the Container Layer

Pros:

• Since there is no theoretical limit to the number of namespaces that can be defined, this model is highly extensible for the organization.
• Application customization is specific to each tenant. It does not interfere with others as the data is isolated for each tenant.
• Amazon EKS provides an RBAC implementation that allows IAM users, thereby helping to gain control over the other layers of isolation.

Cons:

• With many namespaces, the complexity of the system may become substantially higher to manage.
• As a best practice, it is recommended to keep the master and worker nodes secure. This may be difficult with multiple tenants sharing the same worker nodes, which increases the attack vector surface area for hackers.
• If the network policies allow network communications between namespaces, traffic between namespaces can become a security loophole.

6. Tenant isolation at the data layer

At the database level, an application requiring a multi-tenancy architecture can adopt one of the following database architectures. In a typical 3-tier application having shared web and application layers, minute variations can exist at the database layer. The following figure shows possible multi-tenant architectures, assuming that existing applications may have to be re-architected in a tenant-aware fashion. AWS provides its portfolio of compute and database services for this purpose.

Figure 6 - Application & database-level multi-tenancy

a) Instance isolation: The first design involves complete isolation of the application’s compute instance as well as the associated database. The design will require the most compute and storage, and can have customized security policies, user interfaces, and database design. However, this may be much more costly and difficult to maintain. Further, depending on certain compliance and security requirements, co-hosting tenants with different security profiles may not be possible. AWS provides this by design, with its current web service offering.

b) Single instance, separate databases: Each tenant will have a different database for maximum isolation. To enable the application layers to pick the right database upon each tenant’s request, it would be required to maintain metadata in a separate store (such as Amazon DynamoDB) where the mapping of a tenant to its database is managed.

c) Single database, separate tables/schemas: Different database flavors have different constructs, but another possible deployment model could be that each tenant’s data resides in the same database. However, the data is tied to different schemas or tables to provide a level of isolation.

d) Single database, shared schema/tables: In this model, each tenant’s data is placed together. A unique tenant ID column separates data records for each tenant. Whenever a new tenant needs to be added to the system, a new tenant ID is generated, additional capacity is provisioned, and traffic routing to an existing or new stack gets initiated. Although difficult to architect, this is the most optimized architecture which results in economies of scale, higher resource usage, is easier to provision & maintain, and is a cheaper tenancy model with shared components. Its limitations are exposure and vulnerability to shared attack surfaces, lesser control over the environment and customizations, and that in the event of failures, all tenants can be affected.

Pros:

• By utilizing shared components (part d of the above architecture), the design provides the best economies of scale and hence, is the cheapest option to operate at scale.
• It is easier to manage, maintain, and rollback as there is a single deployment, whether in the single database separate schema or in the single database shared schema design.
• This usually involves a single VPC design; however, even if multiple VPC’s are deployed, they would be only be limited in number.

Cons:

• Since the application architecture needs to be multi-tenant aware, applications which do not have multi-tenancy may need to be re-architected.
• Security requirements may mandate an entirely separate implementation; in which case, it would be difficult to utilize this architecture style.

Conclusion

Multi-tenancy architecture allows organizations to efficiently serve customers from all strata, ranging from small and medium customers (whose scale may not require dedicated infrastructure) to large enterprises (that require access to the cloud’s virtually unlimited and indispensable resources). Shared software development and maintenance costs, reduce expenditure, leading to savings that are passed onto the customers.

As I have mentioned earlier, this topic is vast in nature and must touch all aspects of software development implementation from the perspective of SaaS service providers, SaaS enterprise consumers (B2B), and SaaS end users (B2C) and must be thoughtfully evolved. When it comes to a SaaS implementation, there is no magic wand.

Our proven SaaS transformation approach helps our clients adopt the correct SaaS implementation in the proper way. It is based on the specific Cloud Service Provider’s recommended best practices, combined with Nagarro’s expertise and experience from a wide range of SaaS projects. To know more about our SaaS transformation approach, explore our offerings.