Multi-Tenancy Testing: What it is & How to Perform It

In the context of software development, the term “multi-tenancy” refers to an application or service’s ability to support multiple customers, or “tenants.” Each tenant works with a unique set of data sets and configurations on a single software instance. 

Generally, multi-tenant architecture is utilized when different customers or companies use the same infrastructure and computing resources, such as hardware, VMs, or storage. These tenants are usually businesses, organisations, or individual users who share the same underlying infrastructure or application. However, their data and configurations are completely isolated from one another for optimal privacy and security. Each tenant can configure the application as they need while sharing a single instance. 

Needless to say, testing such applications comes with its own set of protocols and challenges—the focus of this article.  

What is Multi-Tenancy Testing?

Multi-tenancy testing covers the entire process of testing the functions and performance of software running on a multi-tenant architecture. This form of software testing ensures that the software in question can support all required tenants without compromising system stability and performance. 

At a high level, this form of testing checks that each tenant’s data is completely segregated from others. It also verifies the system’s ability to handle the magnified workload and resource demands that come with supporting different tenants.

Tests also examine the validity of the software’s security and privacy mechanisms. At no point should one tenant’s data be remotely accessible to others, which is also a necessity for regulatory and compliance regulations in many regions/countries. 

Components of a Multi-Tenancy System

Before testing a multi-tenant system, it’s important to understand what it actually entails:

• The application: The multi-tenant system itself is a software application. It comprises host software with multiple tenants, running on a single body of code. Each tenant, however, must have independent access privileges, role validation, and application data.
  
• The infrastructure: This is the underlying hardware that supports the entire software system. It comprises backup mechanisms, storage space, logs, and transmission capabilities. 
  
• The network: This is the platform that will help isolate apps and data for multiple tenants. Networks must be tested over different connections to check security gaps, data flow, and encryption/decryption status.  

Multi-Tenancy Testing examples

Data Isolation Tests

Let’s say you have a multi-tenant application that stores sensitive business data (customer identities, financial credentials, etc.) for different tenants. You have to test that any one tenant cannot access or modify data belonging to any other tenant. 

Test Steps:

• Create multiple tenants (let’s say Tenant 1 and Tenant 2) using the same base architecture. 
• Create and upload data for each tenant. 
• Run API and/or database queries from Tenant 1 to check that it cannot access data from the other in any instance. 
• Do this for every tenant you’ve created, and monitor if the system denies access when you try to access unauthorized data. 
• If one tenant can, for any reason, view or modify data that does not belong to them, there is a foundation glitch in the system. It is unusable and cannot be released until this error is resolved. 

Customization and Configuration Testing

In this case, test if each tenant can modify their UI, system settings, and features like enabling or disabling any modules they choose. 

Test Steps:

• Configure unique themes, such as diverse color schemes and logos, for Tenant 1 and 2 (more, if you need it).
• Enable a specific model (such as reporting) for Tenant 2, but not Tenant 1. 
• Log in as Tenant 2 and verify if the customised module and settings are working as expected. 
• Log in as Tenant 1 and check that Tenant 2’s customized modules are not available or accessible. 
• Each tenant should have an isolated, wholly unique UI and feature configurations that do not impact any other tenant in any way. 

Performance Testing under Heavy Traffic/Load

In this scenario, tenants have different usage loads and often have to handle heavy traffic and workloads. The system should be able to manage such loads without compromising the performance of any other tenant. 

Test Steps: 

• Simulate heavy traffic and workload on Tenant 1. This can be a high API request rate or any other user activity. 
• Track system performance for Tenant 1 and 2 concurrently. 
• Check if Tenant 2’s performance is not adversely impacted by the high-load operations of Tenant 1. 
• If there is no effect on the operations of Tenant 2, your system is working as expected. If not, debugging is required. 

Performance Testing under Heavy Traffic/Load

In this scenario, tenants have different usage loads and often have to handle heavy traffic and workloads. The system should be able to manage such loads without compromising the performance of any other tenant. 

Test Steps: 

– Simulate heavy traffic and workload on Tenant 1. This can be a high API request rate or any other user activity. 

– Track system performance for Tenants 1 and 2 concurrently. 

– Check if Tenant 2’s performance is not adversely impacted by the high-load operations of Tenant 1. 

– If there is no effect on the operations of Tenant 2, your system is working as expected. If not, debugging is required. 

Security and Authentication Testing

Your application should ensure that all authentication and access control features are accurately operational for all tenants. Security breaches in one tenant should have no impact on the others.

Test Steps: 

– Run user authentication tests for multiple tenants, each with different credentials. 

– Test the system by authenticating a user from Tenant 1 and Tenant 4—again, with different credentials. 

– Run penetration tests and scan for vulnerabilities to check if any loopholes exist to allow cross-tenant access. 

– Each tenant should have completely isolated authentication and authorization mechanisms, which are inaccessible to every other tenant using the system. 

Scalability Testing

Can the system handle more tenants being added? Can it scale appropriately to handle increased workloads and resource requirements?

Test Steps:

– Create a large (say, 500-800) number of tenants and add them to the system. 

– Simulate regular workloads—coming from data entry, user actions, report generation—across all these tenants. 

– Monitor system performance, resource usage metrics, and response times with every increase in the number of tenants. 

– During the entire process, check for any system glitches, performance degradation, and failures as the number of tenants expands. 

Disaster Recovery Testing

If the system experiences any failure, data for every tenant should be recoverable and lossless. 

Test Steps:

– Simulate a system failure and/or database crash. 

– Restore system data from in-built backup. 

– Check if every tenant’s data has been restored correctly. No tenant’s data should be lost or corrupted with data points from any other tenant. 

Billing and Metering Testing

All tenants should be billed on their actual resource usage, such as the amount of storage used and the number of triggered API requests. 

Test Steps: 

• Replicate different usage levels for different tenants. 
• Verify that your billing system tracks usage separately for each tenant. 
• The system should generate separate bills. Check that these are accurate and mapped to resource consumption. 

Compliance and Auditing Testing

All tenants must be able to comply with regional regulations (GDPR, HIPAA, SOC2 TYPE 2) which require operational logs. Your system should provide the same. 

Test Steps: 

• Run actions like accessing, modifying, and deleting data for all tenants.
• Check that separate audit logs record all actions just undertaken for each tenant.
• Validate that each tenant has a specific log, and it contains detailed data for compliance checks. 

Why do Multi-Tenancy Testing?

At a high level, multi-tenancy tests ensure the robustness, security, and performance of applications that serve multiple tenants within a single instance.

• To ensure data isolation for every tenant. No one tenant should be able to access, view or modify other tenants’ data. This ensures data privacy and compliance with regulations like GDPR and HIPAA.
• To prevent data leaks, security gaps, and access issues between tenants. 
• To allow each tenant to have a completely unique experience with their own configurations. Bear in mind that each tenant requires different configurations, including features, themes, and modules. 
• To manage and validate the complexity of resource allocation, usage, and performance in a multi-tenancy architecture. 
• To ensure that the system can handle heavy loads from any tenants without degrading the experience of the others. This is only possible via fair and optimized resource distribution and system responsiveness. 
• To ensure that the underlying infrastructure can scale without issue as more tenants join or if the workloads and demands of current tenants increase. Growth for any tenants includes more features and resource requirements, and the multi-tenant system should be able to accommodate the same without much hassle. 

• To optimize the use of infrastructure and resources. The system should be able to manage and monitor resources so as to prevent hogging or wastage. 
• To guarantee efficient resource usage without overloading or crashing the system. 

• To check that all updates and maintenance activities (patches, bug fixes, new releases) can occur without causing any downtime, disrupting existing functions, adversely impacting tenants or causing data loss/leakage.  
• To check that the systems comply with regulations and requirements around data protection, auditing, traceability, and logging—for every single tenant. Systems must adhere to these regulations to be able to legally operate in certain geographies (like the EU).
•  To verify that data recovery mechanisms are working as expected. In the event of system failure, data recovery should occur automatically without any overlap or major information loss. This includes checking tenant-specific backups and restoration of essential functions. 
• To verify the functionality of usage-based billing models, in which each tenant is charged for their actual resource consumption (API calls, storage, etc.). Tenants should only pay fair for what they have actually used. 
• To ensure that each tenant has a consistently reliable, responsive, and tailored experience while using your multi-tenant architecture. 
• To verify that all tenants can be onboarded smoothly without any configuration errors and that the system can accommodate new users without degrading anyone else’s experience. 

How to perform Multi-Tenancy Tests

Gather and finalize requirements

What are the expectations from the multi-tenancy system/app you are creating? Decide on the number and type of tenants the system should support, as well as the technical requirements around data isolation, performance, and security. 

Design and activate the test environment

Your test environment should replicate the end users’ environments. Create multiple tenants, each with their own database and schema. Also, create a shared database with shared tables with tenant data and tenant identifiers in the same tables. 

Don’t forget to set up mechanisms for data isolation, custom configurations, and resource sharing. 

Plan the Required Tests

You already have a list of tests required to verify a multi-tenancy system. Plan scripts, processes, and timelines to run them. Pay particular attention to load tests, as they verify your system’s ability to manage multiple tenants without compromising on the confidentiality and integrity of any of them. 

Automate the repetitive tests—API validations, data isolation tests, and UI checks. Your CI pipeline should trigger multi-tenancy tests after every deployment. 

Analyze Results

Track results using metrics like response usage and response times. Performance issues, security gaps such as data leaks and scalability bottlenecks, and isolation issues between tenants are the most common bugs in this form of testing. 

Debug and Resolve Functional Anomalies

This is where the developers come in and rework buggy code, configurations, and infrastructure to address errors. Don’t forget to document all the debugging steps and tactics as part of the project’s high-level documentation for future reference. 

Deploy Software & Monitor Performance

Once you’ve released your multi-tenancy system for usage, continue to keep track of its performance with the right continuous monitoring tools. Once again, use performance metrics to evaluate application status and error recurrence in production. 

When introducing updates and patches, don’t forget to run regression tests to ensure that tenant isolation and operability remain intact and unaffected. 

When to run Multi-Tenancy Tests

• When the application being built is meant to support multiple tenants or users. 
• When the application being built is complex, layered, and provides multiple features across different verticals. 
• When the application being built is meant to be used by a large number of end users, resulting in heavy traffic and activity load. 

Best Practices for Multi-Tenancy Testing

• If any one tenant consumes too many resources. The integrity of other tenants will be compromised. Even if all tenants access equal resources, some might engage in aggressive usage, leading to depleted computing power early in their lifecycle. Ensure that resource optimization works for every tenant, under every possible condition.
  
• Test your system’s security capabilities and load bearing capacity periodically after deployment, especially after new tenants are onboarded.
  
• To enable customization capabilities for all tenants, test the configurations for the layout style and the general configuration (the mechanisms used for setting encryption keys and personal credentials).
  
• Prioritize the implementation of data isolation strategies. It has to be built into the underlying architectural design at both functional and non-functional levels. Be it security, design, administration, availability, or performance, no tenant should be able to access anyone’s data but their own.
  
• Use both manual and automated penetration tests to check if any tenant can access others’ data via their UIs or API calls.
  
• Lean into automated testing for repetitive tests like data isolation, performance checks, and tenant-specific customizations.
  
• Test each tenant’s custom configurations (UI themes, language preferences, feature access) separately, with fresh data.
  
• When running security tests, cover penetration tests, authentication token checks, role-based access control (RBAC) validations, and encryption tests.
  
• Run tests with the intention that the system must support an increasing number of tenants, and offer them the same level of high-quality experience.
  
• Test in a staging environment that is as close as possible to the production environment. In other words, run your multi-tenancy tests on real browsers, devices, and platforms. 

Conclusion

Multi-tenancy tests are multifaceted and reasonably complex. They have to be in order to handle entirely different datasets, configurations, features, and customizations. Any platform that can test such a system should be robust and capable of replicating end-user conditions in the real world. 

In this blog, we tried to simplify the various ways that multi-tenancy testing can be conducted.