Overview
Commonly referred to as OAT, Operational Acceptance Testing is the testing done before the solution is released or deployed, just after the execution of user acceptance testing (UAT). The OAT environment is called the ‘pilot’ or ‘pre-prod’ environment. This environment tests the system’s operational readiness, much like the user acceptance testing environment. OAT tests involve verification of procedures, such as performance, stress, volume, support processes, security, backup, and the existence of alerts. Typically, OAT occurs after user acceptance testing (UAT) and is a final verification before a system is released. OAT tests typically employ real users accessing and using the system in a live state. A typical operational acceptance test plan should focus on the recoverability, integrity, reliability, robustness, data integrity, manageability, and supportability of a software or network installation.
Definition
OAT is a common type of non-functional software testing, used mainly in software support and software maintenance projects after the execution of user acceptance testing (UAT). This type of testing focuses on the operational readiness of the system to be supported, or which is to become the production environment. Hence, it is also known as operational readiness testing (ORT).
Operational Acceptance Testing (OAT) provides the following assurances:
- The operation, operability and integrity of backup procedures, to ensure that the operating system and data can be restored successfully at the same site and also at other sites, if applicable. Recovery testing includes the build and configuration of a component.
- Implementation of a change in the production environment will be successful and not adversely affect existing production services. The implementation should be replicable using valid documentation that includes the time required for each step, and the order of implementation.
- Back-out of a failed change from the production environment will be successful and will not adversely affect existing production services.
- The system should be designed to offer transparent failover wherever possible.
- The system should automatically adjust itself to availability of system resources.
- If failover is invoked, failback can be performed successfully, and recovery to the original state is achievable.
- If several components have been affected by a failure, there should be a proven plan showing the recommended order of restart, time to complete, etc.
- The system can be shut down and restarted cleanly, without service disruption, or within an agreed window of scheduled downtime.
- Threshold Monitoring Alerts are in place and issued if agreed thresholds are exceeded. For e.g., disk utilization, CPU, memory, etc.
OAT addresses the concerns of various stakeholders involved in the application cycle. Some of the concerns addressed are:
- For an application owner, OAT confirms the possibility of performing changes to the application, with minimized risk.
- The operations team can gain satisfaction from the fact that there are proper operational manuals, adequate logging, alerts, and recovery procedures for the application.
- Network / system maintenance teams are assured that the application is safe for the hardware setup and has minimized risk for the shared environment.
