Achieving code coverage is a task I have often heard my developers say to be labor intensive, especially in a complex legacy setup, for applications that utilize multiple code repositories owned by multiple teams.
Unit testing plays a vital role in ensuring that every functional unit of the source code works as intended. As part of unit testing, each unit is examined separately from the rest of the code, ensuring code coverage is achieved. Code coverage is a metric that developers use to measure the extent to which source code is executed. It indicates the proportion of the code base that has undergone testing, offering insights into verified versus untested segments.
Achieving high code coverage is not only essential to detect bugs early in the development phase but is also mandated by the likes of Microsoft Azure DevOps (ADO). ADO suggests aiming to achieve code coverage of 70% for pull requests. Traditionally, developers write unit test cases manually. Various tools, such as NUnit, Pytest, Boost Test, to name a few, assist in the structuring of these test cases. However, formulating each test case remains a time-intensive process and as codebases grow, the volume of inputs and outputs grows as well, which increases the time needed to achieve test coverage, possibly leaving edge but critical scenarios untested. Such challenges highlight the need for automated and efficient unit test case generation.
Generative AI (GenAI), a subset of artificial intelligence driven by Large Language Models (LLMs) powered tools emerge as valuable resources in this aspect. By streamlining the creation of unit test cases, GenAI has the potential to accelerate the development process and improve test quality allowing developers to concentrate on building rather than worrying about code coverage.
GenAI is capable of automatically generating new content based on trained data. It has emerged as a significant asset in software development. LLMs, such as OpenAI's ChatGPT, Microsoft's Copilot and Google's Gemini, have been trained on extensive datasets, allowing them to produce contextually relevant material, which includes code snippets, documentation and unit test cases.
By employing GenAI, developers can expedite the unit testing process. Instead of manually drafting each test case, developers can employ GenAI to create a spectrum of test scenarios, including those that might be overlooked otherwise. This not only conserves time but also enhances testing comprehensiveness, improving code coverage and lowering the chances of bugs within the codebase. Consequently, GenAI empowers developers to uphold high software quality standards while alleviating the burdens associated with manual test case development.
A well designed GenAI-based unit testing framework might be able to not only generate unit test cases, but also execute the generated unit test cases, product code coverage reports and iteratively generate additional unit test cases if code coverage is not achieved.
Significant work has already been done in this area by service providers who reported that their framework, written in Python and tested for source codes in Python and C/C++, was able to achieve a 60% reduction in time, effort and cost over the manual approach in automotive software testing. A startup has recently announced an AI agent that it claims can scan code, identify and fix bugs and write unit tests. Platforms that provide AI-powered unit test generation for Java codebases are using reinforcement learning to analyze the code base and generate human-readable, executable tests that cover a wide range of scenarios. HCLTech has a homegrown GenAI tool, HCLTech AI Force, which also allows this capability and is yet to be backed by real-world evidence.
In conclusion, the application of GenAI for generating unit test cases seems to have a huge potential, with a significant amount of work already done in the automation of software testing.
