Currently, the Internet of Things (IoT) is still in the hype phase where focus of customers is on IoT development. On the flip side, customers are worried about liability and risk involved in IoT projects. Considering this, it becomes important to talk about the assurance service line for IoT projects. There are many contributing factors which risk successful delivery of IoT projects; correspondingly, there are multiple assurance dimensions. In this document, we will focus on IoT testing specifically.
According to Statistics MRC, the global IoT testing market is estimated at $302.9 million in 2016 and is expected to reach $1776.52 million by 2022, growing at a CAGR of 34.2% from 2016 to 2022. Internet of Things development could turn out to be the greatest software development and testing opportunity of all time.
Key Characteristics of IoT Testing:
Comparing ‘regular’ testing to IoT testing, the latter is typically way more complex. While general testing practices apply here as well, a few key characteristics of IoT solutions testing include:
- A complex ecosystem
- Involves multiple layers of architecture – hardware, connectivity, gateway, data movement, cloud, analytics, user applications. Understandably, this demands rich breadth and depth of expertise
- High data volume, variety, and velocity
- Infrastructure requirements for ‘at-scale’ deployments
- Greater emphasis on security as in IoT scenario the damage is caused not only to the vulnerable device but potentially across devices, systems, and ecosystems
- Multiple end-point device types which will consume IoT services, making user experience testing crucial
- Growing need for testing interactions/interoperability as devices could be native IoT or brownfield IoTized (i.e., made IoT-enabled post facto)
- Network/Connectivity testing for cloud or edge computing IoT scenarios. Testing needs to check proper data propagation when connection is available as well as a no loss scenario when data is sync up post-connection restoration.
Today, many traditional business models are morphing into a more ‘as-a-service’ model. In some ways, for a product company, ‘service is the new product.’ This occurs naturally in case of IoT. Accordingly, our test offering must be nimble to adapt to ‘assured service release’ as an outcome.
IoT Testing Service Line:
Such a testing line would be modular yet provide end-to-end coverage for the testing and validation of IoT applications. A few of the main modules are:
- IoT assurance consulting
- Connectivity assurance for edge computing
- Data flow assurance for cloud computing scenarios
- Performance assurance using cyber physical model of actual devices and simulators
- Device and cloud security assurance
- Cloud migration assurance
- User Experience assurance across channels — @ nominal as well as @ Stress
Testing approach would include change-based, risk-based analysis testing, test-driven development, integrated test delivery, and service virtualization. It is crucial to have strong test automation framework with sequence of events and alerts being logged to facilitate traceability and debugging.
The service line can be made domain-specific. There will be use of optimal level of automation. These offerings could be 60-70% productized, remaining to be customized for each project.
Three Key Tenets of IoT System:
There are three major tenets in an IoT system. The testing approach can be targeted for them in a layered architecture — things, communication, and processing. Though many aspects permeate throughout architecture, security and performance are classic examples. This section provides some more granularity on the IoT testing services under these tenets:
- Things - Sensors
- Functionality testing
- Power consumption under dynamic conditions
- Environment testing
- Standards conformance
- Communication – Data/Gateway
- Connectivity – functionality as well as robustness under varying protocols
- EMI/Other noise interference Testing
- Load variations/performance Testing
- Data Recovery Audit
- Processing - Platform
- Data integrity
- UI/UX consistency across various computing platform display
- Integration testing
Key IoT Testing Tools/Assets:
- Device/Sensor simulators
- Data simulators
- Protocol simulators
- Test labs, jigs/fixture, instrumentation
- Data acquisition, analytics/ML, and visualization
- Template based checklists
The testing services projects have footprint in ‘market-released products,’ manufacturing floor, as well as new product development projects. Undoubtedly, maximum complexity is in ‘market-released products’ category.
For IoT solutions testing, customers need to be engaged with, since it has recurring customer touch point aspects built in by the nature of frequent IoT-enabled service release and upgrades. Upgrades are also quite involved as typically they extend beyond target device.
It may be important to introduce an alpha release (like in traditional product launch) even for pure testing engagements with provision for sandboxed input or feedback from restricted number of users in limited liability mode. The final release takes place after satisfactory reduction in issues factoring in ruggedization post alpha. This may be necessary for projects with regulatory implications.
IoT testing is considered a crucial aspect of delivering IoT projects (both greenfield or brownfield) successfully. Customers will expect IoT partners to clearly demonstrate how they are de-risking their delivery in light of dynamic nature of IoT and embracing specific measures to assure on-time, in-budget delivery. HCL IoT WoRKS has been investing in outcome and test-driven development, a solid IoT assurance framework in particular, to assure positive outcome in its IoT projects. You are encouraged to get in touch with the author for further queries.