Robotic Automation
Nowadays, robotics are used everywhere, such as in industries, hospitals, testing, etc. Robotic automation reduces the cost of monotonous tasks and saves time. In a production environment, the robot is thus cheaper than a worker in China. A further aspect is that a robot cannot become ill, have children or go on strike, and is not entitled to annual leave. So the robot can be used as a replacement for humans. But actually speaking, only a robot can’t replace the human since it is interfaced with Computer vision. Computer Vision will take decisions like which action and where the robot want to perform.
Computer vision-based Automation Testing
Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images. It involves extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions. Understanding in this context means the transformation of visual images into descriptions of the world that make sense to thought processes. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory.
Currently, computer vision is used for assembly-line inspection to identify the quality of production, the gaps between the two assembly parts, and electrical connector placement, etc. Computer vision can be achieved by a combination of camera hardware and software of image processing algorithms:
- Object classification: What broad category of object is in this photograph?
- Object identification: Which type of a given object is in this photograph?
- Object verification: Is the object in the photograph?
- Object detection: Where are the objects in the photograph?
- Object landmark detection: What are the key points for the object in the photograph?
- Object segmentation: What pixels belong to the object in the image?
- Object recognition: What objects are in this photograph, and where are they?
Robotic-based Device Testing Automation
Device testing functions assure the quality of devices, such as mobile devices, display units, etc. Device testing functions are conducted on both hardware and software. It includes verification and validation of hardware devices and software applications. Robotic automation testing procedures are very useful since they can save a lot of time and human effort.
In the robotic automation testing, all the device front panel UI (User Interface) components, such as touch screen, push-button, tact switch, and knob rotation, are controlled by the robot. To do the same, initially, the robot must have the calibration to all these device UI components, and then it will start to control the same during automation. Calibration is nothing but teaching the robot about the UI components and their position by configuring the x,y,z coordinates of each UI component.
Calibration requires the device under test (DUT) to be fixed in a static position. This can be achieved by using some specially designed mechanical fixtures. By adjusting the mechanical fixtures, different variants of devices are fixed in the same fixture and automated.
Problems
- While changing the device variants in the fixture, mechanical misalignment may happen to the DUT. Even a small change in DUT alignment can generate faulty results and action, so the setup requires a recalibration.
- For each DUT variant, UI components are located in a different position in the device front panel, and the size can also vary. So, the setup requires the recalibration to automate the new variant of DUT.
- Recalibration affects the continuity of automation.
Solution
Using the computer vision object detection technique, the above calibration can be automated. This auto-calibration solution will increase the automation continuity and the setup requires the camera to be placed in front of DUT to enable the computer vision.
Object detection is a computer technology related to computer vision and image processing that deals with detecting instances of semantic objects of a certain class in digital images. Here, we can use object detection for detecting the objects and their location information within the images. This can be achieved by using neural network deep learning models like Convolutional Neural Networks (CNNs). Initially, this deep learning model requires training with all the objects, and it is named as different classes. Once the image is passed as an input to the object detection module, it will apply the convolution technic and the neural network to find the object’s availability and location coordinates.
In this auto calibration solution, the object detection module must be trained with all UI components across different variants of devices. Along with this device, automation must start with this auto calibration which will identify all the UI components and their coordinate’s location with respect to the device front panel, which can be used for further robotic automation without any human intervention.
Overheads:
Deep learning training may be required whenever a new UI component is added, and it will take more time to do the same. This can be managed by using the transfer learning technique.
The quality of the captured image will impact the output accuracy. This can be overcome by using image pre-processing techniques like image Gray scaling, Pixel brightness transformations/ Brightness corrections, Image Filtering and Segmentation etc. Object detection requires high computation power, and it will take more time to generate the output. This can be overcome by using the GPU-enabled PC.
Computer vision techniques involves acquisition, processing, and analysis of digital images. It enables to acquire reliable information from the real-world, which may be described as numeric or symbolic information. Decision based information, Context specific transformation of visual images are some of the examples. Symbolic information extraction uses models constructed using statistics, physics, geometry, and learning theory to understand the visual images.
Device testing is carried out to ensure the device quality. It tends to test the quality of hardware and software of the devices like mobile phones, display units etc by verification and validation process.
Object detection techniques fundamentally uses computer vision and image processing algorithms. It majorly deals with detection of the object instances with the given class in the acquired digital images.
