January 30, 2017


The convergence of AR, VR with IoT

Co-author:Srinivasa Samudrala

Augmented Reality, Virtual Reality and Internet of Things have been making great strides in both technology maturity and user adoption in the last couple years and will continue to do so. Tech Giants have invested heavily in building IoT platforms and OEMs have launched AR/VR headsets providing some very unique consumer experiences. One of the most recent and exciting ones being the Microsoft HoloLens which merges both AR and VR into mixed reality.

The obvious next steps involve the convergence of AR/VR with IoT. While IoT devices collect billions of real world data points for analysis and visualization, helping the end users visualize the analyzed results through the right contextual lens (pun intended) is where AR/VR technologies provide unparalleled benefits. The killer use case for AR/VR technologies is in conjunction with IoT devices and intelligent analytics. This will provide relevant insights in the field, contextually, showing only what the user needs to know while filtering away the irrelevant “noise”.

To start with, AR technology can help technicians to increase their productivity and make fewer mistakes with guided instructions, relevant to the current job and context. As monitoring tools, these devices can make the monitoring contextual, whether its in fleet management (tracking and monitoring items being shipped and checking the conditions of transit on arrival) health of heavy equipment (getting data on when the equipment was serviced and types of jobs they are suitable for), or crops (monitoring the health and potential yield of the farms). In each of these scenarios, it’s the IoT devices and the accompanying analytics that is providing wisdom gleaned out of billions of data points, in conjunction with the contextual visualization from AR/VR devices. Additionally, in the field, in scenarios like fire rescues, smart headsets should be able to scan the environment, provide information about the hot thermal spots, predict the movement of hot air/ fire/backdraft and hence aiding firemen in the life threatening situations.

In addition to using IoT and AR technologies in the field to improve the safety, productivity and efficiency of the people in the field, IoT and VR and AR technologies can also be used to effectively manage Smart Cities, Smart Communities/College Campuses, Smart Events, Smart Stadiums and Theme Parks. One area where AR and VR headsets can be used is for simulating command and control centers that make every place “Smart” using IoT. These headsets make is possible to have a full command and control center without having to invest in multiple huge displays and the icing on the cake is that these command and control centers would now be available anywhere, even in your living room. This “Theatre-mode” but for a command and control center, where by putting on an AR/VR headset the major of a city or the traffic planners can get a view of the “health” of the city and even focus on areas of interest, right from their office or living room. This opens up a world of possibilities outside of the simple visualizations. Also using Intelligence in IoT, devices can predict crime, accidently and other avoidable events, helping plan the city and traffic better. Using VR tech we can create simulations that help cities plan their facilities and emergency preparedness.

Note that we are focusing on the AR/VR technologies and not “headsets” per se. The key is to leverage the software technologies around VR/AR independent of the hardware engineering involved with headsets. Essentially, you could be looking at an assisted Augmented Reality analysis though a tablet, phone or even a watch without having to put on headsets. This is what the technology needs to assist, so that AR/VR can be used anywhere in a device agnostic manner.