AR/VR Briefing Paper

Business Brief

Augmented reality (AR) provides a live direct, or indirect view of the real-world with modified or “augmented” aspects. These computer generated modifications can be applied across multiple sensory modalities, including visual, audio, haptic, somasensory, and olfactory. AR technology provides an enhancement to the surrounding environment and is either constructive of destructive. This means that AR technology can add elements to the environment or it can mask elements through localised occlusion without blocking the entire real world view of the user. This is the distinction between AR and Virtual Reality (VR) which replaces the entire surrounding environment with a completely generated one by occluding the entire field of view of the user.

AR and VR technology provide several beneficial uses within the workplace. VR technology provides a new way for individuals to connect. In today’s market there is no need for employees of the same company to be situated in the same office or even in the same city when working on a task together. However when it comes to training employees the distance now becomes a disadvantage to the company as video conferences and phone calls have shown to be less effective. VR technology allows individuals to be trained in the same virtual room. Even when the training involves the use of complex and dangerous machinery this can be done using VR technology where the virtual items can be manipulated simulating real ones. This application is not just limited to the training of employees but can also be applied with regards to employee meetings. Individuals are able to connect in the same virtual room for a meeting as well as display virtual items to better articulate their points.

AR technology also provides users with simple user interfaces. Users can manage their data and perform tasks usually done on the computer using intuitive hand gestures. This increases productivity of employees who are not as comfortable with computers.

Technical Brief

AR and VR make use of several different technologies. Both use sensors to gain a mapping of the surrounding environment, however VR technology also makes use of sensors outside a headset to provide a more immersive experience. VR headsets like the HTC Vive and Oculus Rift track different body movement in order to allow the user to be able to look around and interact with virtual items. The image processing needs to be done at around a 60 frame per second rate for the user to be able to move their head naturally. Several companies currently working on VR technology are now also attempting to track eye movements within their headsets. This will allow the images displayed on screen to appear even more realistic as the image will automatically focus based on where the user is looking.

AR technology deploys a virtual image over real-world objects. The system will receive input from the camera or other input devices. There are generally three approaches which AR technology makes use of. Simultaneous Localization and Mapping (SLAM) is a set of algorithms tasked with solving complex localization and mapping problems. This technology localizes sensors with respect to their surroundings. It uses this data to map the structure of the environment. Recognition based or marker based AR is another approach to the design of AR systems. The camera of the device identifies visual markers using either, QR/2D code or natural feature tracking markers. When the device senses a marker it places the virtual image at that position. It uses these markers to determine orientation and position of the marker image. Lastly, location based AR is another technology that utilizes GPS, digital compass, velocity meter, and accelerometer as inputs to determine orientation and position of the virtual image. This is a popular approach to AR technology as smartphones already have the necessary infrastructure to provide all these inputs. Page Break

Industry Use

AR and VR technology has a wide variety of applications from both a commercial and business standpoint. AR and VR are already being used by several major corporation’s as marketing tools as well as a means to increase workplace productivity. The Virtual Desktop app capable of being used on the Oculus Rift and HTC Vive is designed to provide users with a virtual Windows Desktop. The interface can be interacted with much more efficiently.

Within the workplace, Google Translate running on a smartphone can be used in AR mode to translate speech and text. When using the camera on a smartphone the Google Translate application can process an image with text written in one language and translate it to another. Another sensory modality of AR is provided by Google in the form of natural language translation. The recently released Google Pixel bud allow the user to get near real time speech translation right in their ear.

Currently in the industry several vendors are leveraging AR and VR technology as a marketing tool as well as a visual aid for engineers and other employees. For example the global architecture firm IA Interior Architecture is using InsiteVR to build models of their designs. This also allows clients of the company to “tour” the designs before designs are finalized to give their input.

Ford is another company using VR technology to help its engineers when designing elements of their vehicles. Audi has taken this technology to the consumer allowing customers to view, configure, and customize certain elements of their vehicles.

Ikea is providing a smartphone appluication for potential customers to let them place furniture right in their home and visualise how it will fit in the real world to help them select the right product for their space.

Canadian Government Use

The Government of Canada can provide better services to citizens and other departments through the use of AR and VR technology. The government can provide a clearer vision of its planning and documentation. For example, when attempting to propose the building of certain infrastructure the project can actually be demonstrated virtually to citizens to give them a better insight on the infrastructure of the project. Policy and documentation can now become more interactive in the world of AR and VR technology.

The use of AR and VR technology can also aid and support public safety and emergency services. For example, if emergency vehicles are fitted with AR Head-Up windshield displays this could help with route navigation, allowing responders to find the quickest route and obtain traffic information in real-time. Location based AR technology could also allow first responders to gain insight on dangers and hazardous conditions around them.

The GC has invested intechnology developed by Vancouver-based software company NGRAIN. The software company has developed an interactive 3D AR platform for aerospace company Lockheed Martin. Lockheed Martin uses this interactive platform as a tool to perform maintenance on fighter jets like the F-35 and F-22. The company reports reduced maintenance time and less errors.

Implications for Departments

Shared Services Canada

Value proposition

AR and VR technology has already proven to help workers complete complex tasks, such as maintenance on fighter jets. If SSC were to provide AR technology to employees who perform maintenance in their data centers, this could greatly increase their ability to resolve issues quickly. VR technology could increase employee connectivity as meetings could be held in virtual conference rooms.

Challenges

The adoption of AR and VR systems present several challenges. With a lack of hardware capable of running AR based applications at the moment most AR apps are mobile apps. This is because a mobile device can track all the necessary information for a properly function application. The challenge is the design approach that must be taken. Location-based AR applications are currently the only type of AR application available for use on mobile phones. This is because Recognition-based and SLAM technology is much too computationally intensive to be done on a phone. Since Location-based software requires only GPS it is a viable approach, however it is still limited due to geomagnetic sensor noise. As the need for more complex AR and VR applications grows so will the computational demands which will not be able to be handled by smartphones alone. It would also mean that applications across all platforms would have to be designed with the ability to interface with AR and VR hardware.

The use of AR and VR in the form of wearable technology in the workplace could also provide a source of distraction. Where the technology can increase connectivity among employees and simplicity of user interfaces, it can also allow cause employees to lose focus when working using the AR and VR technology