Abstract: Augmented Reality (AR) is a technology that has flown under most people’s radar as its cousin, virtual reality, has garnered a larger share of headlines. Catapulted into public attention through the overwhelming success of Pokémon Go, consumers have become aware of the existence of AR while Apple / Google have rushed to create tools for developers to make apps for the AR ecosystem. Within the consumer segment of AR, Apple can leverage its tightly controlled ecosystem and brand loyalty to quickly build a sizable installed base of AR-ready phones. However, within the enterprise segment, Google has quietly been piloting an updated version of the Google Glass to much success. As the consumer ecosystem continues to mature, the onus is now on content developers to create apps that take full advantage of AR capabilities to create valuable user experiences for consumers.
False Starts and Small Niches
Augmented reality, like its cousin virtual reality, is a concept that has been under development for decades. The first workable AR prototypes were available in the early 1990s, but the technology stayed under the radar for most companies and consumers until the ill-fated launch of Google Glass in 2013. The Glass presented a simpler, more limited version of an AR headset with voice activated controls but no way for the device to analyze the user’s environment and “augment” it. The Glass failed to break into mainstream use due to the stereotypical nerdy aesthetic it gave users and the awkwardness of talking to oneself to get the device to work, among other issues. Afterwards, AR took a backseat in the minds of most consumers, with niche devices like Microsoft’s HoloLens and the AR capabilities of Nintendo’s 3DS handheld system catching the attention of small pockets of consumers, but failing to generate broader mass market consumer appeal. On the startup front, Magic Leap, an AR company known as the “most secretive startup in the world”, also became one of the most highly-funded startups ever, raising over $1.8B of funding – however, it has yet to release or announce its first product.
AR’s Blast Off
In July 2016, the stature and consumer awareness of AR skyrocketed with the release of the Pokémon Go mobile game from Niantic. Premised on nostalgia for the late 90’s / early 2000’s cultural phenomenon, Pokémon Go allowed players to go around catching a piece of their childhood and included a simple version of AR. The app would use the smartphone’s camera to show a live feed of the player’s surroundings and render the Pokémon over the video, allowing the player to pretend the Pokémon was embedded within the environment. While rudimentary, this system was the first AR experience for many players and introduced mainstream consumers to the possibilities of AR. Pokémon Go’s impact as a global phenomenon was validated through 750 million global downloads [1] and 65 million[2] monthly active users. This runaway success led to a spike in the popularity and awareness of AR. (See Exhibit 1)
Exhibit 1 – Augmented Reality Timeline, 2013-Present
All Aboard the Hype Train
Seeking to capitalize on the surge in popularity, both Apple and Google have recently announced software development kits (SDKs) to help developers create their own AR apps for iOS and Android. Apple’s ARKit and Google’s ARCore are releasing in late 2017 and various developers have already demoed a wide array of uses, from gaming, a virtual moon landing and a furniture selection tool. The kits themselves are similar in their capabilities, with ARCore taking a majority of its functionality from Google’s now-defunct AR initiative, Project Tango. While there will be some differences in user experience and technical capabilities between the two, the battle for the AR market will be focused on the iOS vs. Android hardware ecosystem that supports the app ecosystem.
Familiar Faces, Familiar Foes
Google and Apple have been competing for share in the smartphone market for years, and both take divergent approaches in managing their ecosystems. In AR, Apple’s tightly controlled iOS ecosystem affords it a major advantage over the fragmented Android ecosystem in both hardware and software.
On the hardware front, while high-end Android and Apple phones share similar specs, the limited choices Apple presents consumers implies there is a much higher chance that a consumer decides to buy an iOS phone (iPhone 8+ / iPhone X) that is AR-ready when compared to the mishmash of Android OEMs, each offering different specifications. When launching ARCore, Google only announced 2 phones, the Pixel and Samsung Galaxy S8, which are compatible with ARCore. Additionally, the fragmented Android landscape will lead to disparate user experiences based on device, unlike the more homogenous consumer experience for Apple devices. From a developer perspective, it is much easier to create a desired experience across 2 devices compared to the dozens in the Android ecosystem, particularly when the experience is so dependent on the hardware of the phone (e.g., cameras, processors).
On the software front, Apple again enjoys an advantage through its controlled ecosystem. Apple’s ARKit requires iOS 11, while ARCore will require Android 7.0, or Nougat. In the iOS ecosystem, updates are controlled by Apple and will go out to consumers in an orderly fashion for all iOS devices, greatly increasing the potential for consumer adoption. It took Apple only 5 months to update nearly ~80% of their installed base to iOS 10 and ~90% within a year[3], which stands in stark contrast to Android, where only ~15% of Android devices have updated to 7.0 more than a year after release[4].
The gap in adoption is driven by the fact that Android OEMs all have their own OS based on Android, and update device software on their own timelines. In addition, OEMs have little incentive to update underperforming devices to the latest software, abandoning many devices a year or two into their lifecycle – over 20% of Android devices are running software that is over 3 years old[4].
Winning Through Installed Base
The hardware and software advantages that iOS enjoys enables it to create a massive installed base of AR- ready phones just through the launch of iPhone 8+ / iPhone X. While Google is working to get their AR software into a wider range of devices, adoption will fundamentally be limited by its OEM partners – Google cannot fully control the direction of their own ecosystem. Even if Google is able to catch up to Apple in terms of AR-ready devices, its ecosystem will remain fragmented, making it more difficult for developers to manage their customer base, whereas the consistency of iOS and iPhones render it much easier to maintain and update apps. Developers of AR apps will likely find it much more appealing to create apps for the iOS ecosystem vis-à-vis the Android ecosystem, allowing Apple to create a compelling library of AR content.
Build it and (hopefully) They Will Come
For all the advantages Apple enjoys for building its AR ecosystem, it may be fruitless if the content that is developed doesn’t catch on with consumers. While apps like Pokémon Go are popular, AR is not required to play and it is not a key game mechanic. Most AR apps are still in the demo phase, but those announced so far may seem more like technology proof-of-concepts than robust, useful apps to most consumers. A sampling of the apps include: identifying constellations, virtual moon landings, the ability to place various virtual characters into pictures, an overlay for MLB games and several AR-based games. These apps, while intriguing to consumers, are either targeted for niche use cases / audiences or can quickly lose their novelty. Even apps that are geared towards real-life applications of AR, such as IKEA’s furniture selector, are limited to narrow / infrequent use cases. There has yet to be a true “killer app” for the AR ecosystem to get consumers truly on board the “AR train”. Without this app, AR may remain relegated as a “cool feature” of high-end phones instead of an essential value driver within the smartphone ecosystem.
It’s all about the Context
The first “killer app” for AR will likely be one that is able to fully analyze the context of the user’s surroundings, and overlay relevant information (think Terminator-style scanning). Some apps can already do specialized versions of this, such as allowing a diner to see a virtual image of a dish before buying or the aforementioned IKEA app – however, these use cases are likely too narrow to drive sustained interest. Instead, imagine holding up your phone to a street full of restaurants, then seeing information such as reviews, pricing, menu, or friends who have visited the place before. Some industry participants believe this type of application that can combine useful information with a social aspect will truly launch AR into the mainstream. Incumbent social media players like Facebook, Yelp, and Twitter have an advantage in developing such an app, as consumers already turn to them for similar uses cases.
AR Headsets / Glasses
Similar to VR headsets, AR headsets / glasses are a standalone product that can run AR apps without the need for a phone, akin to Google Glass. These headsets / glasses are under development by many hardware players, such as Amazon, Sony, Lenovo and potentially Apple. Google has even jumped back into the fray with an updated version of the Google Glass, with a focus on enterprise applications. These AR products are focused on analyzing the user’s environment and “augmenting” it through overlays and contextual information. AR headsets / glasses with an AI Assistant such as Amazon’s Alexa or Apple’s Siri could be a powerful combination to provide consumers with individualized, environment-driven help. However, there are a number of roadblocks in achieving scale, starting with stronger image-recognition and processing technologies. The next challenge is providing enough processing power, latency, and battery life for extended use. Lastly, consumers have to be willing to actually wear the devices – the first iteration of Google Glass was mocked for looking like it came straight from a sci-fi movie, and the latest AR headsets haven’t fared much better – for all the potential benefits of an AR headset, aesthetics might keep these devices out of everyday use in the near term. Rather, these headsets might find a foothold in the enterprise / industrial space. (See Exhibit 2)
Exhibit 2 – Potential Adoption Scenarios for AR
AR and the Factory Floor
Google’s second foray into the AR headset space is focused on the workplace segment rather than the consumer segment and has already found success there – industrial players such as GE, Boeing and Volkswagen have all seen productivity gains from their AR headset pilot projects. In a factory setting, these headsets can provide real-time information to workers who could not access it before. Key information such as assembly instructions or the location of a part can be given seamlessly and hands-free to factory workers. As the technology continues to be developed, a vision of a factory floor fully connected through IIoT / Industry 4.0 and supplemented by AR glasses will develop – workers can be informed on exactly which tool and part to use at each point in the manufacturing process, reducing errors and increasing plant efficiency / output.
Beyond factory settings, AR glasses can help with training, providing new workers with constant guidance on the correct process order and tools to use without the need to dedicate an experienced worker to oversee training. In addition, as digitization of industrial products / equipment becomes more prevalent, repair workers may eventually be able to use AR glasses in conjunction with an IIoT-enabled piece of equipment to quickly diagnose and repair malfunctions or perform routine maintenance.
Pilot programs for these industrial applications have already been successful across a number of companies and verticals, with players such as AGCO, DHL, and GE all reporting productivity gains and expansion of the pilots into larger portions of their operations. A recent paper co-authored by GE in the Harvard Business Review pointed to productivity improvements of 32% across multiple companies and verticals, with some GE units seeing an almost 50% improvement in task completion times[5]. With the Glass and other AR glasses still in a pilot / initial deployment phase, productivity improvements may continue to increase as more advanced use cases are introduced and existing use cases are optimized. Other industrial settings, such as the O&G, defense and mining verticals can leverage many of the same applications to improve safety and efficiency within their operations.
Additional AR Use Cases
Outside of general consumer usage and factory-focused applications, there are several additional AR use cases that are under development:
- Medicine: Startups have already piloted medically-focused applications, such as the ability for a medical scribe or nurse to remotely monitor a medical examination, take notes and provide the doctor with contextual information as necessary.
The immediate availability of information can be expanded to include procedures such as X-Rays or MRIs, and eventually may find their way into operating rooms as a tool to help surgeons better visualize and perform critical operations. In a more connected hospital environment, AR glasses can help nurses keep track of patient needs as well as alert them to emergency situations, improving patient outcomes and reducing the mental burden on hospital staff. - Training / Education: Capabilities available in factory settings can be extended to a wide gamut of other situations. Educators can use AR to provide more interactive learning experiences for their students and subjects like anatomy and chemistry can be enhanced through visual aids. Museums have already started to offer AR-enhanced tours, overlaying additional information, putting virtual flesh onto a skeleton or bringing exhibits to life through animation. With further development of image-recognition capabilities, students may be able to scan their environment and be provided information on the world around them.
- Business / Sales: Businesses such as restaurants and IKEA are already using AR to give consumers preview of their products, and this practice will likely expand to any number of verticals, such as automotive, consumer durables and even things like window shopping. For example, a consumer may scan down an aisle at the grocery store and be able to see specials or things on their grocery list, and a window shopper can scan down a city block to see what kinds of shops are present or find more detailed information on a product in a store.
Not All Sunshine and Rainbows
While there is significant interest from both businesses and consumers in AR, there are roadblocks and pitfalls the industry needs to address before it achieves meaningful scale and usage:
- Battery Life and Processing Power: Analyzing and rendering AR images uses a significant amount of processing power, particularly on smartphones. While high-end Android and Apple phones have the ability to handle these demands, a majority of smartphones today will not be able to unlock the full potential of AR, limiting it instead to a Pokémon Go- like AR-lite experience.
AR is also a very battery-intensive feature – after the launch of Pokémon Go, retailers saw a sudden surge in demand for external battery packs as players struggled with the battery drain the app inflicted on their phones; more intensive applications will only exacerbate the problem. - Avoiding the Gimmicks: Currently, many of the apps for consumers are more akin to tech demos than fully-fleshed applications; while this may be good for drumming up consumer interest and awareness in the technology, future apps will require depth and actual value for the consumer in order for the technology to see true widespread adoption.
- Underwhelming User Interfaces / Experiences: AR is a field that has just started to develop, but consumers are accessing these capabilities on very polished devices. Many consumers will expect a polished user experience out of the box – apps that make it confusing or frustrating for the user to activate AR capabilities may not be given a very long leash.
- Privacy Concerns: Consumers are becoming increasingly worried about the security of their data in light of recent events such as the Equifax breach – developers and ecosystem partners will need to clarify to users the extent of data capture / storage. This is especially important since AR can collect even more data on a user vis-à-vis cameras and photos.
Looking Forward
The AR ecosystem has generated a significant amount of momentum and interest through the launch of Pokémon Go and subsequent support by Apple / Google. This next generation of phones will be the first to include AR support as a key feature and developers are working to build out the content needed to drive consumer adoption and a healthy ecosystem. On the enterprise side, AR glasses / applications are gaining traction in a number of verticals as companies start seeing the operational benefits of AR. Some issues to monitor:
iPhone Sales / ARCore Integrations – Sales of the upcoming iPhone 8/8+/X are going to be critical for Apple to build a strong installed base of phones that have been designed with AR in mind. On the Android front, more rapid development of Nougat-based OS and integration of ARCore will be important for Android to leverage the installed base advantage it has in the overall smartphone ecosystem.
Where’s the Killer App? – Like the VR space, the hardware aspect of AR has outpaced the content in some aspects; strong adoption of AR will be driven by non-gimmicky apps that help augment / change how consumers get information and make decisions – while apps like games can be interesting, it may be difficult for developers to find a steady audience.
AR in Business – AR has already been piloted in some of the largest industrial manufacturers and has proven itself as a valuable tool for companies to optimize their operations. As AR moves from pilots to more widespread adoption within enterprises, players like Lenovo or Apple may introduce AR products targeted towards industrial use cases. Additionally, an ecosystem of developers and system integrators has formed and will continue to grow as companies look for partners to help create and deploy AR solutions.
New Entrants – Other players, such as China’s WeChat have also been rumored to be developing their own AR platforms – in an ecosystem that is bifurcated so cleanly between iOS and Android, an agonistic, 3rd party AR platform might find a place as a bridge between the two ecosystems.
Sources:
[1] [2] Niantic, VentureBeat, Forbes, as of April 2017
[3] Apple, as of September 6, 2017
[4] Google, as of September 11, 2017
[5] GE, Harvard Business Review March 2017
