Snap says augmented reality will be foundation for next major shift in technology

Snap, the company behind Snapchat, has revealed plans for a fully-fledged digital platform taking on not only Facebook but also Google and Apple.

The company is launching an app store, expanding its games platform and offering the facility for external developers to upload machine-learning models to build augmented reality experiences. It is allowing other apps to integrate its camera software for the first time, and incorporating businesses into its maps alongside users’ friends.

The bold moves reflect Snap’s growing confidence that Snapchat will remain the largest non-Facebook social network in the west. Despite a blip in growth in 2018, Snapchat has grown to 229 million daily users, outstripping Twitter’s 166 million but lagging behind Facebook-owned Instagram and Facebook itself.

Bobby Murphy, the company’s co-founder and chief technology officer, told the Guardian: “In terms of the long-term future, we believe very strongly in the idea that computing overlaid on to the world, and augmented reality (AR) and the camera in particular, will be the foundation for the next major shift in technology.

“So you’ll notice that in a lot of our announcements: AR and the camera is actually threading its way into many of the other things we’re doing. We’re at the early stages of seeing AR and the vision of the camera being the centre of computing coming together.”

The features announced by Snap at its annual developer summit, held virtually last week, are the early stage of that revolution. One series of tools, called Scan, let users identify plants, trees, and dogs by pointing their camera at them. A planned integration with Yuka, a dieting app, will offer a similar function for packaged foods.

Another new product lets developers build their own AI filters for cameras. Initially, the tool will probably be used to generate ever more inventive lenses for the company’s messaging product – examples already include a filter that turns a video into the style of Van Gogh’s Starry Night, and a simple hand-tracking tool that places stars at your fingertips.

But Murphy says the goal is that eventually creativity will expand to include utility. Scan is “probably our best representative for pushing into non-creative AR, more utility-based AR”, he said. “With AR specifically, what’s really fascinating is that it quite literally transforms the way you see the world through your camera. It has tremendous potential to change the way we see the world. To educate, inform, help people connect with one another.”

In Snap’s vision of the future, its camera platform replaces the home screen of a smartphone or the newsfeed of Facebook as the default starting point from where all other tasks begin. New AR technology is one way to achieve that, but another is getting more apps into the camera, and its camera in more apps.

The latter is achieved by CameraKit, which lets other applications replace their default camera with Snapchat’s. The idea is that there is mutual benefit: the app doesn’t have to build a fully featured camera function if it just wants to include the ability to take or send photos, while Snapchat’s camera platform becomes increasingly valuable to developers who might be on the fence about whether to build features for it.

For the former, Snap launched Minis, a feature that allows for micro-apps to be embedded within SnapChat, which can be opened without installation. Examples include the meditation app Headspace; an app for putting together a schedule for the Coachella music festival; and a flashcard app, Tembo.

If successful, the plan could lead to Snapchat becoming the closest the west has to a “super-app” like China’s WeChat, where a single app can be the platform for a whole host of interactions.

Or perhaps the company’s aim is higher still. “We are definitely interested in wearable AR,” Murphy said, “which would require a display on your face. That is, in our eyes, the best way to envision the future of AR.”

America faces an epic choice …

… in the coming year, and the results will define the country for a generation. These are perilous times. Over the last three years, much of what the Guardian holds dear has been threatened – democracy, civility, truth.

Science and reason are in a battle with conjecture and instinct to determine public policy in this time of a pandemic. Partisanship and economic interests are playing their part, too. Meanwhile, misinformation and falsehoods are routine. At a time like this, an independent news organisation that fights for data over dogma, and fact over fake, is not just optional. It is essential.

The Guardian has been significantly impacted by the pandemic. Like many other news organisations, we are facing an unprecedented collapse in advertising revenues. We rely to an ever greater extent on our readers, both for the moral force to continue doing journalism at a time like this and for the financial strength to facilitate that reporting.

We believe every one of us deserves equal access to fact-based news and analysis. We’ve decided to keep Guardian journalism free for all readers, regardless of where they live or what they can afford to pay. This is made possible thanks to the support we receive from readers across America in all 50 states.