During Google I/O keynote early this week, Generative AI took all the limelight. Even Android was sidelined this year. Most part of the keynote involved Google showcasing different generative AI capabilities that are coming to their products and services including Search, Gmail, Workspace and more. At Google I/O, Google also made some huge announcements related to its augmented reality platform, ARCore.
ARCore is the largest cross-device augmented reality platform. It is now available on 1.4 billion Android devices and some features are also available on select iOS devices. Google revealed several new groundbreaking ARCore Geospatial capabilities, including Streetscape Geometry API, Geospatial Depth API, and Scene Semantics API that will allow developers to build more immersive world-scale AR experiences.
Streetscape Geometry API for iOS and Android:
The new Streetscape Geometry API will allow developers to interact, visualize, and transform building geometry around the user. For example, developers can reskin buildings, can place a virtual asset on a building, by using a 3D mesh within a 100m radius of the user’s mobile device location. For example, a developer can design a game where users can bounce ball on the building wall.
 Rooftop anchors on Android and iOS:
ARCore already offers Geospatial anchors that allows developers to place stable geometry at exact locations using latitude, longitude, and altitude. The new rooftop anchors will allow developers to anchor digital content securely to building rooftops, respecting the building geometry and the height of buildings.
The new Geospatial depth combines real time depth data from users’ device with Streetscape Geometry data to generate a depth map of up to 65 meters. This will allow developers to build increasingly realistic geospatial experiences in the real world.
Depth is critical to enable more realistic occlusion or collision of virtual objects in the real world.
Scene Semantics API on Android:
The Scene Semantics API uses AI to offer a class label to every pixel in a scene. This will enable developers to create custom AR experiences based on the features in an area around your user. For example, you can create an app that will always display information only on the sidewalks. Or you can create an app that will place information on vehicles alone. For now, twelve class labels are available, including sky, building, tree, road, sidewalk, vehicle, person, water and more.
Google has also created a sample game to demonstrate the above ARCore capabilities. The Mega Golf game will use buildings in a city to bounce off and propel a golf ball towards a hole while avoiding 3D virtual obstacles.
Google also announced Geospatial Creator that will allow developers and creators to create realistic augmented reality experiences. You can read about it here.
