<img src = "https://img.reality.news/img/71/54/63671758488636/0/turn-sky-into-shared-canvas-with-fun-augmented-reality-app.w1
456. jpg "alt =" Turn the sky into a common canvas with this fun augmented reality app  image by laxwolf / YouTube
However, Blue Sky Paint uses a combination of the existing features of iPhones and ARKit, as well as some machine learning features, To identify the sky, he only draws the sky, lines disappear behind trees, buildings and other visible obstacles
According to lead creator Aidan Wolf, the app uses ARKits camera feed to capture the environment and a machine learning algorithm to determine For what is not heaven, the app uses graphical shaders as a mask to hide the user input from view and give the illusion of occlusion user and to determine the built-in compass of the iPhone for directional orientation.
"Basically, it's really general things that apply most of the time across the world: The sky is always so high. If your phone is pointing up and is probably looking at the sky, the sky is generally blue (even the clouds ), while the ground is usually illuminated directly by the sun, giving it a yellow hue, "said Wolf Next Reality. "GPS and compass orientation are used, but we have a re-projection algorithm that redraws the drawing to accommodate different perspectives and distances, and the fun in the sky is that we have a lot of GPS distortion since the sky is 2.5 km off the ground. "
The developers also use the orientation as part of the user interface. Looking down into the app brings up the color palette (another distinguishing feature between the app and Just a Line).
image by laxwolf / YouTube
The dataset for placing AR content allows the app to also provide multi-user experience and permanent content out- of-the-box, far ahead of the similar features of ARKit 2.0 later this month or other mobile AR cloud platforms like 6D.ai and Niantic Real World. This persistence allows users to collaborate on creations, share them with others, and discover drawings left behind by others.
"The app is essentially social, although it may not be obvious at first, they share the sky with everyone around you, so your drawings will be visible to other users in real time," Wolf said. "But as soon as there's a good AR cloud option, we want to bring the drawings down & let's draw buildings and so on, so it could be an AR cloud, but not that kind of cloud!"
image by laxwolf / YouTube
The machine learning and sensor shortcuts to occlusion, persistence and multi-user functionality give the app some inconsistencies. For example, the app may confuse a white building with the sky. In addition, tracking may be inconsistent, as content may be different from the original location.
Nonetheless, the app is an impressive performance that provides a creative approach to the same issues that attack other AR toolkits and AR Cloud platforms with more technically advanced attacks