The longest-running children's television show in the US, the Sesame Street did not survive 50 years without resorting to new technologies.
The latest innovation from the Sesame Street Workshop is Sesame Street Yourself, a new iOS app that uses the new features of ARKit 3.
Sesame Street Yourself is now available through the Apple App Store and uses the Multiple Face Tracking feature to identify users and turn them into their favorite Sesame Street characters. The ad-free app also entertains children aged 3 to 5 years with games, singing and dancing.
] "We just thought [ARKit 3] was a fantastic, gigantic bag of tricks . [We wanted to see] What we [could do with] are the technology for fantastic immersive interactive experiences for children, "said Baz Palmer, the co-founder of Weyo (the developer of the game), in an interview with Next Reality.
Palmer and his team used the features of ARKit for both the rear camera and the front camera. For example, kids on the rear view camera can help Cookie Monster bake cookies. On the front camera you can become Ernie and try different outfits.
ARKit 3 facial tracking technology gives parents the opportunity to play with their children each face carrying a different drawing mask that tracks the person while moving in the camera view.
"Working with the sesame brands, you have these loveable characters and a truly clear demographic that will try to get involved," said Palmer. "It really focused on how we want to use the technology."
Weyo originally contained elements that also took advantage of the People Occlusion and Motion Capture capabilities of ARKit 3, but according to Palmer, the team excluded these features from the final release to make the app easier. However, they plan to offer these experiences later as add-ons for the app.
Since the app uses ARKit 3, it is limited to compatible iPhones and iPads, namely iPhone X S . , X S Max and X R the 2019 editions of the iPad Air and the iPad Mini, as well as the latest 11-inch and 12.9-inch iPad Pro models.
This is not the first outing of the Sesame Street workshop into augmented reality. In 2013, the organization incorporated the Vuforia AR platform in Big Bird's Words, an educational game that helped players find words in the real world using Voforia's computer vision. Vuforia also supplied the AR behind an AR game set prototype.
Sesame Street Yourself is the first known brand name that has introduced ARKit 3. The uniqueness of People Occlusion and Motion Capture may be the secret ingredient to getting app developers to create more innovative AR experiences.