Toy cars controlled remotely by the Nintendo Switch will bring AR racing to your home’s floors in time for the holiday shopping season this year.
The Mario and Luigi karts for Mario Kart Live: Home Circuit each sell for $99.99 and are set to launch on Oct. 16. The toy features an on-board camera that streams back to a Switch allowing for a mixed reality experience. Players can race against their siblings– each with a physical kart — or against virtual racers overlaid onto the real world by the Switch. The physical Kart “responds to boosts in-game and in the real world, stops when hit with an item and can be affected in different ways depending on the race,” according to Nintendo. Nintendo even promises support for “play with up to four players in local multiplayer mode.”
Here’s a trailer announcing the new accessory today:
The product was part of announcements made in connection with Nintendo marking the 35th anniversary of Super Mario Bros. games. Nintendo’s Switch Lite and full-size Switch sell for around $200 and $300 respectively, and each is compatible with Mario Kart Live: Home Circuit.
Nintendo previously made the Labo VR Kit which explored VR-related games and ideas surrounding the Switch. The new AR-focused project looks like a really fun concept for Switch owners and we’re looking forward to getting our hands on the toy to see how it handles in the real world.
Are you interested in the new AR toy from Nintendo and planning to have Mario Kart races in your home? Let us know in the comments below.
Facebook researchers are investigating AR glasses featuring “hear-through” technology powered by specialized in-ear monitors for “enhanced hearing.”
The technology “would be able to recognize different types of events happening around you: people having conversations, the air conditioning noise, dishes and silverware clanking. Then using contextualized AI, your AR glasses would be able to make smart decisions, like removing the distracting background noise — and you’d be no more aware of the assistance than of a prescription improving your vision,” according to Facebook.
The work with beamforming, adaptive noise cancellation, and machine learning is described as “an area we’re just starting to explore” with the goal of enabling “perceptual superpowers” like “enhanced hearing.” One of the possible paths would use the “pattern of your head and eye movements” to “automatically enhance the sounds you want to hear and dim unwanted background noise.”
Safe Use Questions
Michael Abrash, chief scientist at Facebook Reality Labs Research, responded to questions in a briefing call with journalists related to how the use of this technology might affect social norms. Could a personal conversation at a restaurant table be heard by a nearby AR-enhanced patron? Would the nearby patrons ever know their conversation was heard by someone else? And would that conversation be less likely to happen in the first place if people knew mediated hearing was more common than it is at the time of this writing?
Abrash and Ravish Mehra, the audio team lead at FRL Research, gave a few examples of some potential mitigation strategies that might be employed to limit the range of this feature in the future. Mehra explained in a prepared statement they intend “to put guardrails around our innovation to do it responsibly, so we’re already thinking about potential safeguards we can put in place…for example, before I can enhance someone’s voice, there could be a protocol in place that my glasses can follow to ask someone else’s glasses for permission.”
Another line of research at Facebook may explore the use of personalizing audio with a head-related transfer function (HRTF) which accounts for the individual shape of your ears, potentially using an “algorithm that can approximate a workable personalized HRTF from something as simple as a photograph of their ears.” A Facebook representative added “the audio research team is considering several novel approaches to scaling the capture of people’s unique HRTFs.”
A Facebook blog post explains:
“Another issue the team is keenly aware of is the capture of sensitive ear data, both in the research phase and beyond. Today, before any data we collect is made available to researchers, it is encrypted and the research participant’s identity is separated from the data such that it is unknown to the researchers using the data. Once collected, it’s stored on secure internal servers that are accessible only to a small number of researchers with express permission to use it. The team also has regular reviews with privacy, security, and IT experts to make sure they’re following protocol and implementing the appropriate safeguards…”
AR vs. VR And In-Ear vs. Open-Ear
Facebook Reality Labs Research In-Ear Monitors (IEM) audio prototype.
Mehra explained in the call that different audio-based modifications might be accomplished in future AR or VR-based sound delivery systems depending whether they feature in-ear or open-ear designs.
An open-ear audio design like the Valve Index or Oculus Quest, for instance, wouldn’t necessarily work well for the cancellation of sounds from the physical world, the researchers explained.
In-ear designs like Apple’s AirPods Pro might end up being a better fit for some use cases. They already feature “Active Noise Cancellation” and “Transparency mode” to switch between “depending on how much of your surroundings you want to hear.” Another in-ear design, Google’s Pixel Buds 2, offers an experimental feature called “attention alerts” that would lower volume when they detect a baby crying, a dog barking, or an emergency vehicle siren.
Facebook researchers, meanwhile, are using “prototype in-ear monitors” or “IEMs” which they say “can deliver the full experience of auditory superpowers. This lets us enhance the right sounds for you and dim others, making sure that what you really want to hear is clear even in loud background noise.”
“Our IEMs also feature perceptually transparent hear-through,” Audio Experiences Lead Scott Selfon explained in a prepared statement, “making it sound like I have nothing in my ears, and letting me safely hear the entire world around me.”
When asked if this same research could be applied to dynamically lower the sound from a VR headset to better hear someone talking to you in the same physical room, Abrash said he hadn’t thought of that before but it’s a “great idea.”
Future Aims
Facebook aims to make “stylish AR glasses” and pitches the research as being “focused on transforming communication for everyone, everywhere” while citing Johns Hopkins research suggesting one in five people have hearing loss and many “don’t use hearing aids for a variety of reasons including expense, social stigma, discomfort, and lack of reliability.”
“I’ve been wearing hearing aids since I was a little girl,” Technical Program Manager Amanda Barry said, in a prepared statement. “The ability to help people stay connected with their families as they get older and their hearing fades — that’s really pretty exciting.”
Facebook is also researching what it calls LiveMaps (an updating map of the physical world “with shared and private components”) and Codec Avatars (hyper-realistic personalized representations of human bodies) and either or both might be used in conjunction with the audio research. Overall, Facebook researchers seem set on a path to “defy distance” by providing “true social presence.”
“The only reason we need for virtual sound to be made real is so that I can put a virtual person in front of me and have a social interaction with them that’s as if they were really there,” Facebook Research Lead Philip Robinson explained in a prepared statement. “And remote or in person, if we can improve communication even a little bit, it would really enable deeper and more impactful social connections.”
Facebook recently launched a beta for its Horizon social network and added a new requirement that all future Facebook VR headsets be connected to a Facebook account. Starting in October, Facebook will be implementing new terms of service which state that “when people stand behind their opinions and actions, our community is safer and more accountable. For this reason, you must: Use the same name that you use in everyday life” when you’re using Facebook.
Niantic announced in a tweet yesterday that Pokémon Go will no longer support certain old iPhone and Android devices after an update coming in October.
This includes the iPhone 5S and iPhone 6, as well as anything running iOS 10, iOS 11 or Android 5. While you might still be able to play for now, if you fall into one of those device categories then Pokémon Go will not be supported on your device after the October update releases.
The iPhone 5S and 6 released in 2013 and 2014 respectively, while iOS 10 and 11 launched in 2016 and 2017 and Android 5 in 2014.
In an upcoming update to Pokémon GO in October, we will end support for Android 5, iOS 10, and iOS 11, as well as iPhone 5s and iPhone 6 devices. Trainers with devices not specifically listed here will not be affected and don’t need to take any action.
This end-of-support announcement affects iPhone 5S and 6 users the hardest, as there’s no option besides buying a new phone. On the other hand, if you’re using a newer iPhone model running iOS 10 or 11, you can update your operating system to iOS 12 or above to continue to enjoy Pokémon Go after October. The same goes for Android 5 users, assuming your phone supports an update to Android 6 or higher.
While it’s always sad when a new app update ends support for devices that previously worked fine, it seems unlikely that this will affect a huge portion of the Pokémon Go player base. Nonetheless, it sucks if there’s even a few hardcore fans still enjoying the game on an iPhone 5S or 6, who will now have to purchase a new phone if they want to keep playing.
Pokémon Go launched in 2016 and became a worldwide hit. The game has been updated with lots of improvements and new content since release, including the recent addition of AR occlusion for Android devices.
Felix & Paul Studios is developing its first augmented reality (AR) project, titled The Storyteller: The Seven Ravens.
The project will feature an all-new chapter continuing the story from anthology TV series The Storyteller, created by Jim Henson in the 1980s. The new AR experience will feature a storybook that comes to life in augmented reality, with an accompanying story narrated by Neil Gaiman.
Unrelated to AR, a new Storyteller TV series is also currently in the works, led by the Jim Henson Company and written and narrated by Gaiman. According to Felix & Paul, The Seven Ravens AR experience will focus on a young girl who “sets out on an adventure to save her seven brothers from a curse that turned them into ravens.” The AR book will allow users to “interact with, explore, and re-explore the detailed and robust world of The Storyteller through the lens of The Seven Ravens tale.”
The project is co-produced by the Jim Henson Company and Felix & Paul, alongside funding from the Canadian Media Fund, the Magic Leap Independent Creators Program, Epic MegaGrants, and SODEC Quebec.
A preview of the project was shown last week at this year’s SIGGRAPH conference, held virtually, however you can also view some footage in the tweet embedded above.
Felix & Paul are known to have put out fantastic immersive VR content over the last few years, and a portal for the studio’s content became available for Oculus Quest earlier this year. The Storyteller: The Seven Ravens will be the studio’s first foray into AR content.
“Apple buys smaller technology companies from time to time,” the company’s official acquisition confirmation statement says, “and we generally do not discuss our purpose or plans.” If you follow Apple, you’ve seen these words multiple times before — pretty much any time it buys a company that doesn’t have an existing customer base to reassure. That now happens once or twice a month, often remaining under the radar until someone stumbles across it or gets tipped off to an ambiguous “headed in a new direction” final post on a company’s website.
A demonstration of Spaces’ latest tech shows a cartoony teacher offering whiteboard presentations with accompanying lip and body synchronization — a gentle evolution of existing VR avatar technology. You could easily imagine the 3D model replaced with one of Apple’s current Memoji avatars, enabling an iPad- or iPhone-toting teacher to offer a presentation to a virtual class over Zoom. That’s basically the VR video conferencing solution Spaces was offering prior to the acquisition, minus the Apple elements, and the platform-agnostic company promised compatibility with practically every major VR headset and video sharing app around.
I’m not going to tell you Apple’s VR and AR acquisitions have become too numerous to count, but the picture they paint is anything but narrowly focused. Every little company Apple buys feels like another tile in a massive mosaic, contributing its own color and texture to a picture that’s bigger than many people realize. If it seemed Apple was just making AR glasses two years ago, the company now appears to be developing both AR and VR hardware — including key components such as displays. Similarly, if you thought Apple’s AR ambitions were mostly about hardware, nope, it’s filing software patents and buying a lot of softwarecompanies. And services companies.
This won’t surprise anyone who knows that Apple’s core strength is its ability to integrate hardware, software, and services. But it does mean the company’s interest in mixed reality goes far beyond dropping a pair of glasses in the marketplace and seeing how they perform on their own. Apple is building the initial suite of AR/VR applications that will enable the hardware to succeed in its first or second generation, perhaps before there’s a robust “Reality App Store” with third-party apps. Like the classic iPhone apps Mail, Messages, and Safari, Spaces could be the key to Apple’s “Keynote VR” — or its development team may help with collaborative multi-person experiences in rooms, building on lessons learned from the Terminator offering.
Compare Apple’s approach to what we’ve seen with a couple of consumer VR and AR companies, Oculus and Nreal. Both announced hardware and largely let third-party developers loose to create cool games or useful apps that use their technology. Yet both companies (and other XR hardware makers) have realized that they, too, need to develop compelling apps to move their platforms forward. Some of the biggest current and upcoming Oculus titles have been either backed or developed by Facebook. Nreal is similarly collaborating with mobile operators to create game-changing AR apps. Neither waited until software and services were mature to launch hardware, decisions that (thankfully) gave early adopters tastes of our mixed reality future.
It’s been a comparatively long walk toward Apple’s product, with small public steps forward, odd leaks, contradictory reports, and the occasional bad decision. On their own, many of these moves don’t add obvious value to the Apple we know today. But collectively, they’re either going to come together for a massive iPhone-caliber launch or show up as an ever-growing collection of small developments, like the Apple Watch. The reported 2022 release might be getting closer every day, but if these acquisitions keep piling up, expectations for what’s about to arrive should be sky-high ahead of the reveal. Given Apple’s history with the iPhone and the Apple Watch, I don’t question whether the finished offering will have a huge impact, but rather how quickly the world will change as a result.
A new mobile The Witcher AR game called Monster Slayer developed by Spokko, part of the CD Projekt Red family, will bring the monsters of the dark fantasy universe to your neighborhood using Pokemon Go-style gameplay.
That’s right, you’ll soon be able to get your Geralt on and go out hunting for monsters in the streets around you house, catching them using an AR interface that places creatures in your surroundings using a live feed from your phone’s camera.
While the trailer, embedded below, shows a young man playing the game in a beautiful eastern European-looking forest, it’s not likely everyone will have such an apt setting to play the game in. I’m personally looking forward to catching some monsters from the Witcher in a dirty city alleyway.
Spokko says that “preparation is key” if you want to defeat the enemies in Monster Slayer, perhaps hinting at more depth in the mechanics of this title than others in the genre. You’ll have to use potions, oils, bombs, and bait if you want to succeed, just like in the mainline Witcher games.
The game announcement comes at a strange time — while Pokemon Go remains popular, none of the more recent games that followed it seem to have had the same success. Plus, for many people around the world, walking around their neighborhood is not a great idea, or even not allowed, at the moment due to the pandemic. That being said, a release date for the game hasn’t been announced, so that could be less of a concern in the future.
Following its consumer debut in South Korea this summer, Nreal is getting ready to expand availability of Nreal Light, its sunglasses-style AR headset, to Japan with the help of one the country’s largest telecoms, KDDI.
Update (November 10th, 2020): According to Mogura (Japanese), Nreal is preparing to launch Nreal Light to consumers in Japan, with pre-orders starting tomorrow, November 11th. The company says it will support Sony XPERIA 5 II and Samsung Galaxy Note 20 Ultra 5G, as well as future 5G-enabled devices.
Nreal Light is slated to cost ¥69,799 (~$663 USD) both through au Online Shop and KDDI brick-and-mortar stores throughout Japan, presumably offered there in some sort of hardware bundle.
Original Article (August 17th, 2020): As reported by The Verge, Nreal is now taking pre-orders from Korea-based consumers, available through Korean telecom LG Uplus. Light AR will be available in stores there starting August 21st.
When bundled with the Samsung Galaxy Note 20 or LG Velvet and a 5G data plan, Nreal Light AR comes to a pretty reasonable ₩349,500 (~$295). Without the smartphone/5G bundle, the headset is priced at ₩699,000 (~$590).
Image courtesy Nreal
Nreal Light launched pre-orders for its $1,200 developer kit late last year. The dev kit is driven by a tethered compute unit running a Snapdragon 845 mobile chipset, however the slim-line, sunglasses-style headset for consumers is supposed to rely on smartphones to drive the headset, tethered via a USB-C cable.
Like other AR headsets, such as HoloLens or Magic Leap One, Nreal Light AR includes 6DOF inside-out tracking, letting users walk around the room and interact with 3D content naturally. The consumer headset, which sports a relatively wide 52-degree diagonal field of view, comes with a variety of nose clips, a corrective lens insert, and an opaque shielding (dubbed “VR cover”) that is designed to block out light for a brighter, more immersive view. While hand-tracking isn’t available yet, Nreal says it will arrive at some point.
It’s unclear when other regions will see the consumer launch of Nreal Light AR, although Nreal may be planning its launch in stages to better gauge demand. Nreal initially hoped to launch its consumer version Light AR glasses in early 2020, however like many, the Beijing, China-based company was forced to delay production in February due to COVID-19 concerns.
A new HoloLens 2 application uses volumetric capture of a COVID-19 patient to help UK doctors and nurses safely identify symptoms.
HoloPatient: COVID-19 is a free extension of GIGXR’s HoloPatient platform, produced in partnership with the UK’s National Health Service (NHS) and the new Resilient XR consortium. The app presents users with a ‘standardized patient’ showing COVID-19 symptoms. The patient was captured using Dimension’s London-based Mixed Reality Capture studio, with features a rig with over 100 cameras that record humans ten stitch their actions back together as a 3D asset. The company says it shot the footage under safe conditions.
“Shoots are looking a little different at Dimension today, in the name of the health and safety of both the crew and the general public,” said Adam Smith, Head of Production at Dimension. “In line with advice from the APA and the Government, protocols include adherence to the 2m social distancing rule, strict use of PPE and regular temperature checks by a medic on set. To limit the number of people present, we have remote directing capabilities, streaming a live feed of the action from a wide selection of camera viewpoints. It’s not traditional but it’s efficient and hasn’t impacted quality in our experience.”
The app, meanwhile, takes users through four stages of COVID-19 illness, providing a safe means for doctors and nurses to recognize symptoms seen in a typical case.
Resilient is a group consisting of UK-based immersive technology companies such as Dimension, tech hub Digital Catapult and VISR VR. Also included are developer agencies like Fracture Reality and Make Real. It’s also supported by the University of Leeds’ Centre for Immersive Technologies and University College London. Along with this HoloLens app, the group has also been using volumetric capture to deliver online training videos in which viewers can zoom in and inspect processes in greater detail.
A new report from The Information says Foxconn Technology has moved into “trial production” on “semitransparent” lenses for Apple’s AR efforts.
The report indicates that Apple’s partner in iPhone production moved from the prototype stage to trial production of the critical component. Dueling reports in the past few months from Mark Gurman at Bloomberg and reporters for The Information have tried to zero in on Apple’s plans for VR and AR.
It is believed Apple is still some time away from shipping anything to compete with Facebook’s Oculus Quest — a VR headset which targets indoor use. Meanwhile, Apple’s CEO Tim Cook has made comments suggesting he sees huge opportunity in AR. The issue is that there are still enormous technical hurdles to overcome before anyone can ship lightweight AR glasses you use outside the home to insert digital objects into your view. Camera-laden VR headsets, of course, can also provide a passthrough way of seeing the environment around the person wearing the device and thus provide certain features through a kind of AR-in-VR mode. Windows MR headsets, Oculus Quest and Valve Index, for instance, all feature modes that let you view the environment on the other side of the otherwise opaque display.
Microsoft’s HoloLens, of course, is a relatively bulky but completely standalone AR headset and the company has encountered some issues with the display technology.
This is all to say that while The Information reports Apple and its Foxconn partner have been in development for three years with these lenses, it’s unclear exactly what kind of device they are meant for and precisely how far away that device might be from shipping to consumers.
A Florida-based military training and technology company, ECS (Engineering & Computer Simulations) has been awarded a Phase II Small Business Innovation Research (SBIR) project to develop “haptics-based mixed reality training systems within the U.S. Army’s Synthetic Training Environment.”
The project will be developed in conjunction with the Defense Health Agency and includes a partnership with haptic glove company HaptX and the Mayo Clinic at the College of Medicine and Science in Florida.
“This type of haptics integration offers a sense of touch and natural interactions within various virtual, augmented, and mixed reality scenarios,” said Waymon Armstrong, ECS CEO and President. “When applied in a medical environment, this integration with TC3 will provide our Warfighters and healthcare professionals the tools that they need to improve their quality of training and retention to potentially save more lives.””
The HaptX Gloves will be integrated into several areas of medical training in the Department of Defense, which aim to provide haptic feedback in training environments, as to reduce the need for “live tissue training.” The programs will use both augmented and virtual reality systems, with multi-user capabilities and training analytics and assessment tools.
Joe Michaels, the Chief Revenue Officer of HaptX, says that the company’s technology is “uniquely suited” to work with the U.S. Army’s training programs, and feel that the the gloves will be able to “increase the effectiveness of mixed reality training for military and civilian healthcare professionals.”
