Niantic & Digital Catapult Launch Programme to Accelerate Real-World Metaverse Creation

When Pokemon GO creator Niantic Labs launched its Lightship software development kit (SDK) for augmented reality (AR) developers in 2021, the company spoke of its desire to build a “real-world metaverse” rather than a purely virtual one. Today, in the next step down that road Niantic has announced a new accelerator programme in collaboration with UK tech agency Digital Catapult, looking for UK-based startups to help build immersive AR experiences.

Niantic Metaverse

Niantic and Digital Catapult are teaming up with Cartoon Network and internationally renowned choreographer Wayne McGregor on the programme. Successful applicants will utilise the Lightship ARDK to build AR experiences for these brands, exploring the definition of a real-world metaverse, the social interactions that take place and how to bring people closer together.

Three startups will be selected for the Niantic Lightship Augmented Reality Accelerator which Digital Catapult will run under its own FutureScope accelerator programme. Each startup will be awarded up to £100,000 GBP to create a socially engaging AR demo.

“This unique collaboration with Niantic and Digital Catapult allows us to dream bigger and venture further into our evolving metaverse(s), visioning a new kind of shared creative adventure that untethers the imagination,” said McGregor in a statement. “This project provides everybody with accessible tools to tap into their innate creativity, to connect playfully with others and to explore their bodies and spaces they inhabit in surprising, limitless ways.”

Wayne McGregor
Wayne McGregor. Photo credit: Pål Hansen

“This new accelerator program with Digital Catapult will allow startups to push the boundaries of possibility with technology, creating groundbreaking new experiences for leading brands – and we are proud to unleash the potential of our Lightship Augmented Reality Developer Kit as part of this, Kate Carlyle, Senior Marketing Manager, Niantic’s Lightship Platform added.

Details regarding when the programme will begin and what sort of timeframe the startups have to build these demos haven’t been released at present. But if you’re UK-based and wish to apply then email niantic@digicatapult.org.uk.

For continued updates on the latest immersive accelerators, keep reading gmw3.

Top 8 Uses for Augmented Reality

Augmented reality (AR) is a technology with a dizzying range of potential applications. And as new and more powerful AR hardware enters the market (such as Apple’s mooted glasses), we’re likely to see even more uses for AR. 

That’s not to say that AR, as it exists today, is any slouch, and to prove it we’re looking at eight of the best uses for augmented reality.

Virtual try-ons

The retail industry has been one of the most prominent embracers of AR technology over at least the past decade. Most of the industry’s biggest brands offer some form of the technology, which allows prospective buyers to see how a product would look on them without needing to physically try it on, usually utilising the ubiquitous phone camera to display the virtual elements in real-time.

Prominent virtual try-on examples include make-up from Maybelline, clothing from ASOS and Zeekit, and shoes from Vyking.

Vyking AR Shoes
Image Credit: Vyking

Gaming

Augmented Reality has found a natural home in the gaming industry, where it has powered some huge mobile game successes including Pokemon Go and Pikmin Bloom, both from developer Niantic.

Pokemon Go in particular was a smash hit, peaking at over 250 million players per month on the back of an experience that transported the gameplay of the popular Pokemon video game series to real-world locations. That built on work the developer had done in its previous game Ingress, which allowed players to use their mobile phones to interact with virtual portals appearing in real-world locations as part of its science fiction story.

Construction

AR is a key tool in the construction industry, from the design stage right through to the actual building process. For architecture, numerous tools exist to aid in the visualisation of spaces, such as The Wild, which allows designers to view 3D models in both virtual and augmented reality.

On the building side of the equation, AR has uses ranging from training workers on safety to progress capture and tracking functionality that directly compares real-world sites with virtual models in real-time to ensure they aren’t deviating.

VisualLive
Image credit: VisualLive

Surgery

The high-stakes field of surgery is being revolutionised by augmented reality technology which can overlay vital information onto a surgeon’s field of view as they work. Mixed reality headsets such as the Microsoft HoloLens 2 allow surgeons to operate on patients more effectively, blending the real world with projections of computed Tomography (CT), and Magnetic Resonance Imaging (MRI) scans of the patients.

Holographic representations of the area being operated on can also be observed in 3D before surgery takes place to ensure a surgeon has full familiarity of the area they are working on. To find out more about the role of AR in healthcare, read our article on the subject.

The tricky business of finding your way around busy spaces has been much improved with the help of AR, such as the Live View feature offered by Google Maps, which takes existing data from the map app and overlays it on the camera’s view of the real world with help from your phone’s GPS capabilities.

Individual locations have also explored using augmented reality to help guide visitors, such as Gatwick Airport, which installed navigational beacons that guide a passenger’s way back in 2018 – all accessed via a smartphone app.

Google Maps AR / Google Lens
Image credit: Google

Education

From a school setting to on-the-job training, AR can be used to help learners safely interact with materials they would otherwise not be able to gain access to, all while remaining in a familiar setting. Google debuted augmented reality search during the COVID-19 pandemic to help people learn by placing virtual objects such as spacesuits and animals into real-world locations. A host of apps exist to bring similar objects into a classroom setting, including the Merge Cube, which adds tactility to the experience.

Energy giants such as Shell, meanwhile, are using AR to educate workers in the field by bringing in experts who can see through a worker’s eyes and even draw on the screen of the augmented reality display they are using, boosting safety as they interact with potentially dangerous heavy oil and gas equipment.

Design

Designers at all levels are making use of AR to preview how a space will look before any changes are made physically, from those designing individual rooms all the way up to those planning cities.
Non-professionals too can make use of augmented reality to aid in their designs. Just one example is furniture store IKEA’s IKEA Place app which allows users to place 3D models of the company’s goods into their own rooms in order to preview how they would look, automatically scaling them based on the room’s dimensions to ensure they are true to life.

IKEA PLACE AR app
IKEA Place AR app. Image credit: Ikea

Manufacturing

AR is one of the key pillars underpinning the phenomenon of Industry 4.0, alongside such technologies as machine learning and big data. Consultants PwC has estimated that industrial manufacturing and design is one of the biggest potential areas for augmented and virtual reality, with their use in the industry having the potential to deliver a $360bn GDP boost by 2030.
As a result, examples of the technology in action for manufacturing are easy to come by. One example is Boeing’s use of augmented reality to give technicians real-time, hands-free, interactive 3D wiring diagrams. Lockheed Martin also utilised augmented reality in the creation of NASA’s Orion Spacecraft, overlaying information to help with mission-critical procedures such as precisely aligning fasteners.

Peridot is Niantic’s new Tamagotchi/Pokémon AR Mashup

Best known for its augmented reality (AR) titles Pokémon GO and more recently Pikmin Bloom, Niantic Labs hasn’t released its own original content since its very first mobile videogame Ingress Prime. That’s going to change soon with the announcement of Peridot, an AR mobile game where you breed and raise adorable little creatures called Peridots.

Peridot
Image credit: Niantic Labs

Unlike the aforementioned Pokémon GO, Peridot will be a far more caring AR experience that heavily leverages augmented reality technology. Your job is to save the Peridots from extinction by looking after these unique animals. Ensuring they make it all the way from birth through to adulthood, the core gameplay loop is very much like Tamagotchi, playing with, and training them to nurture and build a bond with them.

Of course, Peridot takes things to a more immersive level using AR, taking your Dot (for short) on daily walks to get their exercise in (and yours), discovering hidden treasures along the way and then breeding the next generation. Each Peridot has its own unique DNA which: “blends the best of hand-created assets and procedural generation, creating a stunning array of unique creatures,” says Niantic.

Breeding is only one small part of Peridot because as your creatures grow up they’ll want to do things. Niantic has built in a “desire system” in which your Dot will want to go out and explore, play, and eat. In doing so you’ll help it grow.

Peridot
Image credit: Niantic Labs

AR will be at the core of Peridot, making each Dot seem as real as possible. They’ll be able to recognise different real-world surfaces such as dirt, sand, water, grass, and foliage, digging into each surface to find different foods. All thanks to Niantic’s latest Lightship ARDK technology.

Much like Niantic Labs’ other videogame launches, Peridot will see soft launch beta testing start this month in select markets (none have been revealed just yet) with more added as the beta continues. For continued updates on Peridot, keep reading gmw3.

50 Creatives to Help Build UK-Wide XR Storytelling Project

The UK has a strong history in the XR space, with several new projects appearing at SXSW this week. Today, StoryFutures Academy in collaboration with several other organisations has announced plans to deliver an immersive storytelling experience called StoryTrails. Created by 50 artists, it’ll appear across various UK cities later this year.

nexus AR Mobile

As part of UNBOXED: Creativity in the UK – an initiative celebrating the country’s creativity – StoryTrails will use augmented (AR) and virtual reality (VR) in a bid to bring intriguing stories from the past to life. 15 public spaces will be reanimated thanks to the work of 50 creatives, the BFI (British Film Institute), the BBC, Niantic Labs, Nexus Studios, and ISO Design.

Those 50 roles will be split down into several projects, with 17 making story trails for city centres whilst another 15 will build immersive installations mapping the geography of a location. Nine will be selected to develop VR experiences whilst nine will take on professional placement at one of the StoryTrails partner organisations. The BFI, for example, will provide unprecedented access to archive material.

“StoryTrails will help to shape the future of the creative industries and pioneer creative storytelling in the 3D internet. This moment, when emerging technologies are finding their form is the same moment the UK’s next generation of creative practitioners are emerging. StoryTrails represents a direct engagement with changing the face of the industry to reflect the diversity of talent across the UK and to use that talent to make the UK a world leader in immersive storytelling,” said Professor David Olusoga, broadcaster and film-maker and Executive Producer of StoryTrails in a statement. “As the UK’s largest immersive storytelling project, StoryTrails will be mapping a course for this amazing set of new technologies that engrains diversity at its heart.”

Lewisham - StoryTrails

“StoryTrails is a wonderful project being developed for UNBOXED, which uses cutting edge immersive technologies to tell new stories about people and places across the UK,” adds Martin Green CBE, Chief Creative Officer UNBOXED: Creativity in the UK. “It’s brilliant that StoryFutures Academy are also recruiting and training 50 creatives from diverse backgrounds to help shape it and expand their skills in immersive technology. In doing so, they are playing a vital role in building a dynamic future workforce for the creative industries in the UK.”

The 15 locations offering free two-day StoryTrails residencies are:

  • Omagh
  • Dundee
  • Dumfries
  • Blackpool
  • Bradford
  • Sheffield
  • Lincoln
  • Wolverhampton
  • Swansea
  • Newport
  • Bristol
  • Swindon
  • Slough
  • London (Lambeth and Lewisham).

If you happen to be in the UK this summer StoryTrails runs from 1st July – 18th September 2022, culminating in a new film to be screened in cinemas as well as being made available via BBC iPlayer. For further updates, keep reading gmw3.

The VR Job Hub: VRDays Europe, Liquid City & Niantic Labs

Every weekend gmw3 gathers together vacancies from across the virtual reality (VR), augmented reality (AR) and mixed reality (MR) industries, in locations around the globe to help make finding that ideal job easier. Below is a selection of roles that are currently accepting applications across a number of disciplines, all within departments and companies that focus on immersive entertainment.

LocationCompanyRoleLink
The NetherlandsVRDays EuropeSales InternshipClick Here to Apply
The NetherlandsVRDays EuropeBusiness DeveloperClick Here to Apply
The NetherlandsVRDays EuropeJunior Sales RepresentativeClick Here to Apply
The NetherlandsVRDays EuropeMarketing ManagerClick Here to Apply
The NetherlandsVRDays EuropeHead of ProductionClick Here to Apply
London, UKLiquid CityXR PrototyperClick Here to Apply
San Francisco, CANiantic LabsUnity Technical ArtistClick Here to Apply
Seattle Area (Bellevue), WANiantic LabsUnity Technical ArtistClick Here to Apply
San Francisco, CANiantic LabsGame Design Manager, Pokémon GOClick Here to Apply
Seattle Area (Bellevue), WANiantic LabsGame Design Manager, Pokémon GOClick Here to Apply
San Francisco, CANiantic LabsContent Game Designer, Pokémon GOClick Here to Apply
Seattle Area (Bellevue), WANiantic LabsContent Game Designer, Pokémon GOClick Here to Apply
San Francisco, CANiantic LabsSenior Producer – Battle, Pokémon GOClick Here to Apply
Sunnyvale, CANiantic LabsSenior Producer – Battle, Pokémon GOClick Here to Apply
Los Angeles, CANiantic LabsSenior Producer – Battle, Pokémon GOClick Here to Apply
Seattle Area (Bellevue), WANiantic LabsSenior Producer – Battle, Pokémon GOClick Here to Apply
San Francisco, CA /Seattle Area (Bellevue), WANiantic LabsGame Design Manager, Pokémon GOClick Here to Apply
San Francisco, CA / Sunnyvale, CANiantic LabsTechnical Lead, Build Infrastructure, AR TeamClick Here to Apply

Don’t forget, if there wasn’t anything that took your fancy this week there’s always last week’s listings on The VR Job Hub to check as well.

If you are an employer looking for someone to fill an immersive technology related role – regardless of the industry – don’t forget you can send us the lowdown on the position and we’ll be sure to feature it in that following week’s feature. Details should be sent to Peter Graham (pgraham@vrfocus.com).

We’ll see you next week on gmw3 at the usual time of 3PM (UK) for another selection of jobs from around the world.

The VR Job Hub: VRDays Europe, Liquid City & Niantic Labs

Every weekend gmw3 gathers together vacancies from across the virtual reality (VR), augmented reality (AR) and mixed reality (MR) industries, in locations around the globe to help make finding that ideal job easier. Below is a selection of roles that are currently accepting applications across a number of disciplines, all within departments and companies that focus on immersive entertainment.

LocationCompanyRoleLink
The NetherlandsVRDays EuropeSales InternshipClick Here to Apply
The NetherlandsVRDays EuropeBusiness DeveloperClick Here to Apply
The NetherlandsVRDays EuropeJunior Sales RepresentativeClick Here to Apply
The NetherlandsVRDays EuropeMarketing ManagerClick Here to Apply
The NetherlandsVRDays EuropeHead of ProductionClick Here to Apply
London, UKLiquid CityXR PrototyperClick Here to Apply
San Francisco, CANiantic LabsUnity Technical ArtistClick Here to Apply
Seattle Area (Bellevue), WANiantic LabsUnity Technical ArtistClick Here to Apply
San Francisco, CANiantic LabsGame Design Manager, Pokémon GOClick Here to Apply
Seattle Area (Bellevue), WANiantic LabsGame Design Manager, Pokémon GOClick Here to Apply
San Francisco, CANiantic LabsContent Game Designer, Pokémon GOClick Here to Apply
Seattle Area (Bellevue), WANiantic LabsContent Game Designer, Pokémon GOClick Here to Apply
San Francisco, CANiantic LabsSenior Producer – Battle, Pokémon GOClick Here to Apply
Sunnyvale, CANiantic LabsSenior Producer – Battle, Pokémon GOClick Here to Apply
Los Angeles, CANiantic LabsSenior Producer – Battle, Pokémon GOClick Here to Apply
Seattle Area (Bellevue), WANiantic LabsSenior Producer – Battle, Pokémon GOClick Here to Apply
San Francisco, CA /Seattle Area (Bellevue), WANiantic LabsGame Design Manager, Pokémon GOClick Here to Apply
San Francisco, CA / Sunnyvale, CANiantic LabsTechnical Lead, Build Infrastructure, AR TeamClick Here to Apply

Don’t forget, if there wasn’t anything that took your fancy this week there’s always last week’s listings on The VR Job Hub to check as well.

If you are an employer looking for someone to fill an immersive technology related role – regardless of the industry – don’t forget you can send us the lowdown on the position and we’ll be sure to feature it in that following week’s feature. Details should be sent to Peter Graham (pgraham@vrfocus.com).

We’ll see you next week on gmw3 at the usual time of 3PM (UK) for another selection of jobs from around the world.

2021 in Review: Games, Experiences and Technology

2021 was a stand out year for XR. Both virtual reality (VR) and augmented reality (AR) saw some technological leaps, some great videogames and a glimpse of the future. Both technologies are shaking up several industries while laying the foundations for the metaverse. To celebrate the year, we’ve chosen some of our highlights as we look forward to what 2022 might bring.

After the Fall

Probably the biggest VR launch of 2021, After the Fall brings zombie-slaying back and makes it more fun than ever. With co-op modes, cross-platform play and constant action, it’s a title that feels perfectly at home in VR. The game is gorgeous – aside from the grisly zombies – and playing on high-end hardware ensures a smooth experience. The intuitive controls allow for a great experience, and while there are some minor flaws, After the Fall is set to be one of the biggest and best VR games in recent years.

After the Fall

Resident Evil 4

The best Resident Evil game finally has a VR option! Armature brought everything that made the game such a standout success and revolutionised it with plenty of accessible VR additions. We loved the new interactive features; cocking and reloading the weapons, the malleable storage system, pulling grenade pins, all of these bring the action to life like never before. While it’s not the prettiest game, thanks to the browns and beiges of the original game, it’s still a great spectacle to behold.

Ragnarock

VR often brings out the best in rhythm games, mostly due to the accelerometers embedded in the controllers. It gives a sense of interactivity that button presses can’t achieve. In Ragnarock, thumping away on the drums feels invigorating and refreshing. It helps that this Viking environment is backed by a soundtrack of rock and metal. Energising your boat of rowers, you bash out rhythms and melodies on small drums in the hopes of scoring well. Even when you don’t, it doesn’t feel like a chore replaying songs, because who doesn’t love bashing drums and creating a foot-tapping moment of bliss?

Ragnarock

Pikmin Bloom

Niantic Labs’ games always want us to go outside. They’re urging us to put down the mouse or controller and interact with life outside our four walls. Pikmin Bloom is its latest attempt to get us exercising and interacting with the natural world. It’s more about walking than Pokemon Go, as there’s very little need to stand around. Players must find seeds that hatch into cute Pikmin then nurture the relationship by walking, with the app counting steps. It’s a very sedate experience, it’s one that teams up with the nature around us offering a peaceful escape from our world.

The Climb 2

If there’s a better looking videogame in VR, we haven’t seen it. And we’ve played a lot of games! The Climb 2 is a stunning view, whether climbing snow-capped mountains or high rise skyscrapers. Stopping every few minutes to appreciate the scenery is a joy, and that’s no surprise given the game is running on the Crytek CryEngine. Perhaps better than the view is the feeling of adrenaline when climbing, leaping and saving yourself from a deathly fall. The game gives a light workout to your arms, but it’s entirely welcome. The dynamic objects which could spell disaster at any second keep your heart in your mouth and your fingertips gripping on for dear life. The Climb 2 sounds sedate on paper, but in (virtual) reality it’s a nerve-shredding experience!

The Climb 2

VR/AR Concerts

Sadly, in 2021 the global COVID-19 pandemic is still a thing. This means that artists, musicians and film studios are looking for new ways to interact with fans. VR and AR experiences are a booming business and a guaranteed path to extra revenue in a world where concerts are being cancelled or moved from date to date. Through VR apps like Oculus Venues and MelodyVR, you can still attend the gigs of your favourite stars. Megastars Billie Eilish, Lewis Capaldi and Khalid are leading the way, and the adoption of low-cost headsets will make these experiences even more common in a post-pandemic world.

Wizards Unite is Closing

Sometimes you can have too many eggs in a basket. Niantic Labs has seen massive success with their headline game Pokemon Go and their latest release, Pikmin Bloom. This has perhaps overshadowed Harry Potter Wizards Unite; it certainly didn’t help that players didn’t shift from pocket monsters to waving magic wands. Wizards Unite just wasn’t sustaining itself, making $39.4 million in lifetime revenue compared to the $1.1 billion from Pokemon Go in 2021 alone. Sometimes a smash hit brand just isn’t enough.

Harry Potter: Wizards Unite teaser

Haptic Feedback

As VR technology evolves, so too does the need and want for more haptic feedback. We’re beyond rumbling controllers and racing seats that thud and jerk along with a game. Companies like HaptX, Meta and Tesla are all investing heavily into technology that will encompass our entire bodies; gloves that mimic the pressure and weight of physical objects when in a digital world; bodysuits which can react to impacts or environmental changes in a metaverse space. Each of these companies showcased their tech in 2021 to the astonishment of pretty much everyone, for better or for worse.

Facebook rebrand

If you somehow missed Facebook rebranding to Meta, you must have been living under a rock! Mark Zuckerberg shook up the tech world by announcing his company Facebook would now be known as Meta. Why? Because he envisions the future of the internet as the metaverse, a term first coined in the novel Snow Crash by Neal Stephenson. This future, according to Zuckerberg, will be an extension of our physical selves into the digital landscape of web 3.0, through VR and AR technology. Meta wants to help usher in this technological revolution using its power, influence and money to research and launch new hardware and software which will take us into the next evolution of the internet. 

Meta - Zuckerberg

Metaverse

The metaverse is here! Well, kind of. The latter half of 2021 has been awash with talk around a metaverse. What was once a concept that few people acknowledged has now become a buzzword that even your grandmother knows (Thanks Facebook… oh, Meta). Whatever your thoughts on the metaverse, it’s coming up fast. In fact, to some, it’s already here. If you’re playing Fortnite or Roblox then you’re already on the first rung of the ladder, and projects such as Somnium Space, Decentraland and The Sandbox are waiting for you to jump in. This ownership driven, decentralised digital space is an important change to the way we use the internet. Are you ready?

Unreal Engine 5

2021 finally saw the release of Epic Games’ Unreal Engine 5, bringing a dearth of powerful development tools to the industry. With so many developers utilising Unreal Engine to create their projects, this new iteration gives us a glimpse of what’s to come over the next decade. Launching with an interactive ‘“experience” collaborating with The Matrix Awakens, players and creators have seen the potential and it’s revolutionary. The level of detail and fidelity UE5 will bring is likely to change the landscape of games, from battle royales to VR puzzlers.

Unreal Engine 5

Sony’s 8K headset

As reported by our very own Peter Graham, Sony unveiled a prototype VR headset with not only 8K visuals (4K per eye) but also ultra-low latency. This bodes well for the company’s future, given they are soon releasing an upgraded VR headset for the PlayStation brand. Will we see this fidelity over there? It’s unlikely, but the new technology could make waves in industrial and medical avenues. This jump in technology bodes well for the future of Virtual Reality.

Niantic Lightship

If you’re an AR developer, then 2021 was a good year. Niantic Labs, the company behind pretty much every hugely successful AR mobile game, released their ARDK tools for developers to use. What does this mean? Well, it means that the software they use for their titles, including mesh mapping and semantic wrapping, two features that track and map the world seen through a smartphone camera, as well as their multiplayer API, can be utilised by any development team. This sharing of technologies can only benefit the AR community as a whole and further achieve great things in the world of AR.

Niantic Lightship

AR/VR in medicine

Many see Augmented and Virtual Reality as something built for games and experiences. Contrary to that view, both AR and VR are breaking down barriers in the world of medicine. Therapists are using VR to virtually visit their clients or help PTSD sufferers acclimate to the world. AR helped frontline workers learn how to care for those ill with COVID, using phone apps to triage patients when needed. Back in the virtual world, surgeons are completing spinal surgeries and trainee medical students are learning how to intubate patients using the technology rather than plastic dummies.

Digital Influencers

In 2021, the world of influencers got a bit more digital and a bit more creative. Since the advent of social media, influencers have become ubiquitous with the software – a selection of people touting products for corporations. However, with graphical software innovations, virtual and augmented reality, plus motion capture, we can now find digital avatars living the influencer life. Though right now, they aren’t trying to sell us anything, except maybe their art. CB from Casas Bahias, CodeMiko and Blu are amazing CGI avatars living digital lives, creating comedy, drama or interactive experiences. 

Taking the First Steps Into an AR Metaverse

With so many companies and brands expressing their plans for the metaverse, one technology will be key for early adopters – Augmented Reality. AR technology primarily uses smartphones and, to a lesser extent, smart spectacles to adapt the reality we see around us, often adding a new layer of interactivity. Because this tech uses hardware most of us already own, it’s easy to see that one of the ways the metaverse will start being pieced together is via AR videogames and apps.

If you cast your mind back to the summer of 2016, you’ll remember a large portion of the public were wandering parks and streets trying to catch Pokemon. Pokemon Go, created by Niantic Labs, uses AR technology in a few ways; by overlaying Pokemon onto a map which you are physically walking in ‘the real world’; it uses the camera in your phone to show the Pokemon against the backdrop of where you are currently; and it allows you to interact with other players through their avatars within the game, while they might be stood only a few metres away in-person.

It’s not only Pokemon Go capitalising on AR; Spokko Sp. z.o.o. adapted the popular videogame franchise The Witcher into a monster-slaying game; Niantic Labs grabbed another Nintendo franchise with their recent Pikmin Bloom release and in 2018, developer Ludia brought Jurassic World to mobile platforms so that we could capture a T-Rex in our local parks. Games are a great playground for developers to try new technologies.

Pokemon Go is still a very popular mobile game five years later and the AR technology used by Niantic Labs is advancing steadily. In early November 2021, the company announced Niantic Lightship, a platform dedicated to the development of Augmented Reality which can be shared with other developers. This new technology is built for the metaverse, using our phones to capture images and use machine learning to translate that footage into an accurate 3D model. Alongside this, consumers will be able to view particular objects through their phone screens and enjoy a unique experience collaborating together, or creating memories at concerts, like the upcoming Ed Sheeran performance due to take place within Pokemon Go.

However, what started with gaming, is growing into a slice of the metaverse that will become available to anyone with a smartphone. The possibilities for AR are constantly being explored. Coachella, the American music festival, is implementing AR features that will enhance a band’s performance as lights, colours and creatures mix with the live-action on stage; Science Museum Group plan to use the technology to bring their exhibits to life and take strides to further education; while the meditation app, TRIPP, will use this tech to help users become more mindful while out in the world.


As people become more familiar with AR, our phones will transform the way we interact not only with the world, but also how we shop or converse with each other. From the small ideas such as IKEA creating an app that places furniture choices in the intended room for you to see how it will fit your home style. To the big brain concepts – Google is using AR to power their language translation apps and visual search engine with Google Lens where you simply point your camera at foreign wording to see it translated in real-time. Plus, projecting directional arrows onto the ground or floating in the air, to plot a journey in Google Maps. Of course, there’s still a way to go before we’re all connected to each other in a central metaverse. However, these games and apps point to what we can expect from the future.

This link between AR and smartphones is integral, not only to keep consumer costs low but because the hardware is powerful enough at this stage to implement the tech needed. All forms of AR require several pieces of hardware which every smartphone has – camera, speakers, microphone, accelerometers and processing power. Depth registration, which places a digital item overlaid onto our world, needs that camera. The accelerometer gauges distances and moves the AR objects based on how the user moves. Most important, is that processing power; with so many AR apps and games, the processor must use machine learning to crunch algorithms in order to decipher what the camera, and thereby the user, is observing.

At first, AR was feeling too many as a bit of a gimmick. It’s now evolving into a genuinely helpful and impressive piece of technology that could impact our daily lives. The beauty of this technology, starting with gaming, is that it proves an accessible entry. The public can make the leap from throwing pokeballs to using AR in the workplace. With incremental movements within apps, soon projecting videos against a wall during a meeting with colleagues which can then be edited or annotated by anyone in the room, through their phone, could become a metaverse standard, with swish 3D avatars.

To get a sense of where AR is heading, it’s worth sticking close to videogames. Euclidean Skies is a puzzler that has a lot in common with Escher sketches. Holding your phone in front of you, labyrinthine buildings take over the empty space of your living room. These can be turned and rotated to solve the puzzle. While Smash Tanks conjures digital tanks, with a battlefield, and lays it out on the floor with no mess to tidy once you’re done. Each of these games contains the same technology that we’ll see from developers working on Niantic Lightship.

Just these two games can point towards other uses of the technology or other apps available which use similar ideas. Thyng is an interface that projects images and short video clips within a shared space; perfect for meetings in tiny offices. Quiver is a kids AR app which, when paired with colouring pages, brings a child’s artwork to life using the colours the child used.

It’s not a stretch to imagine that one day you’ll raise your phone, look through the screen at a friend and see their digital avatar. If we’re to believe Niantic Labs, it’s not much of a jump to walk over a bridge in your hometown with your phone held out and watch as the scene fades into what that bridge looked like fifty years ago, everyone digitally dressed in period clothing. Or for a group of kids touring a museum to be handed tablets that bring ancient worlds to life before their eyes or whisk them away to stand on the edge of a volcano.

Niantic Launches Lightship AR Dev Kit & $20 Million Creators Fund

Niantic Metaverse

Tech companies left and right are revealing their metaverse plans and yesterday augmented reality (AR) company Niantic did just that. Famous for its Pokemon Go videogame Niantic announced the official launch of its Lightship platform, rolling out a software development kit (SDK) for creators as well as a new $20 million USD fund to help companies build AR experiences.

Niantic Lightship

Unlike Meta (formerly Facebook) which is looking towards a virtual metaverse, Niantic is going down the path of a “real-world metaverse”, building AR content that supports features like multiplayer in real-world settings. To achieve this Lightship’s Augmented Reality Developer Kit (ARDK) tools are now available, the same ones used for Niantic’s other cross-platform projects across Android and iOS devices.  

Niantic splits these tools into three categories – Real-Time Mapping, Understanding and Sharing. Real-Time Mapping and Understanding are all about using smartphone camera sensors to build knowledge of the world around you, scanning the environment to create 3D mesh maps of where objects are whilst utilising advanced occlusion API’s the digital and real worlds are seamless.

Just as critical is the inclusion of multiplayer. Metaverses are all about social connection rather than isolation with developers able to utilise the Lightship platform to build AR experiences that support up to five people concurrently. Creators will also find that ARDK: “provides some lightweight multiplayer gaming functions, including a player lobby system, a synchronized clock, and session-persistent storage.”

Niantic Lightship

To encourage more companies and developers to utilise Lightship Niantic Ventures has set up a $20 million fund to: “invest in companies building applications that share our vision for the Real-World Metaverse,” Niantic notes. Partners already include Coachella, the PGA of America, Historic Royal Palaces, Science Museum Group and Shueisha, all of which will be releasing apps in 2022.

“Transforming humanity’s relationship with technology by merging the physical and virtual worlds will require the ideas and perspectives of as many people as possible,” said John Hanke, Founder and CEO of Niantic in a blog post. “That’s why we’re so excited to open the vault of technology that powers our own games, so developers, creators and brands globally can build inclusive experiences that push the boundaries of what’s possible in AR.”

As the Niantic Lightship platform continues to grow VRFocus will keep you updated.

Pikmin Bloom Arrives to Brighten Up Any Walk

Pikmin Bloom

Niantic Labs’ videogames continually focus on getting players out and about and back in March the company announced a new project in conjunction with Nintendo based on its Pikmin franchise. Today, the pair have begun the global rollout of Pikmin Bloom, a colourful augmented reality-capable experience all about building up a squad of Pikmin simply by walking.

Pikmin Bloom

Offering a much more casual, combat free experience in comparison to Niantic Labs’ other AR titles, Pikmin Bloom is all about nurturing these tiny plant-like creatures which have appeared in several Nintendo videogames. To do this players need to walk, finding Pikmin seedlings along their path which can then be picked, adding to the Pikmin entourage. All the Pikmin will follow the player making flowers bloom on their journey.

When players take their daily stroll they’ll need to collect fruits along the way to feed their Pikmin, in turn making beautiful flowers bloom atop their heads. Their petals can then be collected and planted to create even more flowers with each step, unleashing a cascading effect of colour. That’s not the only interactive element, players will be able to utilise their red, yellow, blue and more Pikmin to remove giant mushrooms to gain more fruit.

As you may have realised, fitness is an integral part of the experience. Pikmin Bloom will have basic pedometer functionality, recording the number of steps players take as well as the routes they walked. Building upon that healthy lifestyle image they can add notes and photos to a lifelog, take pics with the Pikmin who can also bring back postcards of the places players have visited, shared with friends in the app. 

Pikmin Bloom

Whilst Pikmin Bloom is a free app Niantic Labs monetizing the title with in-game purchases, selling booster items to enhance the experience. There will also be monthly Community Day events.

Pikmin Bloom is available to download for iOS and Android devices on the App Store and Google Play respectively in Australia and Singapore, to begin with. Other regions will slowly be incorporated as Niantic Labs expands the launch. For continued updates on the latest AR projects, keep reading VRFocus.