Cubism: How An Architectural Mindset Spawned An Intuitive VR Puzzle Game

Thomas Van Bouwel is the one-man developer behind Cubism, one the best puzzle games available on the Oculus Quest. Van Bouwel now works full time on Cubism and is developing a set of DLC levels for the game, but five years ago he was working in a completely different industry with no experience in the world of VR.

“My background is actually not in computer science or game development at all. I actually come from a background in architecture,” says Van Bouwel, speaking to me in VR from UploadVR’s virtual studio. “I was a practicing architect for several years, until I made the switch. It was architecture that actually led me to move into VR. I was really fascinated by what VR could do for the design process and how it could make the design and architecture a lot more human-centered. Instead of looking at a plan for a building from top-down, you look at a building from a human perspective while you design it.”

cubism

Van Bouwel was always fascinated with the ways that immersive technologies and game engines could be used within an architectural context, and ended up investigating further during his university studies. In his third year, Van Bouwel gave a presentation of a 3D architecture model displayed in the PC game Crysis, imported using CryEngine. He found that from showing his friends and family the model in-game, he received a lot more feedback compared to showing them plans or scale models. “That was when it first clicked. Like, ‘Oh, these sort of immersive technologies can be very useful for the design process.'”

After working as an architect for several years, Van Bouwel decided to quit his job in 2016 and started learning how to develop for VR. He attended hackathons, went to game jams and landed a job at an enterprise VR startup, InsiteVR (now known as Resolve), within a year and a half. While he had a Vive headset at home, the job gave him access to a variety of different headsets, including an early Quest prototype — the Santa Cruz — in 2017.

It was around then that Van Bouwel came up with a new concept for a VR puzzle game. Working in his free time, he soon created the first build of what would become Cubism — an intuitive, deceptively simple puzzle game where the user must place different shaped blocks together to fit into a larger wireframe shape.

“It sort of really came out of that fascination with minimal design in mobile games and what that could mean in VR games,” says Van Bouwel. Minimalist mobile games like Mini Metro, Hitman Go and Lara Croft Go were big inspirations. “These games can use a very simple look and a very simple input scheme, but then can have a lot of depth to them. And more importantly, [they are] games that are really well scoped for small teams.”

“A big part of choosing to work on Cubism was that it felt like it was small enough in scope that I could work on it in my spare time. At the time I was working at that startup, which I really enjoyed working at. So I knew that if I was going to work on a project on the side, it better be something small and something that was manageable.”

Cubism’s puzzles are a riff on the classic wooden block puzzles that have been around for many years. In real life, the end shape is usually restricted to something quite memorable and solid — a rectangle or a cube. With Cubism, the shapes of the pieces and the wireframes vary wildly in both difficulty and design, no longer held back by gravity pulling at physical blocks.

The real beauty of Cubism is in its simplistic, clean design — the architectural influence is very clear. The entire game can be played with just the controllers and the triggers. If you’re using an Oculus Quest, you don’t even need controllers — you can just use your hands. This game-wide policy of simplicity and accessibility means that almost anyone, even those inexperienced with VR, can pick up the game and play it. “I put a lot of effort into that actually. The game came from being in a stage where it was a lot harder to use and learn.”

Earlier versions of Cubism mapped several actions to different controller buttons. This wasn’t initially a problem, as only experienced gamers and other VR developers were doing the playtesting. “I was missing a lot of UX issues early on, because if people are comfortable with VR or they’re comfortable with playing games in general, they learn these controller mappings pretty quickly. But if people aren’t, it’s really a roadblock for them.”

The UX issues became more apparent when Van Bouwel took his builds on the road to a wider audience. “Putting those people in the game and seeing them sort of struggle with the control scheme, that really sort of flipped the switch for me. Like, I really need to simplify this.”

cubism

“From that point on, I did everything I could to make the whole game playable with just the triggers, which required a few redesigns.” Small changes — allowing the triggers to be used for all actions, placing a menu indicator beneath the puzzle — made the game much more accessible to the uninitiated. “From the start of the game to solving the first puzzle, that became a really fast process. People could get started with their first puzzle within like 10 or 15 seconds of starting the game.”

“That was always a goal of mine — to make sure that it was something that was approachable,” he says. “I think that’s part of the appeal of VR too, right? That it should be more intuitive.”

Cubism released last year, and while the game is available on both PC VR and Quest, it shines on the latter. Not only is it among the best puzzle games for the system, but since launch Van Bouwel has also added hand tracking and 120Hz support, which both elevate the game to an impressive new level. It’s also one of the best showcases of the Quest’s cutting-edge VR technology and features.

Now, approaching a year since release, Van Bouwel is designing and playtesting a set of 30 new DLC levels for the game.

“DLC seemed to be like the best way to give people more puzzles to play,” he says. “It’s still a work in progress, but hopefully [the DLC levels] are going to have a bit of a different look from the main campaign to differentiate them. But they will play very similar to the puzzles in the main campaign.”

“And much like the campaign, there’s going to be a song that’s sort of tied to the puzzles, specific to the DLC.”

Sound is another big part of Cubism’s design. In the main campaign, each different puzzle piece has an associated piano note. When you solve a puzzle, each of the notes will play together to form a chord. If you combine all the chords from each level together, it forms a song. “The main campaign is basically this big song that you unlock as you play it. And the DLC will have a song of its own that you unlock as you play the puzzles as well.”

cubism

But how does Van Bouwel approach designing a Cubism level, both in terms of its form factor and its difficulty?

“Usually, it starts with a shape,” he says. “Then I start filling it in with pieces.”

“I try to make the puzzles in such a way that you can approach them with some reason, so that you don’t just have to trial and error your way through it. There are like certain tricks that I know I can design for — maybe there’s one or two pieces that can only go in several spots of the puzzle, so it gives a starting point.”

Likewise, on the other end of the scale, certain elements will bring complexity. “If there’s a limited number of symmetries in the puzzle shape or in the piece shape, then it becomes a lot harder to sort of do these mental rotations when you’re placing the pieces into the puzzle.” Despite this, playtesting puzzles is essential, which is why Van Bouwel has been running a series of weekly playtests with Cubism fans over the last six weeks as he works on the new levels.

Beyond the DLC, a local editor might be on the cards eventually, allowing players to create their own levels and share them online via sideloading, similar in process to installing custom Beat Saber levels. In terms of adding multiplayer modes or leaderboards, Van Bouwel isn’t too bullish. “It’s something that I’ve sort of consciously shied away from in this game. Part of the design of the game is to be a relaxing game and a relaxing experience. I wouldn’t want any of the mechanics to pressure you into needing to find a solution.”

When it comes to future projects beyond Cubism, Van Bouwel has nothing official to announce just yet. “It’s a bit early,” he says. “I do have some ideas of next projects that I would want to work on, but [they are] ideas without any execution… I’m looking forward to [when] I have time to prototype these new ideas. Hopefully then I have something that I’ll be happy to share.”

Cubism is available on Oculus Quest and PC VR now, with extra DLC currently in development.

Livestream Interview And Q&A: Cubism Developer Thomas Van Bouwel

Cubism is one of the best puzzle games available on Quest, with excellent native VR design and an excellent showcase of hand tracking capabilities on Oculus Quest. We’re sitting down with Cubism’s developer, Thomas Van Bouwel, to talk about developing the game, plans for DLC and more.

The interview is set to take place later today, Wednesday the 16th of June at 2:00pm PST, on our YouTube channel. It will be broadcast live from our virtual studio in VR, where we also host our podcast, The VR Download.

If you manage to join us for the interview livestream, please leave your questions in the chat and we’ll try to get to as many as possible. If you miss it live, you can still check out the recorded broadcast at the same link below.

Cubism launched just under a year ago, but has consistently rolled out a series of impressive updates and new features for Quest since then. Its implementation of hand tracking is absolutely sublime and one of the best showcases of the technology on the system. You can also pair hand tracking with the recently added 120Hz support as well, which rounds Cubism out as one of the best titles to showcase most of the Quest’s cutting edge features all at once.

Last month, Van Bouwel announced that a series of DLC levels were in the works for Cubism and signs-ups were open for those who wanted to help playtest new levels. I was able to join the playtesting group and gave some of the new puzzles a try over the past six weeks. I’ll be discussing the approach taken in designing these DLC levels and the original campaign with Van Bouwel in our interview today, going over the process from start to finish and how the upcoming levels might differ to the existing campaign.

Let us know if you’re looking forward to Cubism DLC and be sure to leave your questions in the chat during the live broadcast.

Cubism To Receive DLC Levels, Playtesting Sign-Ups Now Available

Puzzle game Cubism will receive some new levels in an upcoming DLC pack, some of which are currently available for early playtesting.

The game’s official account announced on Twitter that DLC was in the works, while also gauging interest from players who might want to test the new levels before release.

The beta testing of new levels will run over the next few weeks, and anyone who is interested in giving some new levels a try can sign up here.

If you gain access to the new DLC test levels and already own Cubism, your copy should automatically switch over to the puzzle playtesting release channel. If not, you can manually select it by locating Cubism in your library on the Oculus app and clicking on the version number, which will let you switch between the current live release and the playtesting puzzles release channel.

That being said, you don’t need to own the game to sign up for playtesting. Even if you’re new to the game, the playtesting channel will grant you (temporary) access to the full game as well. If you haven’t tried Cubism yet and you’re on the fence, this is a great way to get a feel for the game before making the full purchase. Even better, anyone who helps test the new levels will receive a free code for the DLC when it releases.

The testing puzzles will run for a few weeks, changing every Friday. You can access them via the new Puzzle Playtesting button on the main menu.

Cubism is one of our favorite puzzle games on Quest — you can read our review and our impressions of the hand tracking update here. The game also recently received 120Hz support, making it one of the few games with 120Hz hand tracking support on Quest.

Best Puzzle VR Games Available For Oculus Quest 2

Looking for some mind-bending puzzle games to play through on Quest 2? Here are our picks.

The Oculus Quest library is only growing larger and stronger by the day with a few options available in most genres. For puzzle fans, we’ve put together this list of some of the best in the genre available on Quest. Some are inventive and creative, some are mind-bending and challenging, but there should be something for everyone.

Here are our picks for the best puzzle games available on Oculus Quest.

A Fisherman’s Tale

Before its more recent game Maskmaker, Innerspace released the acclaimed A Fisherman’s Tale on Quest in 2019. It’s a short yet incredibly charming puzzle game, with ingenious puzzles will make you rack your brain until you finally reach that critical ‘a-ha’ moment that feels so satisfying.

A Fisherman’s Tale was nominated in our Best of VR 2019 Awards for the Best PSVR Game/Experience, Best Quest Game/Experience, Best PC VR Game/Experience, and overall Overall Best VR Of 2019. You can read more in our review.


Cubism

Cubism is a deceptively simple game. Each level features a 3D wireframe shape into which you have to fit different Tetris-like block pieces. The puzzles get harder and the pieces more complex – it’s a slow and measured puzzle experience.

Even better, the game’s recent updates include 120Hz and hand tracking support. The latter in particular is a near-perfect fit for Cubism – all you’re doing is picking up pieces and placing them into the wireframe, but it’s enough to create a sublime hand tracking experience. This isn’t a ranked list, but if it were, Cubism would be my personal #1.

You can read our review of Cubism here (written before hand tracking support was added) and read our impressions of the hand tracking update here.


Floor Plan 2

Floor Plan 2 feels like a VR episode of The Muppet Show, not just in the hilarious absurdity of its world but also in the constant, invigorating ingenuity of its puzzles. It is a logic-based puzzle game in a completely illogical world and one of the best recent releases on the Quest platform.

Read more in our Floor Plan 2 review.


Puzzling Places

This game takes the staples of 2D jigsaw puzzles and brings them into VR with a new element — all of the puzzles are 3D models of real-life places, brought to life through highly detailed photogrammetry captures. The 16 included puzzles range from landscapes to individual objects, each with accompanying ambient sounds that build up as you solve. It’s a slow, meditative but also appropriately challenging puzzle game that offers a similar yet fresh take on traditional jigsaw puzzles.

You can read more in our review.


 

A Rogue Escape

This is a tough, short (1-2 hours), claustrophobic puzzler that will set you stuck in the cramped confines of a submerged submarine. There’s no hand-holding here, resulting in an immersive yet excruciatingly slow set of submarine-based puzzles. If you’re looking for an incredibly short but difficult experience, then this might be the best option.

You can read more in our A Rogue Escape review.


Tetris Effect

If you’re a fan of Tetris, then Tetsuya Mizuguchi’s Tetris Effect is an absolute must-play. It launched first on PSVR and PC VR before finally making its way over the Quest platform in 2020.

It is exactly what it sounds like – it’s Tetris in VR. However, it’s matched with a spectacular, powerful and at times hypnotic amount of visual and audio flair. While these visuals have had to be pared down slightly for the Quest release, we still deemed it the definitive version of the game across all platforms – the lack of tethered wires on Quest remains priceless.

You can read more in our Tetris Effect Quest review.


I Expect You To Die 2

Five years on from the release of the original, I Expect You To Die 2 is a game that executes a specific style of grueling puzzle game incredibly well. It’s a trial-and-error affair where you’ll spend an hour testing things, only to realize that the solution was obvious and right in front of you the whole time. It’s the mark of a quality puzzle — one where the solution is hidden not through obscure design, but through your own short-sightedness — but it can also mean a slow burn and periods of heavy frustration.

You can read more in our review.


The Room VR: A Dark Matter

The latest installment in The Room series is also its first installment in VR. This isn’t a mobile or PC port either – The Room VR: A Dark Matter is made entirely for VR from the ground up.

Developer Fireproof hasn’t lost its penchant for incredibly clever and creative puzzles in the transition. Read more in our review.


Myst

This seminal PC game made its way over to the Oculus Quest late last year. It’s an upgraded and all-around solid port of the original PC title that will have you reaching for a notebook or a friend to help work your way through the tough puzzles.

It’s a game everyone should play once and the Quest 2 port provides you with a modern but faithful way to do so. You can read more in our review.


Angry Birds VR: Isle of Pigs

While the Angry Birds franchise started on mobile, its first foray into VR is a completely natural transition and results in an extremely fun VR puzzle game. It’s an intuitive VR title that’s perfect for VR beginners and veterans alike.

The campaign levels do veer slightly to the easy end of the spectrum, but it’s also a game primarily targeted at kids. This isn’t to say it’s not enjoyable for all ages though, and the custom level builder and online sharing functions mean that you’ll be able to create and play more levels long after you’ve finished the campaign.

You can read more in our review.


What are your favorite puzzle games on Quest? Let us know in the comments.

Every Game That Supports 120Hz On Oculus Quest 2

With the v28 software update, Oculus Quest 2 now has experimental support for a 120Hz refresh rate. Here’s every game that can run at 120Hz on Quest 2 (so far).

The Quest 2 shipped last year with just a 72Hz refresh rate — the same as the original Quest model. Since then, it’s received multiple software updates that add in support for higher refresh rates. First it was 90Hz, and now there’s experimental support for 120Hz as well.

Don’t get too excited though — not all existing Quest games will be able to support 120Hz. In fact, in the past John Carmack has said that only a few existing Quest games will be able to make the leap from 90 to 120, because of the high performance requirements.

To enable 120Hz on the games below, make sure you’re updated to v28 of the Quest software and then enable the 120Hz setting under Experimental Features in your Quest settings menu. If you’re not able to download v28 yet, you may just have to wait — Quest software updates roll out to users gradually, and not all at once.

For some games, you’ll also have to enable a 120Hz setting in the game’s menu as well.

Without further ado, here’s the list of all Quest 2 games that support 120Hz at the time of writing. This list will be frequently updated as more games add support, so check back often. And if you’re a developer with a 120Hz app running on Quest, please reach out to us via tips@uploadvr.com to let us know.

You can pick up all these titles on the Oculus Store (or via App Lab, where noted).

Echo VR

Echo VR Quest

Zero-G team sport Echo VR now supports 120Hz “during core gameplay,” according to developer Ready At Dawn. Certain areas of the game including the Arena tunnels, goal celebrations and the lobby still run at 90Hz. For those unfamiliar, Echo VR is a disc-throwing game that’s extremely similar to the sport from Ender’s Game. It’s also extremely competitive so getting in there and experiencing it at 120Hz could give you a sense of whether the increased frame rate can have an effect on competition.


Superhot VR

One of VR’s best games offers experimental 120Hz support on Oculus Quest 2. The Matrix-style title where time moves only when you do can now be played just a little bit smoother with the feature activated. The developers warn that it is experimental, so if you encounter any problems maybe revert the settings back to a lower refresh rate.


Cubism

cubism 120hz quest

Cubism was the first app to add 120Hz support on Quest 2. We reviewed the game last year and thought it was absolutely fantastic. 120Hz support is just the latest in a string of updates that bring support for the latest Quest technology to the game — a few months ago, Cubism also received fantastic hand tracking support.


Eleven Table Tennis

By way of Reddit post, we’ve seen that Eleven Table Tennis is getting 120Hz support on Quest 2. For a game that’s widely considered to be one of the most accurate table tennis simulations out there, the smoother it can run the better. This will undoubtedly make Eleven an even easier game to recommend for Quest 2 owners.


Gorilla Tag

gorilla tag

Gorilla Tag is a PC VR and Quest title in Early Access that is exactly what it sounds like – it’s tag, but in VR and everyone is a gorilla. It has a deep and complex locomotion system that players have really grown to love.

It added 120Hz support recently and is now available on App Lab for Oculus Quest 2. You can read more here.


Racket Fury

racket fury 120hz

The second app to add 120Hz support on Quest 2 was the table tennis game Racket Fury. You can enable 120Hz in the video options menu in-game.


Yupitergrad

Yupitergrad

Yupitergrad dropped an update in May that adds 120Hz support for the entire game, alongside some new Time Attack levels as well.

The game will see you swing like Spider-Man through difficult obstacle courses. Except instead of New York City, you’re in a derelict Russian space station and you’ve swapped out web shooters for a pair of arm-mounted projectile plungers.

You can read our review here.


Synth Riders

synth riders quest

As one of the biggest and oldest VR rhythm games on the platform, it’s great that Synth Riders added 120Hz support for Quest 2 in June. Not only will game now run at a silky smooth high framerate, but it also features a wealth of content that has arrived in updates over the last few years. There’s multiplayer, new stages and DLC tracks that feature increasingly well-known artists such as The Offspring and Muse.

It also recently received visual updates that should make the game look better across all platforms.


Virtual Desktop

virtual desktop update

Virtual Desktop’s 120Hz update means that you can play your PC VR games wirelessly at 120Hz using the service — something that a wired Oculus Link connection or Oculus’ newly-announced official wireless streaming competitor, Air Link, won’t support in v28. However, it’s currently only available in the beta release of Virtual Desktop as an opt-in service.

To opt into the beta release, go to the Virtual Desktop store listing, either online or on the Oculus App, and click on the version number in the information section. You can then select and opt into the beta release. Download it onto your headset and enable 120Hz in Virtual Desktop’s settings and you’re good to go.


Hyper Dash

hyper dash featured image art

Hyper Dash is a competitive VR shooter that is all about fast movement and pure chaos — it can be both overwhelming and supremely satisfying. There’s a variety of different modes, along with PC VR crossplay to help find matches quickly. If you’re looking for a VR shooter that veers away from the realistic military vibe and goes for something unique, give Hyper Dash a try at 120Hz on Quest 2.

You can read more in our review (written before 120Hz support) here.


Down the Rabbit Hole

Down The Rabbit Hole Hands-On

In early June, Down the Rabbit Hole added 120Hz support for Oculus Quest 2. This VR puzzle game takes place in Lewis Carroll’s fantastical world of Wonderland — on release, we said it was “as strange a VR game you’ll find” with some fantastic moments throughout.

You can read our full review here.


Rhythm ‘n Bullets

rhythm n bullets

This is the only game on the list that isn’t available on the Oculus Store — it’s actually an App Lab title. It’s a blend between a shooter and a rhythm game, as the title might suggest.

Just like Virtual Desktop, 120Hz support is currently only available in the beta release of Rhythm ‘n Bullets. To opt into the beta release, go to the Rhythm ‘n Bullets App Lab listing, either online or on the Oculus App, and click on the version number in the information section. You can then select and opt into the beta release. Download it onto your headset and enable 120Hz in-game and you’re good to go.


What apps do you want to get 120Hz support? Let us know in the comments and check back for updates to the list.


Editor’s Note: Since publication this post was updated to include several more games. It was originally published on April 16 and the publish date changed with updates to reflect games adding support.

Cubism Adds 120Hz Support for Oculus Quest 2

Cubism

Two days ago Oculus announced what new features would be added in the next software update for the Oculus Quest platform. v28 will see several additions specific to Oculus Quest 2 including 120Hz refresh support in the experimental section. Today, developer Thomas Van Bouwel has ensured his popular puzzler Cubism is ready when that time comes.

Over on the Cubism Twitter account, the developer confirmed that patch 1.2.4 has now gone live, adding a 120Hz refresh rate button in the settings menu. You’ll have to be patient to try it out as you’ll need the v28 update first which is being gradually rolled out to users. Once you have it, head to the Experimental Features section to activate the 120Hz mode before diving into Cubism.

Just like Van Bouwel has done, 120Hz support has to be implemented by developers so their applications can run at higher frame rates. For a simple puzzle title like Cubism which isn’t heavy on performance, those higher rates are achievable, but many titles on Oculus Quest 2 likely won’t manage it. As John Carmack previously Tweeted on the subject: “Only a few existing games will be tweaked for 120, but some new titles will consider it an option in their design phase.” So native support could be a common occurrence in the future.

Cubism launched in September 2020 and quickly became a favourite of VRFocus as well as many other Oculus Quest owners thanks to its addictive puzzle gameplay. “Easy to pick up yet difficult to complete, it may take some an hour whilst others will take five. It’s the kind of VR game you can give to anyone and they’ll instantly know what to do, because who doesn’t know how to place a few blocks together,” VRFocus said in its review.

Cubism

Van Bouwel has been quick to enhance the videogames’ mechanics since launch, with hand tracking support being a major addition recently. With it, players can grab a puzzle piece rather than merely pinching and buttons can be naturally pushed for a more immersive feel.

VRFocus will continue its coverage of Cubism as well as any other Quest titles adding 120Hz support, reporting back with regular updates.

Cubism Adds 120Hz Support For Quest 2

A new Cubism update is live, adding support for 120Hz for Quest 2 users on v28.

Earlier this week, we learned that the Quest 2’s next software update, v28, is coming soon and it’s a big one. Not only will the update add support for streaming PC VR games wirelessly to your Quest with Air Link, but it also adds the option to turn on experimental support for 120Hz refresh rate.

The Quest 2 shipped with the same refresh rate as the original quest, 72Hz. In November last year, support was added for a higher refresh rate, bumping up the max from 72Hz to 90Hz. Now, certain apps will be able to support an even 120Hz higher refresh rate. With Cubism adding support already, it looks to be the first title to support 120Hz on Quest 2.

Great as Cubism is, though, it’s a fairly simple game in terms of performance, making it the ideal candidate for 120Hz. As John Carmack has pointed out, only a few existing Quest games will actually have the performance overhead to run 120Hz support reliably with the Quest 2’s hardware. That being said, Carmack expects future games might be developed with 120Hz in mind during design, making them more likely to meet the required benchmark for reliable performance at 120Hz.

We reviewed Cubism back in September last year and absolutely loved it. It remains one of the best puzzle games on Quest and it’s recent hand tracking update is a fantastic addition to an already game great.

To enable 120Hz in Cubism, you will need the Quest v28 software update, which is rolling out to users gradually. You’ll also have to enable 120Hz support in the Quest menu under Experimental Features, and then tick 120Hz in Cubism’s setting menu as well.

Case Study: The Design Behind ‘Cubism’s’ Hand-tracking

Hand tracking first became available on the Oculus Quest back in late 2019. Out of enthusiasm for this new input method, I published a demo of Cubism to SideQuest with experimental hand tracking support only a few days later. Needless to say, this initial demo had several flaws, and didn’t really take the limitations of the technology into account, which is why I decided to initially omit hand tracking support from the full release of Cubism on the Oculus Store. It took more development, leaning on lessons learned from the work of fellow developers, to build something I was happy to release in the recent Cubism hand-tracking update. Here’s an inside-look at the design process.

Guest Article by Thomas Van Bouwel

Thomas is a Belgian-Brazilian VR developer currently based in Brussels. Although his original background is in architecture, his current work in VR spans from indie games like Cubism to enterprise software for architects and engineers like Resolve.

This update builds on lessons learned from many other games and developers who have been exploring hand tracking over the last year (The Curious Tale of the Stolen Pets, Vacation Simulator, Luca Mefisto, Dennys Kuhnert, and several others).

In this article I’d like to share some things I’ve learned when tackling the challenges specific to Cubism’s hand interactions.

Optimizing for Precise Interactions

Cubism’s interactions revolve around placing small irregular puzzle pieces in a puzzle grid. This meant the main requirement for hand tracking input was precision, both in picking up and placing pieces on to the grid, as well as precisely picking out pieces from a completed puzzle. This informed most of the design decisions regarding hand input.

Ghost Hands

I decided early on to not make the hands physics-based, but instead let them pass through pieces until one is actively grabbed.

This avoided clumsily pushing the floating puzzle pieces away when you are trying to grab them mid-air, but more importantly, it made plucking pieces in the middle of a full puzzle easier since you can just stick your fingers in and grab a piece instead of needing to figure out how to physically pry them out.

Signaled by their transparency, hands are not physical, making it easier to pick out pieces from the middle of a puzzle.

Contact Grabbing

There are several approaches to detecting a users intent to grab and release objects, like focusing on finger pinches or total finger joint rotation while checking a general interaction zone in the palm of the hand.

For Cubism’s small and irregular puzzle pieces however, the approach that seemed to handle the precision requirements the best was a contact based approach, where a piece is grabbed as soon as thumb and index intersect the same piece and are brought together over a small distance, without requiring a full pinch.

Similar to the approach in The Curious Tale of the Stolen Pets, the fingers are locked in place as soon as a grab starts, to help give the impression of a more stable looking grab. The piece is parented to the root of the hand (the wrist) while grabbed. Since this seems to be the most stable tracked joint, it helps produce a steadier grip, and guarantees the piece stays aligned with the locked fingers.

Piece is grabbed when thumb and index intersect it and are brought together slightly. Rotation of index and thumb are then locked in place to help give the impression of a stable grab.

As soon as a piece is grabbed, the distance between thumb and index is saved, and a release margin is calculated based on that distance. Once thumb and index move apart beyond that margin, the piece is released.

Several safeguards try to prevent unintentional releases: we don’t check for release when tracking confidence is below a certain threshold, and after tracking confidence is re-gained, we wait several frames until checking for release again. Fingers are also required to be beyond the release margin for several frames before actually releasing.

Debug visualization: during a grab, the initial grab distance between fingertips is saved (outer red circle). The piece is released when the real position of the fingertips move beyond a certain margin (blue circle).

There is also a system in place similar to Vacation Simulator’s overgrab method. Due to the lack of haptic feedback when grabbing a piece, it’s not uncommon for fingers to drift closer to one another during a grab. If they close beyond a certain threshold, the release margins are adjusted to make releasing the piece easier.

Try it yourself: to see these debug visualizations in-game, go to ‘Settings > Hand Tracking > Debug visualizations’ and turn on ‘Interactions widgets’.

Debug visualization: If fingers drift to each other during a grab over a certain threshold (inner red circle), the release margins are re-adjusted to make releasing the piece feel less “sticky”.

One limit to this approach is that it makes supporting grabbing with fingers other than the index a bit harder. An earlier implementation also allowed grabbing between middle finger and thumb, but this often led to false positives when grabbing pieces out of a full puzzle grid, since it was hard to evaluate which finger the player was intending to grab a specific piece with.

This would not have been an issue if grabbing revolved around full finger pinches, since that results in a more clear input binary from which to determine user intent (at the cost of a less natural feeling grab pose).

Midpoint Check

Besides checking which piece the index and thumb are intersecting, an additional check happens at the midpoint between index fingertip and thumb fingertip.

Whatever piece this midpoint hovers over will be prioritized for grabbing, which helps avoid false positives when a player tries to grab a piece in a full grid.

In the example below, if the player intends to grab the green piece by its right edge, they would unintentionally grab the yellow piece if we didn’t do this midpoint check.

Left: thumb, index & midpoint between fingertips are in yellow → grab yellow. Right: thumb & index are in yellow, midpoint is in green → grab green

Grabbing the Puzzle

Grabbing the puzzle works similar to grabbing puzzle pieces, except it is initiated by performing a full pinch within the grab zone around the puzzle.

The size of this zone is dynamically increased when switching from controllers to hands. This makes it a bit easier to grab, and helps reduce the likelihood of accidentally grabbing a piece in the grid instead of the grid itself.

The grab zone around the puzzle expands when switching from controllers to hands, making it easier to grab. Although it requires a full pinch, grabbing the puzzle works similar to grabbing puzzle pieces.

Dynamic Hand Smoothing

The hand tracking data provided by the Oculus Quest still can have a bit of jitter to it, even when tracking confidence is high. This can actually affect game play too, since jitter can be much more noticeable when holding the puzzle grid or a long puzzle piece by the edge, making precise placement of pieces on the grid harder.

Smoothing the tracking data can go a long way to produce more stable looking grabs, but needs to be done in moderation since too much smoothing will result in a “laggy” feeling to the hands. To balance this, hand smoothing in Cubism is dynamically adjusted depending on whether your hand is holding something or not.

Try it yourself: to see the impact of hand smoothing, try turning it off under
‘Settings > Hand Tracking > Hand smoothing’.

Increasing the smoothing of hand positions while holding objects helps produce a more stable grip, making precise placement on the grid a bit easier.

Pressing Buttons

One thing I noticed with Cubism’s original hand tracking demo was that most people tried pressing the buttons even though that was not supported at the time. Therefore, one of my goals with this new version of hand tracking was to make the buttons actually pushable.

Buttons can be hovered over when a raycast from the index finger tip hits a collider at the back of the button. If the index finger then intersects with the collider, a press is registered. If the index intersects the collider without first hovering it, no press is registered. This helps prevent false positives when the finger moves from bottom to top.

There are a few more checks in place to prevent false positives: the raycast is disabled when the finger is not facing the button, or when the player is not looking at their finger when pressing.

Try it yourself: to see this debug visualization in-game, go to ‘Settings > Hand Tracking > Debug visualizations’ and turn on ‘Interactions widgets’.

Debug visualization: a raycast from the index tip checks whether the finger is hovering over a button. To help prevent false positives, interaction is disabled when the finger is not facing the button, or when the player is not looking at their finger.

Guiding Interactions

One of the main challenges of building any interaction for hand tracking is that, in contrast to buttons on a controller which are either pushed or not pushed, there are many different ways people may try to approach an interaction with their hands while expecting the same outcome.

Playtesting with a diverse set of people can help you learn how people are approaching the interactions presented to them, and can help refine the interaction cues that guide them to the expected gestures. Playtesting can also help you learn some of the outliers you may want to catch by adding some interaction redundancy.

Interaction Cues

There are several cues while grabbing a piece. When a user first hovers over a piece, their index and thumb take on the color of that piece, both to indicate it can be grabbed, and to signal which fingers can grab it (inspired by previous work by Luca Mefisto, Barrett Fox, and Martin Schubert). The piece is also highlighted to indicate it can be grabbed.

Several cues also indicate when the grab is successful: the fingertips become solid, the highlights on the piece flash, and a short audio cue is played.

Various cues both on the hand and the puzzle piece guide and confirm the grab interaction.

Buttons have several cues to help indicate that they can be pushed. Much like with puzzle pieces, the index fingertip is highlighted in white once you hover over a button, indicating which finger can interact. Like they did with controllers, buttons extend outward when hovered, but this time the extended button can actually be pressed: once the index touches it, it follows the finger until it is fully pressed down, at which point an audio cue confirms the click.

A subtle drop shadow on the button surface indicates where the position and distance of the index to the button and helps guide the press interaction.

Various cues guide interactions with buttons: buttons extend outward when hovered, the index fingertip is highlighted, a drop shadow shows where the tip will interact, and the button follows the finger when pushed.

Interaction Redundancy

Since some people may approach some interactions in unintended ways, it can be good to try and account for this where possible by adding some redundancy to the ways people can use their hands to interact. Interaction cues can still guide them to the intended interaction, but redundancy can help avoid them getting unnecessarily stuck.

When it comes to grabbing pieces, a few playtesters would try to grab pieces by making a fist at first instead of using their finger tips. By having the colliders cover the entire finger instead of just the fingertip, a decent amount of these first grabs will still be registered.

I should note this approach still needs some improvement, since it also introduces some issues producing unintended grabs in cases when there are a lot of pieces floating around the play area. A better approach in the future might be to also perform a check on the total finger rotation to account for fist grabs instead.

Though grabbing is designed around fingertips, colliders on index and thumb cover the entire finger to help catch different forms of grabbing.

With buttons, there were a few playtesters who would try pinching them instead of pushing them. In part this seemed to occur when they previously learned how to pinch buttons in the Oculus home screen, right before launching the game.

For this reason, buttons can also be clicked by pinching once they are hovered, and hopefully cues like the highlighted index and drop shadow will eventually guide them to pressing the buttons instead.

Pinching while hovering over buttons also registers as a click.

The first button players encounter when using hands also explicitly states “Push to Start”, to help transition people from pinching to pushing after coming from the Oculus Home menu.

Teaching Limitations

Although the quality of Quest’s hand tracking has improved over the last year, it still has its limitations — and a player’s awareness of these limitations can have a big impact on how good they perceive their experience to be.

Cubism implements a few ways of teaching player’s about the current limitations of hand tracking on Quest.

When the player first switches to hand tracking (either at launch or mid-game), a modal informs them of some best practices, like playing in a well-lit space and avoiding crossing hands.

When a user switches to hand tracking, a modal informs them about limitations and best-practices. The “Push to Start” instruction helps teach new users that buttons can be naturally pushed in this game.

It is important to acknowledge that most people are likely to immediately dismiss modals like this or quickly forget its guidelines, so signaling why things can go wrong during the experience is also important.

In Cubism, hands will turn red to signal when tracking was lost. In some playtests, people would keep one hand on their lap and play with the other, and be puzzled why their lap hand would appear frozen. To help inform cases like this, a message is displayed on the hand to clearly state why the hand is frozen if tracking loss persists. If tracking is lost specifically because the player is crossing their hands, the message changes to inform them not to do that.

Left: hands turn red when tracking is first lost. Middle: when tracking loss persists, a message informs the player about what is going on. Right: if tracking is lost due to occluded hands this is also indicated

For more seasoned players, or players who prefer playing with one hand, this feature can be replaced in the settings by having hands fade out when they lose tracking instead, more closely resembling the behavior in the Oculus home menu.

The red hands and warning messages can be replaced in the settings by fading hands.

Future Work

Hand tracking on Quest still has its limitations, and though Cubism’s support for it is already in its second version, there is still plenty of room for improvement.

Regardless, I’m excited to start exploring and supporting these new input methods. In the short term, I think they can help make experiences like this more accessible and easier to share with new VR users.

Mixed reality footage captured on an iPhone with Fabio Dela Antonio’s app Reality Mixer gives an idea of what it may be like to play Cubism on an AR headset in the future.

In the long term, there seems to be a good chance that hand tracking will be the go-to input for future standalone AR devices, so hopefully this update can be a first small step towards an AR version of Cubism.


If you enjoyed this look at at the hand-tracking design in Cubism, be sure to check out Thomas’ prior Guest Article which overviews the design of the broader game.

The post Case Study: The Design Behind ‘Cubism’s’ Hand-tracking appeared first on Road to VR.

New Cubism Update Is An Excellent Showcase For Hand Tracking On Quest

Cubism’s latest update adds hand tracking support to the full game on Quest and Quest 2 and it works incredibly well.

Cubism, which launched midway through last year, is a fantastic puzzle game available on PC VR and Quest. The concept is simple — fit the pieces into the 3D wireframe with no gaps and nothing sticking out — but it’s one that strikes a perfect balance between being easy to understand and challenging to solve.

Before its full launch, a short demo of Cubism was available to sideload with basic hand tracking support. However, when it arrived on the Oculus Store, the game only supported Touch controllers. While they worked well enough, it sometimes felt a bit clunky to position the controllers in the right orientation when placing a piece.

Now, Cubism offers a more natural solution with hand tracking for the entire game and the implementation is better than ever.

As with all hand tracking on Quest headsets, it’s not perfect. However, Cubism has designed its hand tracking support to be as user-friendly as possible. An introductory dialogue box educates you on how to get the best performance — playing in a well lit space, not crossing you hands over each other, etc. — and from there it’s very simple. Grab pieces and place them in the desired position.

What sets Cubism apart is the intelligent design safeguards in place to account for unavoidable technological shortcomings. The tips of your fingers will light up when grabbing a block to indicate which specific fingers are being tracked as part of the ‘grab’ action. This adds a lot of clarity, as it lets you know exactly which fingers to release when you want place a block accurately.

Likewise, your hand will go red when it’s occluded by the other or when it’s in a position that’s difficult to track accurately. It’s a subtle feature, but one that makes it incredibly clear when you need to adjust something to improve the experience.

It’s these little additions that make Cubism’s hand tracking feel more refined than it would otherwise. It works within the limitations of the current technology, not in spite of them.

There’s still bumps in the road — sometimes a piece will get ‘stuck’ on your hand after releasing, and sometimes it’s hard to twist your hand to the right angle without encountering tracking hiccups. However, it’s definitely one of the more natural hand tracking experiences on the Quest — it feels like a perfect fit for the game and not just a gimmick.

The hand tracking update is available now for Cubism on Quest. Hand tracking is not supported for PC VR, so the Rift version of the game will only support Touch controllers.

You can read our full Cubism review for more details.

Cubism Adds Oculus Quest Hand Tracking & More

Cubism

It’s been six months since 3D block building puzzler Cubism arrived on Oculus Quest, offering its simple yet addictive gameplay. Today, indie developer Thomas Van Bouwel has released the next major update to Cubism, adding hand tracking support as well as other content improvements.

Cubism

Whilst the original SideQuest demo of Cubism did feature hand tracking, that wasn’t implemented into the official Oculus Quest release until now. On that first iteration van Bouwel notes in a statement the: “initial version didn’t properly take the limitations of the technology into account, meaning the quality of people’s experience would vary highly depending on their lighting conditions and their expectations of the feature.”

As for the improvements the developer has made to get the hand tracking to this stage, these include:

  • Puzzle pieces can be normally grabbed with fingertips instead of needing to be “pinched”
  • Buttons can be naturally pushed instead of needing to be “pinched”
  • Hand motion is smoothed out to prevent shaky-ness, which also results in more stable looking grabs
  • Fingers lock in place once an object is grabbed to give the appearance of a stable grip
  • Fingertips are coloured according to the puzzle piece you’re interacting with
  • Additional audio cues indicate when a piece is grabbed or a button is pushed
  • The game starts with a disclaimer to inform players on ways to improve the hand tracking experience (playing in a well-lit area & not crossing hands)
  • The hands indicate when they have lost tracking by fading to red, and inform the player if this is caused by them crossing their hands
Cubism

“Hand tracking still has its limitations to be sure,” he adds, “but I’m excited to start supporting these new input methods that will hopefully make games and experiences like this more accessible and easier to share with new VR users.”

In addition to the hand tracking the update also features 5 new classical piano pieces by Robert Schumann and Pyotr Ilyich Tchaikoyvsky and repositioning of the play area by pressing down on either joystick.

Cubism became one of VRFocus’ favourite puzzle titles of 2020, saying in its review: “Here you have a good example of why indie developers are such a vital part of the VR industry. Cubism would never have been made by a big studio yet it deserves your attention nonetheless.”

VRFocus will continue its coverage of Cubism, reporting back with the latest updates.