Beat Sage launched a new update that adds in more features, including support for 90 degree levels, single saber mode and obstacles. It also makes some minor improvements to the AI modelling used to generate levels, providing users with more options.
Released last month, Beat Sage is a tool that uses AI and neural networks to generate custom Beat Saber maps from any song, based off a training model that uses human-made maps to learn how to make a good beatmap.
With increasing support on Patreon, the tool’s creators have pushed an update that fulfills the rewards promised to supporters, including the ability to generate 90 degree and single saber modes, along with support for obstacles (aka walls) in your maps. The maps environment is also now changeable, allowing you to select any of the standard Beat Saber map environments or any of the DLC song pack environments as well. If you want to generate a Beethoven Beat Saber map and play it in the Green Day-themed environment, now you can!
You can also now select the version of the AI modelling that you want to use when generating a track. There’s an option for original V1 AI, the new V2 AI and an alternate ‘V2-Flow’ AI that supposedly aids flow of maps but results in them being ‘less creative’. In some limited testing, we found that the V2-Flow maps did help constrain the beatmap a bit more so that you arms aren’t wildly flying around, but also resulted in slightly more repetitive maps. It’s a trade off that will probably come down to your personal preference.
We also gave the 90 degree maps a try, and found the results were mixed. You can check out a very short snippet of a Beat Sage-generated 90 degree level, with bombs and walls enabled, above. We found enabling all 3 of these lead to some wacky (and occasionally impossible or illogical) beatmaps — there’s definitely still some issues to iron out with the new features. You might need to do some tweaking to get better results on the 90 degree maps, such as disabling walls, bombs or both.
That being said, we didn’t get to test everything comprehensively and the tool is very much in active development. The developers behind Beat Sage told us that changes will continue to be made to improve the AI, so expect these new additions to get better in the future. If you’re interested in how Beat Sage’s AI works and how it studies human maps to produce better Beat Saber maps, check out our deep dive with the developers from last weekend.
Late last month, Chris Donahue and Abhay Agarwal launched an impressive new tool called ‘Beat Sage’, which uses artificial intelligence and neural networks to automatically generate custom Beat Saber maps from any song in seconds.
We tried it out, and were left suitably impressed — the resulting tracks are fun, challenging and better than many other auto-map generators for rhythm games. With the tool still in active development, we reached out to Donahue and Agarwal to get a better understanding of what makes Beat Sage tick and how it might be improved in the future.
Getting Started
“I first tried Beat Saber in December 2019 and loved it immediately,” Donahue, an AI researcher at Stanford University, explained in an email. A month later, in January 2020, he began intermittent work on what would become Beat Sage and spoke to Agarwal, who runs an AI design firm called Polytopal and agreed to help out. The two of them worked on Beat Sage in their spare time for the next few months and launched in April.
Since launch they’ve added support for bombs and the ability to generate maps from MP3 uploads. Their next update will deliver a set of much anticipated features, including one saber maps, obstacles and support for 90 degree map generation.
The pair reiterate that their tool remains in active development with more features and improvements to the AI still to come. Beat Sage may not be perfect, but it does appear to be one of the better automatic map generators available for any rhythm game, let alone Beat Saber. At its best, it produces AI-generated maps that are engaging, nuanced and sometimes even challenging.
The Secret to AI Success
So why does Beat Sage work so well? Well, it uses an AI that has been trained on human-made maps and applies that to neural networks, allowing it to develop a custom map for any piece of music. To be clear, this isn’t just a computer determining the beats per minute (BPM) of an audio track and assigning notes and patterns that align with the beat.
Instead, Agarwal and Donahue curated a sample of select official and community-made maps which the AI used to teach itself how Beat Saber maps are constructed. It essentially tried to teach itself what works well for certain types of audio, music and instruments, based of the selection it was given.
“Beat Sage is looking at the audio spectrogram and learning (from data) to answer the question ‘how likely would it be for a human to put a block at this point in time?'” the creators explained in an email.
The resulting AI is completely self-taught and can generate a map from any audio track.
Genre Bias
However, because the model’s sample tracks mainly feature fast electronic-based music, the AI does skew itself to work better for certain types of songs. Certain genres, such as pop and electronic music, often result in better generated maps, thanks to the similarities they share with the model tracks. Other genres, such as rock music or songs with a slow tempo or sparse instrumentation, can be a real mixed bag — the result can range from entertaining to inconsistent to just boring or even fundamentally broken.
“We are aware that Beat Sage does better on some genres than others,” they explained. “For the time being, we’re reasonably happy with how the audio analysis system performs overall. Improving the audio analysis is definitely on the roadmap though.”
Donahue explained that Beat Sage “shines brightest on tracks which have faster beats and a wide variety of rhythm changes. Songs with rhythms that change frequently are more difficult for humans to map, but because Beat Sage doesn’t depend as much on tempo as humans, it tends to excel at this category of songs.”
A consistent music tempo, or BPM, is relatively unimportant when Beat Sage generates a map. In fact, the tool doesn’t really take overall BPM into consideration — instead, it looks at each slice of music individually. “It only looks at a small window of the spectrogram,” they explained. “It focuses less on tempo and more on the instantaneous presence of percussive instruments.”
A Human Touch
Beat Sage tracks often surprise you with nuanced moments that feel eerily… human. The AI can pick up on significant moments — such as a drum fill, a big vocal moment, or a big drop — and turns them into fun moments for you to play through. If a human-made map uses beat blocks to emulate a drum fill, Beat Sage will learn to integrate similar moments its own tracks.
“We didn’t program this behavior. It comes from the aforementioned tendency of Beat Sage to place blocks based on the presence (or lack thereof) of instruments as opposed to the underlying tempo,” the creators explained.
“All of these moments you mention are shocking/cool primarily because they’ve likely been “discovered” by humans first. We’re a long way away from AI learning to do these kinds of things without any training data at all. The AI would probably have to know a lot more about humans in general, what interests us, how our emotions work, how we respond intrinsically to music, etc.”
Coming Soon
Beat Sage’s next update doesn’t feature any huge overhauls to the AI system, except for a slightly “redesigned training pipeline.” The selection of curated maps used by the AI has also been slightly altered to exclude some that took advantage of non-standard game modifications that they weren’t initially aware of. “The result is that the newer model seems to be doing a bit better than our previous one on quantitative metrics. Whether this will result in a perceivable change in map quality remains to be seen.”
Donahue and Agarwal expressed interest in building the tool into a fully-fledged game modification, that could allow users to instantly generate maps from within Beat Saber itself and instantly give feedback on each map after playing it. They also plan to implement “guard rails” into the AI, which will stop it from generating blocks and patterns that lead to undesirable experiences like handclaps.
For now, the biggest roadblock is time. While Beat Sage does have a Patreon page, the tool is still very much a side project.
“We’ll see where it goes. We’re both gainfully employed, so we don’t have any immediate need to earn a living off of Beat Sage … That said, if anyone has any cool problems that they think this tech could solve, definitely reach out to us!”
It’s nearly time to jump into the cockpit of elite jets with the new Ace Combat 7 PSVR support. Before you do that, though, check out this behind the scenes video of its development.
This six-minute clip takes an in-depth look at the new game’s VR campaign, which is exclusive to PSVR. In it, Brand Director Kazutoki Kono and VR Producer Jun Tamaoki talk about the history of the project. Ace Combat 7 was first announced all the way back at the 2015 edition of the PlayStation Experience. Even then it had VR support but, as Tamaoki explains, an early version was scrapped for feeling ‘tacked on’.
Instead, the team moved to make a native VR campaign exclusive to the headset. It’s undoubtedly much shorter than the campaign in the main game but it offers a more immersive experience. We’ve gone hands-on with this mode a handful of times and we think it’s nothing less than a blockbuster game with production values rarely seen in VR
It’s pretty interesting to see the work that went into making this experience tick. A VR flight combat game seems like a no-brainer but, clearly, making it a reality was no easy task. That might go some way to explaining why there isn’t more of it. The video also gives you a detailed look at an Air Show mode, which allows you to enjoy other pilots doing fancy tricks.
Ace Combat 7: Skies Unknown launches this Friday, January 18th. We’ll have a full review of the game’s PSVR campaign around that time.
VR Visio’s Unreal Engine virtual reality (VR) shooter, Special Forces VR, has been announced for PlayStation VR, HTC Vive and Oculus Rift. Now the studio are giving us a behind-the-scenes look at the game in a new video. The developers recently announced a variety of details about the game, including a closed beta, due to go live in September 2017. The game will be compatible with a variety of peripherals too, including the PlayStation Aim controller, and a range of specialised VR peripherals, such as haptic suits and guns.
VR Visio boast; “HTC Vive, Oculus Rift and PlayStation VR gamers will soon get a chance to test this next-level multiplayer VR shooter. It is one of the first titles offering cross-platform multiplayer, allowing console and PC gamers to fight in a single deathmatch.”
In the video below, VR Visio show off behind the scenes shots of their studio, and the development of Special Forces VR. They also talk about some of the techniques they used to create the game, and what players can expect from the closed beta, coming in September.
Special Forces VR is very promising – there’s definitely room in the VR space for a great cross-platform shooter, and that’s exactly what VR Visio are promising. Perfect, given that not everyone owns the same VR headset, and PC players will now finally be able to enjoy games with their PlayStation VR pals.
If you’re interested in the Special Forces VR beta test, sign up here. For more on the game, watch the behind-the-scenes video below.
For everything on Special Forces VR and the best multiplayer VR videogames, stay on VRFocus.
The Surge developer Deck13 weighed its stuntman down during the motion capture process in order to portray heavier, more brutal combat. This was used for both slow and fast attack styles.
VR Visio boast; “HTC Vive, Oculus Rift and PlayStation VR gamers will soon get a chance to test this next-level multiplayer VR shooter. It is one of the first titles offering cross-platform multiplayer, allowing console and PC gamers to fight in a single deathmatch.”