Blocks-Projekt: Super Mario World in der virtuellen Realität

Googles kostenloses Blocks ermöglicht auch Einsteigern, schnell und einfach 3D-Modelle in der virtuellen Realität zu bauen. Vladimir Ilic aus Pforzheim hat sich nun dem Super-Nintendo-Klassiker Super Mario World vorgenommen, um die Oberwelt nachzubauen. Er präsentiert sein Ergebnis im Video und auf diversen Screenshots.

Super Marion World in VR mit Blocks

Über Sinn und Unsinn lässt sich zwar streiten, aber die Möglichkeiten, die Googles Software Blocks offenbart, sind faszinierend. Der Pforzheimer Vladimir Ilic, der unter Artstation als VRHuman fungiert, veröffentlichte ein YouTube-Video, das den Entstehungsprozess eines besonders schönen Projektes dokumentiert. Ilic baute kurzerhand die Oberwelt aus Super Mario World nach und zeigt bei der Modellierung viel Liebe zum Detail. Allerdings ist die Klötzchen-Arbeit nur eine Vorstufe, denn das Modell landet anschließend in der Spieleentwicklungs-Umgebung Unity, um interaktive Elemente zu realisieren. Die sind mit Blocks derzeit nicht möglich. Weitere Bilder stellt Ilic auf seiner Seite auf artstation.com zur Verfügung.

Super Mario World VR

Es ist nicht das erste Experiment dieser Art: Der 3D-Künstler Jarlan Perez erschuf in nur zwei Wochen ein eigenes Projekt in Blocks. Perez nutzte für das interaktive Finishing des VR-Rätselspiels allerdings nicht Unity, sondern die Unreal Engine. Das Projekt Blocks Isle lässt sich kostenlos für die HTC Vive von dieser Seite herunterladen, das gesamte Projekt hat Perez in seinem Blog dokumentiert.

Super Mario World VR

Erst jetzt hat Google seine 3D-Software Blocks auf die Version 4.0 gehoben und einiges an Anwender-Feeback umgesetzt. So gibt es jetzt beispielsweise verbesserte Snap-Funktionen sowie neue Hintergründe. Die Google-Software lässt sich kostenlos für Oculus Rift und HTC Vive bei Steam herunterladen, außerdem steht sie für die Rift im Oculus-Store bereit.

(Quelle und Bilder: Artstation, Vladimir Ilic)

Der Beitrag Blocks-Projekt: Super Mario World in der virtuellen Realität zuerst gesehen auf VR∙Nerds. VR·Nerds am Werk!

Check Out This VR Sculpture Of Super Mario World Made In Google’s Blocks

Check Out This VR Sculpture Of Super Mario World Made In Google’s Blocks

I’ve always wanted to visit the Mushroom Kingdom. Ever since I was a kid watching the Super Mario Bros. Super Show, despite how terrible that show is nowadays in hindsight, the world just seemed like such a magical place. VR artists are making my childhood dreams come true because not only can I visit Zelda’s Hyrule already, but now the Mushroom Kingdom is getting the VR treatment as well.

A VR Art Director on Twitter by the name of Vladimir Ilic (otherwise referred to as VRHuman) recently published a video of a sculpture he did that completely recreates Super Mario World’s map in VR using Google’s Blocks 3D modeling app. It’s truly impressive.

Over the past several months we’ve seen a lot of creative uses for Google’s Blocks, such as this developer that built a small game in just a matter of weeks. Ilic’s rendition of Super Mario World’s map is remarkably accurate, as he built it by referencing screenshots every step of the way. You can see the whole process in the YouTube video up above.

Although Nintendo is far from interested in VR at this time, it’s still great to see their properties getting re-imagined in fun and inventive ways by dedicated artists.

Let us know what you think of this project down in the comments below!

Tagged with: ,

Google ‘Blocks’ Update Matures Tools for Even Easier 3D Modeling in VR

Google Blocks, the 3D object creation tool for VR/AR app developers, just got a big update that includes new environments, improved object snapping, and a number of experimental features that the company says makes Blocks “more powerful and even easier to use.”

Released to Rift and Vive users for free back in July, Google intended the creation experience in Blocks be more akin to playing with children’s blocks than working with traditional 3D modeling software like Z-Brush or Blender. Giving you a simple set of shapes, a color palette, and a number of simple tools, the user can easily create objects from within VR that you can then export to a standard OBJ file for inclusion in other 3D creation software.

Now, Google has added a few more features to make creation a little more hassle free.

Background environments now include a night version of the desert environment, and also plain white and black background options. An improved snapping function has also been included, which the company says address the “often unpredictable or difficult to control” snapping tool from the previous build. The company says they’ve “vastly improved [their] snapping algorithm and introduced a brand new user experience to guide you.”

SEE ALSO
Google is Developing a VR Display With 10x More Pixels Than Today's Headsets

Google has also added a host of experimental features, many of which are responses to user requests. For more information on how to use each tool, check out Google’s official blog post announcing the update.

New Features

  • Non-coplanar face mode: Now you can enable non-coplanar faces to avoid creation of extra triangles.
  • Loop subdivide: Subdivision can be a really powerful tool. It’s even more powerful if you can cut a loop around an entire mesh. With loop subdivide enabled, simply long press on the trigger while subdividing to see a perfect subdivision loop form around your object.
  • Edge, Face and Vertex Deletion: With this feature enabled, you can use the eraser tool to delete a single edge, face or vertex. We’ll “collapse” the mesh based on the edge, face or vertex you delete.
  • Worldspace grids: This feature will show grids along every side of your worldspace. The grid units are equivalent to the actual worldspace grid units, so you can precisely measure and place objects along the grids.
  • Volume insertion ruler: This experimental feature allows you to enable a ruler when you are inserting a mesh. As you insert the object, you’ll see relative measurements in meters appear on each axis so you can precisely and accurately measure every object relative to the others.
  • Expanded mesh wireframe: When reshaping a mesh you see a helpful wireframe around the section of the mesh you are reshaping. Now, you can turn that wireframe on for the entire mesh.
  • Stepwise selection undo: You can now undo and redo steps in your multi-selection. You can use the undo and redo buttons on your non-dominant controller to undo or redo the selection of objects in order.

Check out Google Blocks on Oculus Home (Rift) and Steam (Vive and Rift).

The post Google ‘Blocks’ Update Matures Tools for Even Easier 3D Modeling in VR appeared first on Road to VR.

Verknüpfung für Kreativ-Apps: Google veröffentlicht Poly API und Poly Toolkits

Mit Poly hat Google vor einem Monat eine Bibliothek für 3D-Objekte veröffentlich. Zwei Wochen später folgte die Integration der Bibliothek direkt in Tilt Brush, sodass man das Programm nicht verlassen muss, wenn man ein Objekt einfügen will. Nun geht Google mit der Veröffentlichung von der Poly API und den Poly Toolkits für Unity und der Unreal Engine noch einen Schritt weiter.

Poly API und Poly Toolkits für VR-Erschaffer

Langsam aber sicher baut Google seine Kreativwerkzeuge für das Erstellen von virtuellen Welten direkt in der VR aus. Das 3D-Malprogramm Tilt Brush begeistert viele Anwender schon lange, danach folgte mit Blocks eine Lösung, um einfach 3D-Objekte in der VR zu erstellen. Die lassen sich beispielsweise auf Poly hochladen und dann in Tilt Brush weiterverwenden.

Dank der Poly API öffnet sich das Google-Universum auch anderen Apps: Mit ihr kann in Zukunft – wenn die Entwickler sie einbauen – jede VR-Software lesend auf die 3D-Objekte in der Poly-Bibliothek zugreifen und sie laden. Die Poly API ist auch Grundlage für die Poly Toolkits, die Google für Unity und die Unreal Engine anbietet und die noch einige Aufrufe enthalten.

Poly API

Auch wenn man über die Schnittstelle die 3D-Objekte auf Software für den Desktop holen kann: Ziel  ist es, Inhalte für VR direkt in der virtuellen Realität zu erstellen und einen möglichst einfachen Zugang zum kreativen Schaffen zu ermöglichen. Außerdem soll man alle Arbeiten in der virtuellen Umgebung erledigen können, ohne sie verlassen zu müssen.

Poly API

Erst kürzlich berichteten wir über ein Google-Projekt, bei dem ein 3D-Künstler innerhalb von zwei Wochen ein VR-Spiel entwickelte. Jalan Perez nutze dafür die Software Blocks sowie die Unreal Engine. In Zukunft kann er also seine 3D-Objekte direkt in der Spiele-Entwicklungsumgebung einbauen, ohne sie erst umständlich ex- und importieren zu müssen. Das Video zeigt zudem, dass auch andere Software die Schnittstelle nutzt oder nutzen wird: Beispielsweise die Spiele-Entwicklungsumgebung Modbox , die soziale Musik-Plattform TheWaveVR sowie die Zeichnen- und Animations-Software AnimVR.

(Quellen: Google und Upload VR)




Der Beitrag Verknüpfung für Kreativ-Apps: Google veröffentlicht Poly API und Poly Toolkits zuerst gesehen auf VR∙Nerds. VR·Nerds am Werk!

Blocks: 3D-Künstler entwickelt interaktives VR-Spiel in zwei Wochen

Das Modeling-Tool Blocks dient zur kreativen Schaffung von VR-Inhalten für Personen auch ohne Vorkenntnisse in diesem Bereich. Dabei entwickelt Google die Software stets weiter, um den Einstieg und die Nutzung so simpel und intuitiv wie möglich zu gestalten. Vor kurzer Zeit luden sie den 3D-Künstler Jarlan Perez ein, um in einem gemeinsamen Projekt ein VR-Spiel in nur zwei Wochen zu erschaffen.

Google Blocks – Intuitives VR-Spiel in zwei Wochen

Google beweist mit ihrem neuesten Projekt, wie einfach der Einstieg in die Entwicklung von VR-Inhalten mit ihrem Modeling-Tool ist. In nur zwei Wochen erschuf der 3D-Künstler ein interaktives VR-Rätselspiel mit Blocks und der Unreal Engine 4. Dafür entwickelte er verschiedene Konzepte, wobei er sich letztlich für die Schaffung einer Insel bestehend aus verschiedenen Einzelteilen entschied, auf der die Spieler verschiedene Rätsel lösen können. Mit Blocks modellierte er so die Objekte für sein Projekt. Anschließend startete der Künstler jedoch sein gewohntes 3D-Programm, um den Objekten unter anderem UV-Texturen mitzugeben, bevor die Interaktion hinzukam. Von dem Prozess zeigte sich Perez begeistert:

Blocks ist ein großartiges Tool zur Erstellung von spartialen Protoypen. Bevor ich mein Projekt in Unreal importierte, vergrößerte ich meine Insel und simulierte die Fortbewegung. Das half mir zu verstehen, wie die Spieler später durch das Spiel navigieren werden und sparte mir dadurch eine Menge Zeit.”

Google-Blocks-Blocks-Isle-HTC-Vive-Perez

Doch Perez wollte nicht nur eine simple Umgebung schaffen, sondern dem Ganzen eine Portion Interaktivität verpassen. Deshalb entschied er sich zusätzlich zur Nutzung der Unreal Engine 4, da er diese seiner Ansicht nach auch mit minimalem Wissen benutzen konnte. Entsprechend implementierte er seinen Prototyp und verwendete Blueprints zur Integration von Logik und Interaktivität.

Die einfache Geschichte hinter dem Spiel besteht darin, euren Freund Jasper auf der mysteriösen Insel Block Island zu finden, nachdem ihr wochenlang kein Lebenszeichen von ihm erhalten habt. Auf der Insel wurden allerlei Logbucheinträge und Nachrichten eures Freundes verstreut, die ihr finden müsst. Insgesamt 20 Minuten Spielspaß bringt das VR-Spiel mit sich.

Google-Blocks-Blocks-Isle-HTC-Vive

Das Resultat des Projekts Blocks Isle ist kostenlos hier für die HTC Vive erhältlich. Die komplette Dokumentation des Projekts von Perez befindet sich in seinem Blog.

Dank solcher kreativen Experimente versuchen die Entwickler von Google Blocks, die Einstiegshürde in die VR-Entwicklung noch weiter zu reduzieren. Dadurch soll jedermann problemlos Freude an der Entwicklung seiner persönlichen VR-Inhalte haben.

(Quellen: Upload VR | Perez Blog)

Der Beitrag Blocks: 3D-Künstler entwickelt interaktives VR-Spiel in zwei Wochen zuerst gesehen auf VR∙Nerds. VR·Nerds am Werk!

Made with Unity Showcase at Unite Austin 2017 to feature Blocks, Pixel Ripped 1989 and More

Unity Technologies Unite Austin 2017 event will be taking place in October. The software company will be hosting its usual Made with Unity showcase and because Unity is one of the most popular videogame engines for virtual reality (VR) development several VR titles have made the cut. 

Made with Unity is an area of the conference dedicated exclusively to projects created by the Unity community, where attendees can take a break from the various talks taking place to play some videogames.

Pixel_Ripped_1989_MWU_Showcase_Austin_image1-8

There’s going to be plenty of VR titles on show with Unity choosing: Blocks by Google, Cadillac Virtual Showroom by All Things Media, I Expect You to Die by Schell Games, Pixel Ripped 1989 by Pixel Ripped, To the Top by Electric Hat Games and Voltron VR Chronicles by Digital Domain and DreamWorks.

In addition to those titles VR specialist Fast Travel Games will be showcasing a currently unannounced project for visitors.

Unite Austin 2017 runs from 3rd – 5th October in Austin, Texas. Full conference tickets are on sale now for $500 USD, with student/educator tickets on sale at $175.

For further Unity updates, keep reading VRFocus.

Update von Tilt Brush integriert Blocks und bietet neues Auswahl-Tool

Googles Tilt Brush für die Oculus Rift und HTC Vive gehört zu den ersten und faszinierendsten Kreativ-VR-Apps. Mit der Software können Anwender direkt in der virtuellen Realität ihre Kunstwerke erstellen und sogar begehen.  Ebenfalls von Google stammt Blocks, mit dem sich sehr einfach 3D-Polygonobjekte in der virtuellen Realität erstellen lassen. Mit dem ab sofort verfügbaren Update v14 von Tilt Brush baut Google eine Brücke zu Blocks. Polygon-Objekte lassen sich jetzt direkt in Tilt Brush anzeigen dort weiterbearbeiten.

Tilt Brush goes Blocks

Das 3D-Malprogramm für Künstler in der virtuellen Realität Tilt Brush erhält mit dem Update v14 Zugriff auf die Blocks-Galerie: Die 3D-Objekte lassen sich somit direkt in Tilt Brush verwenden. Alle Eigenkreationen, die man auf vr.google.com/objects veröffentlich hat, stehen automatisch im Malprogramm zur Verfügung. Um Modelle anderer Anwender zu nutzen, reicht es, auf der Google-Seite die gewünschten Kreationen zu liken. Google zeigt auch erste Beispiele, was mit der Kombination möglich ist.

Neues Auswahl-Tool erfüllt Anwenderwunsch

Laut Google haben sich Anwender das neue Select-Tool gewünscht. Mit ihm lassen sich Pinselstriche sowie Objekte auswählen und beispielsweise duplizieren. Das Tool befindet sich neben der Radier-Funktion. Um etwas auszuwählen, benutzt man den Trigger der Touch Controller von Oculus oder HTC. An Pro-Anwender richtet sich laut Google das ebenfalls neue Quick Tool, mit dem sich ein Menü für den schnellen Zugriff auf diverse Tools öffnet. Somit brauchen Kreative nicht in die Palette zurückkehren, was eine Menge Zeit sparen dürfte. Gleichzeitig hat Google die Oberfläche des Malprogrammes überarbeitet. Wer etwas vermisst, sollte unter dem neuen „More“-Menüeintrag nachschauen. Einige Tools haben die Entwickler dorthin verschoben. Weitere Funktionen findet man außerdem unter dem „Labs“-Eintrag.

Tilt Brush ist im Steam Store für die Oculus Rift und HTC Vive erhältlich, Rift-Besitzer finden sie auch im Oculus Store. Die Software kostet regulär 20 Euro. Blocks ist ebenfalls auf Steam zu finden und kostenlos zu haben.

(Quelle: Google, Updatebeschreibung)

Der Beitrag Update von Tilt Brush integriert Blocks und bietet neues Auswahl-Tool zuerst gesehen auf VR∙Nerds. VR·Nerds am Werk!

Tilt Brush Gets Blocks Integration And More With New Update

TiltBrush Blocks

Google updated their art application, Tilt Brush today. The new version includes several small changes- such as tweaks to icon location, and the movement of several tools to the catch-all “more” category in the menus- and three significant new features.

Tilt Brush users can now import objects from another Google VR art program, Blocks. Announced in July, Blocks allows users to combine and modify a variety of pre-set geometric shapes. As we commented in our original story, Tilt Brush requires the steady hands of a painter or even a surgeon, but in Blocks, you just drag, drop, and modify.

After importing Blocks into Tilt Brush, you can make your changes, and then share them on the web to vr.google.com/objects. You can find others’ Blocks design at the same URL- hit the like button, and it’ll be imported into your Tilt Brush library. We’ve embedded a video from the Tilt Brush Twitter feed demonstrating the new functionality below.

The improvements don’t stop with Blocks integration. Google also added in a new “selection” tool that allows artists to duplicate strokes and models quickly. According to the patch notes, it works like this:

• “Selection” is found next to the “Eraser” button on the Tools panel.
• Select strokes using the trigger. You can also flip into deselect mode by tapping the thumbpad (Vive) or “A/X” button (Rift).
• If you want to move your selection, grab it as you would any other model in your scene. To delete a selected group of strokes, just throw it away.
• Duplicate selections by tapping the thumbpad (Vive) or “A/X” button (Rift) while intersecting your controller with selected strokes.

Finally, the Tilt Brush team added a “quick tool” menu, allowing artists to swap tools without having to go back to the palette.

You can check out the release notes here. We’re excited to see what artists can do with Blocks integration, and Google has promised to share some of the best works on their Twitter. If you’re an artist with the hardware, feel free to share your own amazing creations with us.

Tagged with: ,

The Coming Wave of Virtual Reality Creation Tools

The Coming Wave of Virtual Reality Creation Tools

I’ve always prized creativity. As a developer, writer and musician, it’s what drives me. But as much as I value creativity in my work and free time, I have to confess I’ve never really felt compelled to make visual art. That’s not to say I don’t apply a design eye in my web work or that I don’t have opinions about style — it’s just that my creativity manifests in other ways. I considered this a blind spot and never felt inspired to develop those skills.

That was true until a few weeks ago, when I got my hands on a new class of virtual reality 3D creation tools including Google BlocksGoogle Tilt BrushOculus MediumOculus Quill, et al. Each in its own way, these apps try to fundamentally change the process of 3D art and asset creation — modeling, sculpting, painting, etc. — moving away from two-dimensional computer screens and into room-scale three-dimensional VR space. Even in their infancy, these tools have already inspired me to think and work and create in entirely new ways.

The Old Way & The New

To contrast, think about 3D content creation happening on a standard 2D computer monitor. The simplest task, like rotating your subject, requires knowledge of a labyrinthine software suite and its many unique series of hotkeys, scrolls, mouse clicks, drags and so on (to say nothing of more complex modeling and sculpting tasks/techniques).

In new VR creation tools like my personal favorites, Google’s Blocks and Tilt Brush, you can literally step around what you’re working on, or reach out and grab it, physically manipulating and scaling objects effortlessly in real-time. It’s a revelation — a quick trailer:

There are true masters of the old way, for sure. But the barrier to entry is super high — in addition to the already-Herculean task of having to be being artistically inspired. Why shouldn’t this process be more natural, spatial and physical?

The work already being done with this technology by artists and developers at the highest levels is astounding:

 

My Process

So, what could I possibly do with it? Aside from a couple Blender tutorials, I’ve got no experience making 3D models (just pushing them around in Unity) and no real visual arts experience, period.

I’m an amateur, but as soon as I picked up the palette and drawing hand in Google Blocks, it just made sense. Versus any 2D interface I’ve encountered, it’s incredibly intuitive and fun and compelling to model characters, objects and scenes in real space. While you can import models from Google’s crowd-sourced, public repository to construct a scene, I wanted to try my hand at some totally, completely original work.

Another option using VR creation tools would be to bring my unpainted model into another app for freehand painting with a wider array of brushes, effects and color options. But ultimately, my drawing is lousy, I preferred the symmetrical simplicity of the low-poly patterns for this design and for this exercise I wanted to experiment solely in Blocks.

Spent hours peering down my nose at a 2nd screen before noticing the very helpful “Add reference image” option.

I spent two-to-four hours (time flies) on my first-ever 3D model. It was a truly exciting learning experience. Considering this is a brand new tool exploring brand new UX parameters, I admire what the Google team has already accomplished and look forward to continued improvements.

So, that’s it, right? I made a model — what now? Ah-ha, but you can publish your models to Google’s public repository so they can be downloaded and reused in other people’s scenes. Check out my full BB-8 model here which you can inspect, manipulate, download and remix yourself.

Even more powerfully, you can export Blocks models and scenes in .obj format — with materials included in .mtl — for use elsewhere, including other creation tools, 3D engines, VR/AR and 3D printing. For me, this is where Unity comes in.

For the uninitiated, Unity is the game engine that powered last year’s smash hit Pokemon Go! and many thousands of indie and high-end titles besides. It’s extremely easy to use while also being spectacularly powerful and customizable. I started learning Unity in C# two years ago (despite or perhaps because of my years in web development, I steered clear of the JavaScript API) and I’m still regularly blown away by the amount of free, amazing resources available to Unity developers.

I’m using an Oculus Rift, so my path of least resistance in Unity is to prototype a scene using the Oculus Sample Framework from the Unity Asset Store (another Unity strength, that asset dispensary). This package provides many of the core prefabs and scripts you’ll need to put together a totally custom Oculus app. It’ll give you access to an API to program responses to HMD and Touch controller input data, as well as standard models to visualize HMD and controller tracking in real-time.

Once those controls were in place, my initial game logic goal was simple: BB-8 should roll to wherever I’m pointing.

Again, Unity is so powerful and these use cases so well-tread that certain aspects of this are practically free to implement. By establishing a NavMesh on the game floor and adding a NavMeshAgent component to BB-8, he instantly knows how to move about the terrain once given a goal destination, navigating obstacles and even coordinating with other moving NavMeshAgents in real time.

Since his goal destination should be wherever I’m pointing to, I just need to do the following:

  1. detect when the pointing gesture is being made
  2. when that happens, get the transform value at the end of the raycast extending X units (arbitrary) from the pointer finger. If the raycast hits the ground, get the transform of the collision point
  3. set BB-8’s NavMeshAgent goal destination to that transform
  4. apply some rotation animation to BB-8’s ball-body while he’s moving, maybe with a little dust trail effect at his base

Interestingly enough, I don’t think there are direct API methods to detect common Touch controller hand gestures (like making a fist or pointing). Instead, I had to monitor several inputs at once for the magic combination that make up the gesture, as done in a rudimentary way below:

private bool IsPointing () {
// if (!OVRInput.Get(OVRInput.Touch.PrimaryIndexTrigger)) { Debug.Log(“No PrimaryIndexTrigger touch detected, could be pointing!”); }
// if (OVRInput.Get(OVRInput.Touch.PrimaryThumbRest)) { Debug.Log(“PrimaryThumbRest is being touched!”); }
if (!OVRInput.Get(OVRInput.Touch.PrimaryIndexTrigger) && OVRInput.Get(OVRInput.Touch.PrimaryThumbRest)) return true;

else return false;
}

So I can call this in Update() and, if it returns true, ask BB-8 to roll to wherever I’m pointing. To get him to the goal destination, I just have to access his NavMeshAgent component and assign it to a transform in the inspector.

// in our declarations:
public Transform goalDestination;
// to store transform data for BB-8’s goal destination.
public GameObject bbBody;
// allows us to access BB-8’s body for rotation control
Update () {
UnityEngine.AI.NavMeshAgent agent = GetComponent<UnityEngine.AI.NavMeshAgent>();
if (IsPointing()) {
agent.destination = goalDestination.position;
}
if (agent.remainingDistance > 0.25f) {
bbBody.transform.Rotate (agent.velocity.magnitude * bbRot * Time.deltaTime, 0f, 0f);
}
}

The last conditional is a primitive rotation for BB-8’s body (but not his head, which I’ve separated out as another GameObject in my hierarchy), which will cause it to roll forward on its X whenever BB-8 has a goal destination but hasn’t arrived there (i.e. the NavMeshAgent is moving). The amount of rotation will be roughly determined by the velocity BB-8 is moving at, which we can easily set in the editor via NavMeshAgent (some braking speed is also conveniently baked in).

That roughly takes care of my simple logic goals. Like, it’s probably wise not to test the notoriously litigious Lucasfilm organization further, right?

Oh, what the hell, I can’t resist. This is a great opportunity to experiment with another new Unity feature: Windows 10 Voice Recognition functionality!

Unity now offers a native integration of Microsoft’s speech recognition tech, which can run locally with no internet connection — as opposed to a Apple’s Siri, Amazon’s Echo or Unity’s other native voice recognition APIs, which offshore your vocal input for processing in high-powered, highly-optimized language processing machines in the cloud.

Here’s some code, but Michael Moiropoulos at Lightbuzz did a much better tutorial on this than I could manage, so head there for details on integrating Unity and Windows 10 speech recognition.

It’s an impressive advancement. In my early experiments, I’ve had to allow for a fairly low confidence threshold and I’m unsure about the total load this locally-driven processing puts on your machine (no time to run tests), but I really enjoyed being able to plug keywords in and immediately start coding game logic for them. I set up custom responses to “stop” and “go” so that BB-8 will only move to where I’m pointing if I give him the vocal command and added “return” logic to make him come back to the player’s position.

Since BB-8 now has to listen to me, it’s only fair that I let him get a word in edgewise. To the Google Machine we go! Predictably, I found a big cache of questionably-sourced Star Wars sounds, downloaded them, and did not think twice about it because BB-8 deserves a voice. “Hi/hey/sup/thanks” now all prompt vocal responses from BB-8, as do the movement commands.

Instagram-Ready

Game logic aside, BB-8 could use some sprucing up. I started by adding some colored spotlights around his various LEDs, positioning them in relative space above the model. But the real magic happened when I worked through 

Back to Reality, Temporarily

At this point, BB-8 is alive (the big reveal in a moment) and we’re best friends. I’ve spent hours staring into his bulbous black visual sensors, up close and personal. We’ve had lengthy conversions, discussing this and that. He’s as real as I could possibly have made him without knowledge of advanced robotics.

But something more may yet be done! I feel BB-8 needs a physical manifestation of some kind, and it just so happens that right down the street  is the University City Public Library. They have a 3D printer available on-demand for the fine sum of one US dollar per hour (yes, $1/hr). It’s a Makerbot Replicator Mk5, and while it’s not exactly top-tier as 3D printers go, it’s available to every man, woman and child in the community for a super low cost. As these creation tools, VR experiences and 3D printing technologies mature alongside one another, publicly-available tech resources like this will be so critical. Behold, BB-3DP!

The Promised Big Reveal

What’s all this been building to? Well, I’ve become an evangelist for these technologies in my personal life and work— and like any good evangelist, I need to spread the word.

I’m now hosting monthly XR office hours in the studio at my workplace to allow co-workers to experience high-end VR, many of them for the first time. Last week at the inaugural event, I demoed these creation tools and my BB-8 mini-game process alongside other Oculus experiences — plus live 4k mixed reality camera tracking, mobile VR, and samples of Apple’s upcoming ARKit functionality. I’m pleased to say it was a big hit and has already started generating a lot of client-facing ideas for the company. Even BB-3DP showed up to watch the curtains pulled back.

Well, let’s have it then:

Conclusions

To be able to work with Google Blocks at such an early stage feels like a privilege. I can see this tool and others like it really coming into their own in the next few years and completely redefining 3D creation workflow as we’ve known it.

Masters of the old way — 3D workflow on 2D screens — will be remiss to give up their finely tuned hotkey combos, and that’s fine. But for coming generations of XR-native 3D designers, artists and developers, I truly believe this more spatial, more physical creation process will become industry standard as tools and best practices mature with time. For now, there are things I wish Blocks could manage that it can’t — and a few suggestions are appended to this section.

What it can’t do (yet), others are attempting. More creation tools are pouring into the marketplace daily, with Gravity Sketch being the latest entrant (now available in early release on Steam). UK-based developer Tim Johnson is working on his own creation tool in his spare time, and it already does a few nifty tricks Blocks can’t yet handle.

Ultimately, while standalone 3D asset creation apps are great, to truly unlock the power of your work— to make it do something — you still have to export it, right?

That’s why, to me, Blocks- and Tilt Brush-like functionality would be most powerful as native Unity or Unreal 4 feature-sets — an API or plugin for use directly within these editors, where I can really make magic happen as a developer.

 

Perhaps there’s room for Google and Unity to partner on this — or perhaps I’m hopelessly naive for even suggesting such a thing? No idea. Regardless of feasibility, bringing all the functionality of Blocks/Tilt Brush directly into Unity itself would be an amazing achievement and one I’m starting to ponder the how-to’s of.

My questions for the Google teams would be around goals for Blocks and Tilt Brush. Are these fun tools/toys or is this just the start of a grander vision, some kind of unified VR editing suite that can totally redefine 3D creation best practices as we know them? Their blog post detailing new VR animation tools built for Blocks during a recent hackathon shows the team is thinking creatively — but what is the long-term strategy?

Here are a few suggestions from an outsider’s POV. Maybe these are unrealistic goals for reasons I don’t understand, and maybe I am using poor terminology, but fwiw:

Shape/object grouping hierarchy UI — create a panel to manage shape/object grouping, a la the layers menu in Photoshop

Symmetrical extrusion — select multiple faces & extrude simultaneously

Subdivison deletion — ability to delete subdivisions (besides undo). How about… while the pliers tool is selecting a specific subdivision, hitting undo deletes it? Or an option of the eraser that switches to subdivision deletion instead of shape deletion?

More snap-to options — ability to toggle through a few different snap-to guesses. Sometimes it’s intuitive, sometimes uncooperative

Vert & face welding — demonstrated in Tim Johnson’s work, among other things

General 2D Windows app frame insertion into VR workflow (beyond “Add reference image”)

Voice commands — if 3D modeling is becoming more physical, why not involve speech? Stuff like “rotate object 180 degrees” while selecting something… maybe

Postscript: Thanks to the Community

I have to give a shout out to the Twitter and web/blog communities surrounding VR/AR. Working on XR solo at a small company in the Midwest, I haven’t had a lot of local resources to lean on while trying to shift from web/app development (though that is starting to change). But thanks to Twitter and the incredibly supportive community there, I still feel super connected to what’s going on in XR on a daily basis. Everyone is so excited and energized by this technology and so eager to share their knowledge. You can really feel the common sense of purpose: spread the word about XR.

Special thanks to the project manager working on Blocks, Brit Menutti (@britmenutti). Just the fact that someone on this team, let alone the PM, was willing to respond to FAQ tweets from a pleb like me was inspiring. @_naam from Twitter’s videos are some of the best work I’ve seen using Blocks/Cinema 4D/Unity. Mind-blowing. 3Donimus’ Blocks models are… impossibly complex to me. A true artist. Jarlan Perez is also in that category. Thanks again to Danny Bittman and Tim Johnson. Stephanie Hurlburt works on shader stuff I don’t yet understand but is super positive and encouraging to people looking to break into the tech industry.

I went to Unity’s Vision VR/AR Summit in Hollywood, CA, this past May and saw a lot of speakers who similarly inspired me to keep at this (even before I had a real HMD in my hands). Some of my favorites were Kat Harris and Estella Tse, and Unity’s evangelist from Australia John Sietsma ended up providing me with a great jump-start on my 360-degree video work.

There are many others, but I won’t belabor the point. Thanks to you all.

James C. Kane is a developer (web, apps, XR), writer and musician out of St. Louis who’s working to win hearts and minds for VR and AR. @jameskanestl on Twitter. This post originally appeared on Medium.

Tagged with:

Google Blocks: Roboter bauen, Steam-Card gewinnen

Ein netter kleiner Wettbewerb fordert Kreative heraus, mit dem VR-Programm Google Blocks einen Roboter zu bauen. Als Gewinn winkt ein Gutschein für Steam über 50 US Dollar. Bevor man loslegen kann, müssen Teilnehmer das Robot Kit von Google-Mitarbeiter Jarlan Perez installieren. Das enthält über einhundert Teile und lässt damit viel Spielraum für eigene Kreationen. Blocks ist kostenlos für die Oculus Rift und HTC Vive erhältlich, das Robot Kit ist ebenfalls gratis.

Google Blocks: Einfaches 3D-Modelling in VR

Mit Blocks können Anwender in der virtuellen Realität auf einfache Art 3D-Modelle bauen. Die Software setzt auf Low-Poly, was einen zeitlosen und vor allem ressourcenschonenden Grafikstil ermöglicht. Im Test konnte Vladimir in kürzester Zeit Modelle in der virtuellen Realität entwerfen, die mit traditioneller Software sehr viel umständlicher zu realisieren gewesen wären. Die Ergebnisse lassen sich im .obj-Format exportieren und damit beispielsweise in Unity weiterverwenden. Damit macht die Software nicht nur Spaß, sondern kann sogar für Entwickler interessant sein. Vor wenigen Tagen hat Google zudem vorgestellt, wie sich Objekte direkt in Blocks mit verschiedenen Techniken animieren lassen. Ob und wann diese Funktionen zur Verfügung stehen werden, ist noch nicht bekannt.

Google Blocks
Die Teilnahme am Wettbewerb ist denkbar einfach: Zuerst lädt man sich das Robot Kit von Jarlan Perez herunter. Es enthält über 100 Einzelteile, aus denen man ein Robotermodell erstellen kann. Der Download fällt mit 2,6 Megabyte sehr schlank aus, die Datei liegt nach dem Entpacken im .obj-Format vor. Die Farben der Einzelteile hat Perez sehr einfach gehalten, um die spätere Anpassung zu erleichtern. Nachdem der Roboter zusammengebaut ist, speichert man das Ergebnis als Bild, Video, GIF oder veröffentlicht es in der Blocks Gallery. Wichtig: es muss den Hashtag #BotsWithBlocks tragen. Weitere Voraussetzungen: Teilnehmer dürfen ausschließlich Blocks und das Roboter Kit verwenden. Außerdem muss man unbedingt viel Spaß haben.

Die Ausschreibung begann am Montag, die Abgabefrist läuft noch bis zum Sonntag, den 20. August. Der glückliche Gewinner erhält eine Steam-Card im Wert von 50 US Dollar. Erste Modelle lassen sich bereits in der Galerie bewundern. Blocks ist als kostenloser Download für die HTC Vive und Oculus Rift bei Steam erhältlich.

Der Beitrag Google Blocks: Roboter bauen, Steam-Card gewinnen zuerst gesehen auf VR∙Nerds. VR·Nerds am Werk!