Polyworld VR Experience for HTC VIVE & Oculus Rift featuring Leap Motion Hand Tracking

Polyworld Virtual Reality (VR) Experience

You’ve got the whole world in your hands

Learn how we created our own virtual reality (VR) experience using Leap Motion’s hand tracking technology.

A planet of possibilities at your fingertips

As children, we’re often told the world is ours to shape. At Local Wisdom, we’ve never outgrown that sentiment. Introducing Polyworld, our own VR world builder that uses the only controllers that truly feel natural: your hands. Built entirely in-house and out of pocket, Polyworld began as a passion project and ended as a proving ground for how we could help shape the future of VR.

Optimized for the VIVE and Oculus platforms, Polyworld features:

  • Hand-tracking controls: Playing Polyworld is like conducting musical terraforming with nothing but your hands. Push to carve a valley, pull to raise a mountain, pinch to pick up a tree, and more. It’s simple and satisfying.
  • First-person exploration: Want to get up close and personal with your creation? Touch the planet to land on its surface and continue shaping it from the ground up. Or relax by taking a walk on the beach and a swim in the ocean. Just remember to come up for air.
  • Environmental destructibility: No world-building game is complete without it. Don’t like the location of that lighthouse? Blow it up. Think that forest could use some thinning? Perform a few simple hand motions to set it ablaze. Be as merciless as you want to be.
  • Original music: We wear many hats at Local Wisdom. Luckily for us, “musician” is one of them. Our designers composed their own songs to complement the often serene—and often chaotic—action of global sculpture.

As whimsical, musical, and fantastical as your inner child can imagine, Polyworld puts aside your daily stresses to help you answer an exciting question: If the world was really yours to shape, how would you?

Oculus Rift plus Leap Motion


We faced unique challenges during Polyworld’s development, especially when it came to the controls and user interface (UI). Although VIVE and Oculus Touch controllers were great starting points, we wanted to push them further. Reality often pushed back, but that has never stopped us before.

Calibrating the controls

VR Interactions

Original concepts offered the player nuanced controls with a steep learning curve. We realized that what worked “on screen” or “in theory” often lead to a poor experience. The controls were complicated and limited by current VR tech.

VR interactions

We tied the controls to the location of the user’s hands. Pulling creates mountains, pushing digs craters, and so on. Extensive testing proved the age-old design theory true: Game controls should do their best to get out of the player’s way.

Designing the UI

tool selection

An early concept tied the UI to the bottom of the virtual “screen.” Although this allowed users to select tools with either hand, it was ultimately too confusing. To make matters worse, the tools were too awkwardly placed in relation to the user.

Leap Motion hand tracking

We restricted the left hand to planet rotation and added icons on both hands to denote which tool the user selected. We removed the toolbar and placed the tools conveniently inside the user’s hands. Each hand’s function is now crystal clear.

Simplifying the painting process

VR interactions

From the very beginning, we wanted users to be able to paint their own creations. Unfortunately, we made the classic design mistake of complicating the functionality with unnecessary controls and overwhelming options. More turned out to be less.

tool selection

We added a bookshelf of objects to the left side of the user and a menu of object coloring tools to the right side. The system is easy to access and even easier to understand. Testers instinctively reached to grab objects, which were then placed on the top of their hand for feedback.


Traditional design principles still apply to VR, but the tools needed to execute them are quite different. User experience (UX)/UI designers need to expand their skill sets to include 3D with physical interaction and spatial awareness savvy. Here’s what Polyworld has taught us.

  • Test immediately

    Failing fast and hard isn’t a new concept, but it’s a good principle to live by in VR and game design. Spending too much time on paper or mock-ups isn’t always as effective as quickly drawing up ideas or prototyping virtual interactions. VR is a very physical experience, so you have to dive in the deep end. Don't be afraid to show clients rough work to get their feedback early.

  • Design in 3D space

    3D skills are must-haves for the future of VR/augmented reality (AR) UX design. Whether you’re creating a mock-up or polishing an interaction in VR, make sure you’re using a 3D program such as Unity 3D. At Local Wisdom, we love Sketch, but the program isn’t too useful when it comes to VR design. Ideally, every designer should have their own VR headset to tweak their own designs in 3D spaces.

  • Embrace skeuomorphism

    In VR, a skeuomorph is an object that copies the design cues of its real-life counterpart (e.g., a virtual dial that requires you to physically turn the controller). Today, skeuomorphism has fallen by the wayside in favor of traditional flat design, but VR users find it hard to fight their natural instincts. If a light switch looks like it can be pressed, the user will likely reach out and try to press it.

  • Don’t make your users lose their lunch

    Unless you want to create a vomit comet simulator, don’t move the user around too quickly. Through significant user testing, we were able to clamp the default speed just below the threshold where users would begin to feel uneasy. Polyworld players are free to move around slowly in any direction they point toward with the agency to adjust their speed and the camera according to their comfort levels.