Building a world with your own hands in VR

Our design teams shifted our app and web design thought process to adapt to the evolving world of VR.

The whole world created in your virtual hands

Polyworld is our whimsical approach to a mini world building experience that provides god like powers to it players as well as a way to present how VR can tilt your sense of scale. Although, interacting with your environment in VR can feeling awkward unless you use the right input methods. The Vive and Oculus Touch controllers are a great first step, but we wanted our real hands in VR. We opted to use a Leap Motion to track your hands and aimed to make it as natural as possible.

The first step was allowing the user to create mountains and valleys with their real hands by intergrating leap motion with oculus. By grabing and pulling it feels like you are conducting musical terraforming. Each action follows the consistent push and pull motion allowing the user to draw objects, and add colors by painting. Players can touch the planet to be transported onto the surface and experience a mind bending change in scale and perspective. Of course, no experience is complete without some destruction, so players are free to walk around and blow up their creation with some simple hand motions. Getting to the final product took learning from our failures and turning them into successes.

A look into the design process

Early ideas of pushing, pulling and carving of the landscape had nuanced controls that required too much fidelity. We quickly found UX ideas that worked “on screen” or “in theory” often lead to a poor experinece. Often this was due to current limitations in VR and the Leap Motion.

Changing to a model that lets the user control pushing, pulling and carving by the location of their hand in space lead to success. Pulling your hand away creates mountains while pushing into planet creates craters. Finding the right balance of control took extensive user testing.

Here an early concept tied the user controls to the bottom of the “screen.” This failed in multiple ways. We allowed users to touch tools with either hand allowing left or right handed use but it lead to confusion. Additionally the tools were too awkwardly placed in relation to the user.

We simplified the interaction and found success. Making your left hand only rotate the planet removed confusion. Adding icons on the user’s hands to denote which tool they had selected made each hand’s purpose clear. Finally, we removed the concept of “the toolbar” and tethered the tools to the inside of the user’s hand to elevate any awkwardness.

From the beginning we wanted users to paint with objects. The main miscalculation was allowing too many options at one time. We made the classic mistake of making it too complex, and offering controls that were unnecessary. Additionally, our ideas required too much fidility and could not be achieved with current tech.

Simplifying worked wonders in testing. Creating a bookshelf of objects on the left side of the user led to easy access and understanding. Users instinctively reached and grabbed objects. The objects were then placed on the top of their hand for feedback. To the right side, an object’s color could be easily accessed.

Refining our process for UX and VR

Traditional design thinking principles still apply to VR, but the execution and tools necessary to achieve success are different. The skill set of a UX/UI designer needs to be expanded into 3D with a understanding of physical interaction and spacial awareness. Here are some of our key learnings.

  • Test immediately

    Failing fast and hard isn’t a new concept, but it’s a good principle to live by in VR and game design in general. Spending too much time on paper or “mock-ups” weren't as effective as drawing up quick ideas and developing quick prototype interactions. VR is a very physical experience, and traditional UX approaches will not work. When working with clients don't be afraid to show rough work to get feedback early.

  • Design in 3D space

    If you are doing a mock-up or polishing an interaction, use a 3D program. Ideally, designers should design in Unity 3D or a 3D application. We love Sketch for UX at Local Wisdom, but Sketch is not going to be too useful for VR design. Ideally, you want your design team to each have a VR headset, and have them check and tweak their designs in 3D space. Having 3D skills will be a must in the future of VR/AR UX design.

  • Skeumorphic UX interactions

    Designing for how we interact with real world objects works really well in VR. Skeumorphism has become a dirty word and has been dropped for flat design in traditional apps and web design, but in VR its a key way to help a user understand their new enivronment. Think in the physical, such as light switches, dials, and use natural human motion. If something can work like real life, most likely a user will try that interaction in VR without being told how.

  • Treat physical locomotion with great care

    Unless you want to create a vomit comet simulator, don’t move the user too quickly. The user must be able to control the direction of the camera at all times. We found success in moving the user slowly in the direction in which they pointed. Using a consistent UX on the inside of the palm the user could adjust between three movement speeds. The default speed was found through user testing. We clamped the speed just below where people began to feel uneasy. Note, dont just test a few people, you need a large sample of people to find a good balance.