Having believable characters within virtual reality can be one of the most challenging but most important components to get right within a virtual reality experience. Stefano Corazza and Mixamo.com have been working on a set of tools to make it easier to create & animate 3D characters within your virtual worlds.
Stefano talks about some of Mixamo’s services like Fuse to create characters, their auto-rigging functionality, their Decimator to optimize the polygon count, and how they have over 10,000 different animations available in different packs. They also have developed Face Plus to be able to motion capture the facial expressions and emotions for a character, which is the technology that High Fidelity is using in order to translate what your doing with your face onto a VR avatar.
Finally, he talks about the principle of the uncanny valley, and how Mixamo avoids it — as well as the importance of the expression of emotions within VR.
Mixamo also recently updated their Fuse product to make it easier to create 3D characters by making it possible to upload your own content that can be customized with their tools.
They have a couple of pricing options from pay as you go to a $1499/year professional account where you get access to all of Mixamo’s services. Fuse is also available on Steam for $99.
Reddit discussion here.
- 0:00 – Intro
- 0:34 – Emotional importance of 3D characters – Face Plus facial animation, Fuze character creator
- 1:56 – Optimizing high polygon count characters with Decimator & transforming Kinect scans into rigged 3D characters
- 2:53 – 10,000 animations in pre-set packs and importing into game engine like Unity
- 3:33 – Face Plus motion capture of facial expressions, Unity plug-in & variety of applications of Face Plus
- 4:30 – Mapping emotions to a face via abstraction process
- 5:34 – How Mixamo avoids the Uncanny Valley via stylized characters
- 6:47 – Creating efficient game characters
- 7:11 – How people like Nonny de la Peña are using Mixamo
- 7:48 – High Fidelity & their use of Face Plus & using facial animation in game play
- 8:41 – Pricing structure for Mixamo
- 9:10 – Future of Virtual Reality & importance of emotions
- 10:02 – Representing yourself in virtual reality
Theme music: “Fatality” by Tigoolio
This interview with Nonny de la Peña was by far my favorite discussion from the Silicon Valley Virtual Reality Conference & Expo. It’s moving to hear about the type of emotional reactions that she’s receiving from her human rights-centered, immersive journalism pieces that are experienced within virtual reality. She has some amazing insights into VR storytelling, virtual identity, and the importance of bringing in more diversity into VR.
Nonny has been working on VR storytelling since creating a Virtual Guantanamo Bay prison cell in Second Life in 2009. She started working on with VR HMDs before the Oculus Rift existed, and in fact was a part of USC’s Institute for Creative Technologies when Palmer Luckey was there. Luckey even provided Nonny with a pre-Oculus Rift HMD for her 2012 Sundance showing of “Hunger in LA.”
She’s also worked with Mel Slater, who has explored a lot of interesting effects of Positive Illusions of Self in Immersive Virtual Reality
Nonny has a ton of insights on the components of creating a compelling VR experience from starting with great audio to creating a believable virtual humans. I also found her vision of a tiered VR future of the untethered IMAX-like experience to the Oculus Rift home experience, and then finally mobile VR to be a compelling distinction for the different levels of VR immersion and associated technologies.
For more information on the service that Nonny uses to create her virtual humans, then be sure to check out this interview with the founder of Mixamo.
Reddit discussion is here.
- 0:00 – Intro to Immersive Journalism & how it got started
- 1:29 – Recreating a scene of a Guantanamo Bay prison cell in Second Life
- 3:30 – Taking control of somebody’s Second Life avatar, and the type of reactions of going through an virtual re-enactment of being a Guantanamo Bay prisoner
- 4:29 – How people identified with their avatar being bound
- 5:14 – What were some of your first immersive journalism stories that used a fully immersive, virtual reality head mounted display? Identifying with a VR dody in stress position
- 7:12 – Institute for Creative Technologies, Mark Bolas, and her connection to Palmer Luckey
- 8:02 – Immersive VR piece on “Hunger in Los Angeles” & starting with audio
- 9:20 – Palmer Luckey & pre-Oculus Rift, VR HMD prototype for Sundance January 2012, and audience reactions
- 11:42 – Commissioned VR piece on Syrian refugees shown at the World Economic Forum
- 13:21 – Witnessing a border patrol taxing death
- 13:56 – Next projects and the potential of immersive storytelling
- 15:20 – What are some key components of storytelling within an immersive VR environment?
- 17:32 – Why is the reaction of empathy so much stronger in immersive VR?
- 18:38 – What are the risks of putting people into a traumatic VR scene and triggering PTSD?
- 19:47 – How do you direct attention within a immersive VR story?
- 20:55 – Are your immersive journalism pieces interactive at all?
- 21:30 – How else are people using this immersive VR medium to tell unique stories?
- 22:47 – What type of software and hardware are you using for your virtual humans in your immersive VR pieces?
- 21:15 – Being the only woman panelist at SVVR and importance of diversity to VR’s resurgence.
- 26:36 – Bringing into more diversity into VR storytelling
- 28:19 – The tiers of VR experiences of IMAX, home and mobile.
- 29:20 – Location-based, untethered VR experiences being equivalent to going to an IMAX movie.
Theme music: “Fatality” by Tigoolio
This is the first of 44 interviews that I conducted at the Silicon Valley Virtual Reality Conference & Expo. I was able to capture over 11.5 hours worth of material from 3/4 of the speakers, 2/3 of the exhibitors and 11% of all attendees, and I’ll be doing a daily podcast over the next month and a half (and maybe beyond). A full list of interviewees is listed down below.
Palmer Luckey is the founder of Oculus VR, and I had an opportunity to conduct a brief interview with him. I noticed that Palmer’s bio mentions that he founded the ModRetro Forums, and it turns out that there’s some interesting connections between ModRetro and the founding of Oculus VR.
He talks about some of his first and forgettable VR experiences, and the process of starting what he claims is the world’s largest private VR HMD collection. He also covers his connection to the /r/oculus reddit community, the reaction to the Facebook acquisition announcement on Reddit, as well as what he sees as what the future of VR.
Reddit Discussion here. And be sure to check out the Rev VR Ubercast featuring an in-depth discussion with Palmer here.
- 0:00 – What led to founding the ModRetro forums & how has the console modding scene evolved?
- 2:29 – How did VR evolve out of being involved with the console modding scene?
- 3:38 – What do you remember about your first VR experience?
- 4:20 – When did you start your VR HMD collection and what were some of the first VR headsets that you got?
- 4:56 – Did you end up fixing a lot of these VR HMDs?
- 5:17 – What have been some of your favorite VR experiences?
- 5:47 – How often do you read the Oculus subreddit community?
- 6:25 – How did you react to the Reddit community’s downvoting and skepticism of the Facebook acquisition?
- 7:14 – What were some of your takeaways from the panel about the next five years of VR?
- 8:11 – What has been your experience of this first consumer VR conference & what it means?
Full list of interviews from the 1st Silicon Valley Virtual Reality Conference
- Aaron Davies, Head of Developer Relations at Oculus VR
- Aaron Lemke, Unello Design founder & Eden River game
- Amir Rubin, Sixense Entertainment CEO
- Ben Lang, Road to VR founder & executive editor
- Bernhard Drax, Draxtor.com Second Life documentarian
- Blair Renaud, IRIS VR Co-Founder & Lead Designer of Technolust
- Caitlyn Meeks, Unity Asset Store Manager
- Cosmo Scharf, Founder of VRLA
- Cris Miranda, EnterVR podcast
- “Cymatic” Bruce Wooden, VR evangelist
- David Holz, Leap Motion CTO and Co-founder
- Denny Unger, Cloudhead Games President & Creative Director of “The Gallery: Six Elements” game
- Ebbe Altberg, Linden Lab CEO (Second Life)
- Edward Mason, GameFace Labs Founder and CEO
- Eric Greenbaum, Jema VR Founder
- George Burger, Infinadeck founder
- James Blaha, Diplopia game for lazy eye
- Jan Goetgeluk, Virtuix CEO & developer of the Virtuix Omni
- Jesse Joudrey, Jespionage Entertainment founder & creator of VRChat
- John Murray, Seebright founder & CEO
- Josh Farkas, Cubicle Ninjas CEO
- Matt Bell, Matterport founder & CEO
- Matt “Stompz” Carrell, Stompz founder & co-host of PodVR podcast
- Max Geiger, producer at Wemo Lab
- Mike Sutherland, PrioVR & VP of technology at YEI Technology
- Nathan Burba, Survios CEO
- Nick Lebesis, Network Flo
- Nonny de la Peña, Immersive Journalism founder & Annenberg Fellow at USC School of Cinematic Arts
- OlivierJT, Synthesis Universe creator
- Palmer Luckey, Oculus VR Founder
- Paul Mylyniec, MakeVR Head of Development at Sixense Entertainment
- Peter Sassaman, Gauntl33t Project Haptic Feedback Glove
- Philip Rosedale, Founder of High Fidelity & Second Life
- Reverend Kyle Riesenbeck, Rev VR Podcast & Road to VR contributor
- Scott Phillips, VR Walker Project
- Sean Edwards, Director of Development Lucid VR & Shovsoft & Lunar Flight & ZVR
- Simon Solotko, All Future Parties Founder
- Stefan Pernar, Virtual Reality Ventures founder & Virtual Reality Fashion project
- Stefano Corazza, Mixamo CEO & Co-founder
- Tony Davidson, Innervision VR & Ethereon game
- Tony Parisi, Vizi Founder, & co-creator of the VRML & co-chair of the San Francisco WebGL Meetup
- Vladimir Vukicevic, Gaming director at Mozilla & inventor of WebGL
- Walter Greenleaf, Stanford University, MediaX Program & medical applications of VR
- William Provancher, Tactical Haptics founder & Reactive Grip™ touch feedback
Theme music: “Fatality” by Tigoolio
Jason Jerald of NextGen Interactions has been involved with creating computer graphics and next-generation 3D computer interfaces for 20 years. His virtual reality consulting client list ranges from Oculus VR, Valve and Sixense Entertainment to NASA Ames Research Center, Battelle Pacific Northwest National Laboratories, Naval Research Laboratories, HRL Laboratories, DARPA & NIH.
We talk about some of his research and thoughts on VR latency, simulator sickness, presence and VR input devices & 3D user interface constructs. We also cover highlights from the IEEE VR, 3DUI, SIGGRAPH & Neurogaming conferences.
- 0:00 Intro
- 1:58 Consulting work with Oculus VR
- 2:46 Jason’s Ph.D work in reducing latency leading to work with Valve & Oculus VR
- 4:08 The 20ms latency threshold target
- 5:41 Research process for measuring VR latency
- 7:37 Other VR user studies comparing 3D user interface tasks with 2D equivalents
- 9:00 3D User Interface (3DUI) conference contest
- 10:46 The importance of VR hand input, point-to-fly UIs, & going beyond 2D menu constructs
- 12:43 VR input options of vision-based systems, physical based devices and data gloves
- 15:01 Comparing and contrasting the strengths and weaknesses of VR input devices
- 16:19 IEEE VR highlights including the Head-Mounted Display panel that Jason moderated
- 19:07 IEEE VR perspective on Facebook acquisition, and Henry Fuchs’ inspirational keynote.
- 20:24 The biases towards low-risk dissertations that prevented academia from making a VR breakthrough
- 22:25 IEEE VR Unity 3D workshop, MiddleVR, Virtual Human Toolkit, and AutoVerb binaural audio plug-in
- 25:27 Adoption of Unity in Academia
- 27:04 Academic VR frameworks & toolkits and UE4
- 28:04 Unity Asset Store and the Impulsonic AutoVerb Unity Plug-in for binaural audio
- 28:54 SIGGRAPH computer graphics conference and it’s connection to Virtual Reality
- 30:27 Jason’s background in real-time 3D graphics
- 31:24 Neurogaming conference impressions
- 32:34 Tradeoff of consumer EEG interfaces of ease of use vs. more powerful EEG signals with more electrodes & paste.
- 33:48 Using palm sweat and heart rate to measure VR presence
- 36:34 Quantitative and qualitative measures for researching simulator sickness
- 37:39 Sixense’s serious game grant for “Motion-Controlled Gaming for Neuroscience Education”
- 39:39 Potential of getting a VR dream job in academia
- 42:28 Keenly interested in the open problems of 3D user interfaces, researching simulator sickness best practices & moving towards higher-level VR problems rather than implementation
- 44:50 Wrap up and conclusion
Music: “Fatality” by Tigoolio
Oliver “Doc_Ok” Kreylos is a research scientist / computer scientist who develops virtual reality applications for scientific research, specifically immersive 3D visualizations for the department of geology at the W.M. Keck Center for Active Visualization in the Earth Sciences (KeckCAVES).
He is an active participant in the Oculus subreddit posting as “Doc_Ok” where he has been gathering a lot of attention for his innovative data visualizations as well as an array of Kinects to create a “pseudo-holographic” avatar within virtual reality.
Oliver has been making virtual apps since 1998 including the Virtual Reality User Interface (aka Vrui) for navigating and interacting with immersive 3D visualizations of scientific data. He has a wealth of knowledge about VR, and has been providing a lot of insightful commentary on his blog at Doc-Ok.org.
- 0:00 Intro & enabling scientific VR visualization
- 1:57 Remote collaboration with scientific collaboration
- 4:49 Developing the Virtual Reality User Interface (Vrui) toolkit
- 8:01 3D visualizations that are impossible in 2D
- 10:37 Converting 2D CAT scan slices into full 3D medical visualizations
- 14:12 Future of Kinect-enabled telepresence collaboration
- 16:32 Hardware & software stack for setting up a calibrated Kinect-array for telepresence
- 19:07 Speculation on hacking Kinect V2
- 21:37 How Kinect VR avatars can provide a sense of presence
- 24:12 The importance of implementing positional head tracking for presence
- 25:32 Importance of using 6DOF Hydra & STEM controllers with Vrui & data visualization
- 27:29 Importance of supporting VR input devices in a unified manner to avoid previous VR mistakes
- 28:32 Prophetic feedback on the Oculus DK1 that has been integrated into DK2
- 31:33 Find Oliver online at Doc-Ok.org, @okreylos & Doc_Ok on Reddit. Oliver’s future projects
Music: “Fatality” by Tigoolio
Professor Eric Hodgson explores some of the psychological impacts of virtual reality when it comes to spatial perception & memory, and effects like change blindness. He also talks about how to walk in infinite VR spaces within limited physical spaces with a technique called redirected walking. Finally, he talks about insights from IEEE VR community on this consumer VR revolution, compares the split between the old VR community to the new VR community, 3D user interfaces, data visualizations, and some examples for how corporations and the military are using VR.
Eric Hodgson is the director of the Smale Visualization Center & also works with the Armstrong Institute for Interactive Media Studies at the Miami University of Ohio. He runs the HIVE (huge immersive virtual environment), which is a gym-sized VR Lab. For more information on Eric, here’s some of his VR research publications.
- 0:00 Intro
- 1:04 Redirected Walking
- 3:50 Factors that go into spatial perception & visual dominance
- 5:26 Why was the HIVE lab built?
- 8:10 How our spatial memory works
- 10:49 Change blindness for changes that are out of the field of view
- 12:12 Navigating geometrically impossible spaces & how we mentally fill in gaps
- 13:29 Movement speeds in virtual environments – natural vs. VR movement speeds
- 14:38 Why VR movement speeds are 2x normal walking speeds. Field-of-view?
- 15:42 IEEEVR community perspective on the rise of consumer VR
- 17:20 The divide between Old VR vs New VR communities
- 19:03 Future of Professional VR companies & high-end VR markets
- 20:41 How the military is using VR
- 22:42 How corporations are using VR
- 25:04 Why now? The timing of this VR revolution. Previous low-end VR headsets
- 27:01 Cutting-edge demos and prototypes at IEEE VR conference
- 30:03 3DUI conference, and user interface research in VR environments
- 30:58 Highlights of the future of telepresence keynote from the IEEE VR conference
- 34:03 Where VR is headed within five years & what gets you excited about what’s to come
- 35:41 Data visualization within 3D VR environments
Music: “Fatality” by Tigoolio
I’m joined by D of the eVRydayVR YouTube Channel to discuss:
- Virtual Reality evangelism
- VR design tips
- VR communities
- The future of VR and Education
Here’s a link round-up and overview of topics discussed:
Get in touch with eVRydayVR on YouTube and Twitter and Facebook and at gmail.com.
Music: “Fatality” by Tigoolio
I’m in the process of learning everything I can about the principles of what makes an incredible virtual reality experience. It’s the wild west of