Dora Cheng is one of the co-founders of uForis VR, which is a VR company that is focused on creating pragmatic applications of virtual reality. One of their first big VR projects was to create virtual tours for the Domus Student Housing. They had to figure out a production pipeline that could operate at scale in order to capture thousands of photo spheres of apartment units within a 3-month period. Dora talks about all the production and post-production challenges that they had to solve that included processing, stitching, color correcting, and creating a VR application wrapper that’s viewable within the Gear VR.
LISTEN TO THE VOICES OF VR PODCAST
Domus Student had the problem of high-turnover rate with a lot of students changing their apartments at the end of year, and they saw that virtual reality could help the students be more efficient in their apartment-hunting journey. Instead of seeing dozens of units in real life, prospective tenants can go into an office and very quickly see a dozen units to get a sense of the look and feel of the apartments at scale. From there, then they can narrow down the options, and then only go to see a handful of their favorite ones.
Dora breaks down some of the challenges that uForis VR had to overcome in order to both capture and process thousands of photo spheres over many months. They’re looking to expand to other real estate applications, but also expand into other pragmatic applications of VR.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:12.095] Dora Chen: Hello, my name is Dora Cheng, and I, along with two other partners, are actually in a company called U-Force VR. And what we have done is we have just finished a real estate app for our clients in student housing in Waterloo, Ontario, which is more of a college town. And we're also packaging up the tools that we've used for it. And we sent out an alpha as well for people to try. So yeah, if anybody wants to download and give us a lot of feedback, tear it apart, please do. It's alpha, so it'll be buggy. It's not going to be perfect. But what we need is feedback.
[00:00:46.789] Kent Bye: Great. So yeah, maybe talk about the process of what you had to do to actually get the content for this real estate app.
[00:00:52.795] Dora Chen: Sure. So our client has a lot of properties in their portfolio for rental. So what we've done is the three of us, we went out and got a 360 camera rig and went to all their units and took spherical panoramas, 360 degree, and put them into a tour. a walkthrough of each and every one of the units. So because there was about hundreds of them and every unit is composed of about 10 to 15 panoramas, we took about a thousand of these panoramas in the span of about a month just between the three of us. So we've learned a lot about the process on how to do it. Now, of course, the pictures, you know, they are great quality. So we're very proud to say that. But it's not going to be a National Geographic kind of quality, of course. But it's great quality for our clients, clients who are students, to have a very good idea on what the properties look like. And a lot of our shooting process, our capturing process, could be sometimes very challenging because a lot of these units, for example, they would have a bathroom that's very small. For example, an en-suite bathroom, which may not always be as big as a bedroom, right? So because of the tight spaces, and sometimes you're unevenly lit as well, because if you're in an en-suite bathroom and you're trying to take a picture of the bathroom, Usually the bathroom is darker than the rest of the room outside. So because it's 360 degrees, you capture the whole thing. So it's the tight spaces and these unevenly lit places that we have to figure out a process to compensate for, I guess. And on top of it all, you have to finish this volume of panoramas in a month. So at the end of it we were able to get the process down so that each panorama takes about three minutes to shoot and the resulting pictures are fairly evenly lit. The dark areas are not too dark and the bright areas are not too bright. It's fairly even. We need to do a bit of post-processing and all that and it seems to work out fairly well and we're not quite done yet actually because My client wants us to actually add more properties to the app. So we're pretty excited about that and we're always, always trying to improve the process as well. For example, we're going to shoot some of the amenities and the surrounding areas of the buildings. So that's outdoors and that's a different process entirely. There's cars moving by and all that stuff. So it's a continuous process right now. And doing panoramas on an industrial scale is something that we ourselves have never attempted before. And when I went out and did some research, I haven't seen anybody looking at that either. So it's very challenging, but very rewarding. It really gives me a very good idea of what it takes to do VR content. You know, I used to do gaming, so I kind of have an idea of what it looks like for gaming, but doing it on a utility standpoint is very different.
[00:03:53.511] Kent Bye: Yeah, so maybe tell me a bit more about the camera and if it's optimized for taking still photos or if you can also do like live capture and then, you know, what kind of tools you had to use or put together in order to actually create these spherical photo spheres.
[00:04:08.415] Dora Chen: For sure. So what we did was we looked at a bunch of different 360 cameras and we figured that well for what we need we need to go out and get a good old DSLR camera. So we went out and got one. But we're not going to shoot a 360 degree panorama and thousands of them by hand because that would be way too much time and frankly it's probably going to be impossible. And it's just the three of us. So we went and we actually got a rate called the GigaPen. and programmed it, optimized it so that it just turns by itself and we just have to run and duck all the way. But yeah, it's just right now it takes about one panorama in like three minutes as I said. And of course we have to get all the corresponding accessories for it. the tripods and everything. And we actually also use the Matterport. And for that part of it, we also take the Matterport rig around the layout at the same time that we're going around taking pictures. And the Matterport really gives us the lay of the land. So what the layout actually looks like because of their services. And that also, we've asked the students who've tried it, and the students said combining the layout and the feeling of being in the unit itself through VR really gives them the idea of what the units they're looking at, what the differences between the layouts are, which is the most important part. So they can choose the one that fits them the best. I should mention, actually, that the app is on the Gear VR. And that is specifically what our client requested because it's untethered. And it's a lot more convenient for everybody. And the customers who walk into their office to try it will have an easier time dealing with it than something that is tethered.
[00:05:49.758] Kent Bye: I see, so it sounds like it's something that if they don't have their own Gear VR headset, they would go into the offices and then go into a room or whatever and start to go into VR to be able to walk through these different photo spheres.
[00:06:00.587] Dora Chen: Yeah, that's actually correct. So our client, Domus Student Housing is our client, and they are actually very much into VR and the possibilities of VR. So they've actually invested in a few of the new Note 4s, and they will have a few Gear VRs in the office, so that when the students come in, they will have a chance to try these on.
[00:06:23.526] Kent Bye: Yeah, and what did you have to do in order to turn all these photospheres into an app? You put them into Unity, or what's the process of actually allowing people to navigate these spaces?
[00:06:32.792] Dora Chen: Oh, that's a very good question. So we use actually our own proprietary engine, so we actually wrote our own stack for it. After we took the photos, we processed them actually by hand for the highest quality. And we put them through a program called PT GUI, which is one of the most popular 360 degree panorama programs out there. Stitch them all, post-process them, make sure they all look great. And then we actually also then put it together into a tour. and we run it through a pipeline, which is something that our chief engineer created himself, and then it spits out the tour, and then we put all these tours into the building, and so on and so forth. So it's all automated. Every time there's a new building, we don't have to send out a new build. It just shows up automatically. All of this is streamed as well from our backend. So whenever there's a new building, we post-process it, we run it through the pipeline, and it just pops up into the new building. It's a lot more convenient for our client too because Right now, as the developers know, you need signature files and then you need to push the bill physically and so on and so forth, right? But we don't have to do that. We can just install the app once onto our clients' phones and then whenever there's something new in the layout or we fix a bug or whatever it is, we just send it out and it just pops up in their apps.
[00:07:53.823] Kent Bye: Yeah, and that's one thing that I guess I never really connected the dots, but it's very true that as a photographer, if you're taking a photo and you sort of have the window in the background, it's going to blow out the shot and kind of make everything in the foreground really poorly lit. But in the 360 spherical videos, you kind of inevitably are always like having a window or having some part blown out. So are there things you have to do to like cover all the windows first of all, and then maybe talk a bit more about what you're actually doing to color correct to these?
[00:08:22.523] Dora Chen: Oh, so that's a great question. The funny thing is sometimes we'll go into a room and say the light bulb went out, you know, so we will go around and steal a light bulb from another room and put it in there for the photo itself. So luckily we were able to figure out a process so that we don't have to, unless it's extreme cases like that where the light bulb went out, the camera and the post-processing compensates for those lighting conditions. So DSLR cameras are great at this, right? So we just made sure that the ISO, the shutter speed, and the aperture, the three things of exposure, is automated enough so that it compensates for the lighting of the room. And then we make sure that the parallax effects and all that stuff is minimized by aligning all the nodal points and so on. So that in an unevenly lit, very tight bathroom, the lighting will still come out quite even and you will not notice any kind of seams. It's not perfect, you know, because it's really hard when the door frame or the door, especially if it's reflective somehow, is right in the camera's face basically. So there's only so much that we can do to compensate for that, but we are able to make it so that as much as possible you do not notice little little nicks here and there.
[00:09:45.620] Kent Bye: And I imagine that doing this type of work for real estate, that's a huge market. I'm sure you guys will be keeping plenty busy with this. And as a niche and a specialty, I'm curious from your own perspective of going out and actually doing these shoots, if you foresee yourself actually being on the front lines and enjoying being able to see new places each and every day, or if you foresee branching out into other domains beyond just like the real estate.
[00:10:11.169] Dora Chen: Right, so the funny thing is that real estate is our first foray into practical, pragmatic applications of VR. We are very much, the three of us in our little team, very much interested What are some of the problems, like practical everyday problems that VR can solve? And in our client's case, in Domas' case, there happens to be a need. Students, their turnover is yearly because they graduate or they move to another school or they want to move and live with a different set of friends, whatever it is. So they have a high turnover rate and they want to make it so that the students don't have to look at 10 different places physically. Maybe look at one or two physically before they make a decision. So it saves everybody's time. So there's an actual need here that we can actually fulfill. But we also have other projects in the works actually. We're looking at potentially some kind of travel slash museum apps for kids. We're looking at social app for meetups, for friends of ours, and a few other things, you know, and as I mentioned before, we have put out the tools to see if those tools may be of use to somebody else who is looking at something with panoramas or even video panoramas, because our tools also play video panoramas. So we have many irons to fire, actually, because we want to look at as many things that we can use VR for as possible.
[00:11:41.215] Kent Bye: What has been the reaction of the client so far? I mean, how are they measuring that this is a successful endeavor for them?
[00:11:47.401] Dora Chen: Great question. So our client has been very happy with the progress so far. And one of our goals was to have a public beta in the month of September in the city where they're based, which is Waterloo, Ontario. And actually just last week and yesterday, We brought the app to River Laurier University and University of Waterloo, which is where I graduated from, actually. And we showed it to students. So it's a public beta, basically. And the students were blown away by it. They were very curious about VR, because a lot of them haven't tried it. And they saw the headset, and they're like, oh, I've got to try it. So there's a lot of curiosity there. And we made sure that we get their feedback as well on the app. And they give some really good stuff, like you can do these things better, which we're going to do. I mean the clients also have given some really good feedback talking to their students. But overall they're very very happy and they're very surprised that VR could be used in this way and that it's useful for them. So of course you know as we all know in the VR community there are many things that VR can improve upon. So for example in a static panorama, for example, how do you give the sensation of moving from say the living room to the bedroom and so on? Now, do you even want to do that? Because that involves motion. And for somebody who has never tried VR before, is that a good idea to have motion? All of these different questions that we are exploring and finding answers to. So we're very happy that initial public beta reactions from both our client and our students are very, very positive. And I think it can only get better because we're learning so much from it. We're learning how to navigate, we're learning how to start people off. explain to them that the, for example, explain to them that the touch pad is on your right side, you can touch here, easiest way to put your headset on your head is over the eyes, and then the strap, and so on and so forth, just even little things like that, and could really make a difference in easing somebody into the experience of VR.
[00:13:52.969] Kent Bye: And finally, what do you see as the ultimate potential of virtual reality, and what that might be able to enable?
[00:13:59.791] Dora Chen: Wow, I'm a believer. I'm a fanatic in terms of VR, and I think that VR could be used everywhere. You know, I mean, real estate, travel, architecture, even medicine. You know, imagine being able to look at 3D renderings of a patient's body or even disease tumors and so on and so forth. That would be really useful. But I think that there are many things that could still be discovered that we just don't know about. We never knew that it would be useful for student housing. We never thought, but our clients saw the possibility and told us, yes, we can use it. And I think that what's necessary is for people who are interested in VR and people who are obsessed about VR, to try things, to create things, and to bring it out there and see what's possible. And I think that ultimately VR is a new medium and it will change the way that we consume content, it will change the way that we interact. Because I can imagine, for example, Kent, that we could have an interview in VR, you know, and probably sooner than we think. And I think that would be great. I think we would have arrived if, say, five, ten years down the road, people write think pieces in The Economist, Time Magazine, and The Atlantic, and New York Times about the impact of VR and what it means, kind of like how people write about Twitter, or people write about Reddit, and people write about Facebook, and how bullying was amplified or mitigated through social media or through smartphones or, you know, with the current crises in Europe with the migrant refugee crisis and so on. And people write about the role that smartphones play, you know, and I can see it happening, you know, the role that VR plays in our society. Hopefully sooner than we think, but even if it's later, I'm very excited about that.
[00:15:57.194] Kent Bye: Awesome. Well, thank you so much. Thank you. And thank you for listening. If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash Voices of VR.
