Here’s my interview with Tony Bevilacqua, Founder and CEO of Cognitive3D, that was conducted on Friday, June 2, 2023 at Augmented World Expo in Santa Clara, CA. See more context in the rough transcript below.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.458] Kent Bye: The Voices of VR podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So continuing my series of looking at AWE past and present, today's episode is with Tony Bevilacqua. He's the founder and CEO of Cognitive3D. So I've run into Tony a number of times over the years where he has got this data analytics platform where he's able to create this plugin for Unity to gather all this additional information. And there's a number of different ways that customers can start to use this data from training and simulation, consumer research, games and entertainment, especially around trying to get a sense of the performance of the application, but also just kind of what the users are doing and also like academic research. And so there's a lot of ways that he's able to capture this data, but also have like a broader privacy framework of like do not track. So giving the power to users. the users to say, hey, I actually don't want you to track this very specific data that your company is also tracking these other use cases that are not in this kind of more surveillance capitalism context. And so the thing that XR affords is all this new insight into people as they're in these different situations. And so often these companies would put people in VR to get all this extra data to get more insight into how to design grocery stores or any number of different applications up to and including different entertainment, gaming and stories and whatnot. So always great to catch up with Tony just to see like what's the bleeding edge of the different things that they're able to extrapolate from taking all this data from XR and to apply it within the context of these enterprise applications. Also worth noting that there's kind of a freemium version of this plugin, so you can go check it out. And if you have not a lot of users, then you can start to use some of their different tools to get a sense of what's even possible with their platform. So we're covering all that and more on today's episode of the Voices of VR podcast. So this interview with Tony happened on Friday, June 2nd, 2023 at Augmented World Expo in Santa Clara, California. So with that, let's go ahead and dive right in.
[00:02:15.376] Tony Bevilacqua: Again, my name is Tony Bevilacqua. I'm the founder and CEO of Cognitive 3D. We do analytics for virtual, augmented, and mixed reality. Ultimately, what that means is we collect behavioral data on how people are interacting inside 3D environments. We also help developers understand and enhance their experiences, make them better, make better content, and also measure performance in these applications.
[00:02:36.429] Kent Bye: Great. So maybe you could give a bit more context as to your background and your journey into VR.
[00:02:41.294] Tony Bevilacqua: Yeah, for sure. So I'm a second time founder. Prior to Cognitive 3D, I also worked in the mobile analytics space and we were building kind of a product that was very similar to a lot of the existing mobile analytics platforms on the market. You know, I got early exposure to the DK2 GDC way back in the day and kind of realized there was an opportunity to do like actual innovation and build something different, something unique. And so that's really where I kind of formed this idea of measuring how humans interact in 3D-based environments and using the headset as a vehicle for data collection. And so that really kind of culminated in 2015. I formed what was called Cognitive VR at the time. And yeah, we had our first employees 2016, first product in market in 2016.
[00:03:24.463] Kent Bye: I think we met maybe at Silicon Valley Virtual Reality Conference back in the day, or one of those conferences.
[00:03:29.505] Tony Bevilacqua: Exactly, yeah. It's kind of funny. I remember all the events back in the day, like VRLA, SVVR, all those types of events as well, where we were trying to promote it to the public that this is an interesting medium and you should be interested in it as well.
[00:03:42.631] Kent Bye: So walk me through a little bit of the different use cases and industry verticals where someone might use something like Cognitive 3D.
[00:03:49.724] Tony Bevilacqua: Yeah, for sure. So Cognitive 3D, we've been focused on the enterprise category for about the last five, six years now. And so training simulation, consumer research, and so consumer research for us encompasses AAC, so architecture, engineering, construction, also large product design. So that could also be like aircraft, vehicle design, those types of things. CPG and retail, and finally academic research. And we've got about 35 academic institutions that use our platform for varying levels of research from eye tracking to biometrics to just kind of like raw behavioral data.
[00:04:24.272] Kent Bye: So it's an idea that both for if someone's building an airplane and they want to do like ergonomic testing or to see how humans are going to be interacting and say maybe the cockpit or something like that. So maybe flesh out a little bit of some of those different use cases.
[00:04:37.024] Tony Bevilacqua: Yeah, for sure. Across the board, right? So if we're thinking about an aircraft, we might be interested about ergonomics, how the aircraft is going to be serviced, ultimately how people might train on that digital twin as well. You know, you might also be interested in private planes. and being able to understand what buyers are interested in, what they pay attention to, what actually matters. Same kind of concept applies to vehicle design. If we think about vehicle design, if I'm going to test a new design, being able to understand how buyers interact with that, but also ergonomically interact with the vehicle as well, and find the things that they're looking for, such as the volume controls and other types of elements as well. So you can use these digital twins to draw a correlation with the real world, and that can apply kind of from buildings and real things all the way through to small products.
[00:05:24.807] Kent Bye: And so there's both the human aspect of the behavioral and how they're interacting with the piece, but there's also the hardware side with how the actual application is performing. And so I imagine that because you have the head tracking data, you can say they were looking at this object and then this is the performance. You're able to maybe draw some correlations between what was happening in this virtual world versus what's happening on the backend. So maybe talk about some of the performance aspects of how that evolved out of what maybe it started with a little bit more of the behavioral aspects.
[00:05:55.666] Tony Bevilacqua: Yeah, for sure. So our tools are entirely focused on being able to aggregate spatial data from the 3D engine. So our plugin actually goes into Unity and Unreal and collects that data. So when we talk about gaze data or head orientation or eye tracking, we're subscribing that data to the different surfaces and items and dynamic objects that might exist within that space. So when you're looking at that from a practical perspective, we can break the scene into its core components and say, you know, this particular packaging design drew this user's attention, but it also certain elements of that packaging design were the key elements that drew their attention. And then not only by not doing this in video and looking at more of a spatial data, you can then aggregate it. And that's where things start to get really powerful, where we can start talking about an anonymous segment of users that you can kind of like take all that raw behavior and turn that into a single type of insight and be able to look at all your products together and how they're interacting with them.
[00:06:54.730] Kent Bye: So I imagine that there is a lot of the same type of functionality that you're building in more of this enterprise context for enterprises and other companies that some of the consumer producers of XR, like maybe Meta as an example, were very interested in what people might be looking at and extrapolating that data to be used in more of a surveillance capitalism context. So I feel like there's this overlap between some of the core technologies you're developing in the analytics sense versus what some of these same technologies used in a different context may be something that may be transgressing different ethical boundaries. So maybe you could talk about the context in which you're building this, but also the broader ethical implications of some of this.
[00:07:31.319] Tony Bevilacqua: Yeah, so there's a really fine line in terms of how this data can be used and in terms of how we allow our customers to use the platform as well. We are starting to get into consumer-based applications, and when we think about consumer-based apps, the types of things that we're trying to help our developers solve is really around understanding comfort, the level of presence that a user might have within an experience, as well as the application's performance. And so we've spent some time building effectively a barometer that takes these raw inputs and boils these things down So developers can understand like where they're going to spend time and how they're going to engage. One of the things that Cognitive has been spending a lot of time on is we built a new thing called the XRPF. It's called the XR Privacy Framework. And effectively, it's a do not track consent framework. And so if you think about the web and mobile space, do not track really got nowhere. You know, like everybody just kind of said, hey, there's no standard, so we're not going to respect it. And so what we ended up doing was building the standard and then putting it into our SDK. And then we built a Unity and an Unreal prefab that you can then put into your scene. And it has five different types of data inputs. So behavioral data, social data, all these other types of data. a disclosure around how that data is ultimately going to be used, here's our privacy policy, consent or not consent, and then we respect the do not track signal for each of the different inputs. So I think that that's important. We have not seen a lot of traction on interest in filtering out data, but we wanted to release something so at least our customers that are spending time on this have a tool they can use.
[00:09:02.228] Kent Bye: That's really great to hear because, you know, it's obviously as we move forward, we have implications of all this biometric and physiological data. And what Nita Farahani in her book, Battle for the Brain, is advocating for these new human rights of cognitive liberty that preserves our freedom of thought, our mental privacy and our self-determination. So all these aspects of as we move forward, things like this XR privacy framework is going to be really important to the larger ecosystem. I think the caveat is that it often takes these big, major corporations like Meta and Google and other companies that are interested in surveillance capitalism to sort of also adopt them and respect those. Because if that's not at that level, then there's issues. But let's go back to the behavioral aspect, because it's very fascinating that in a market research context, I imagine that you may have people go into these virtual environments and maybe they're walking through a store and they're trying to get a sense of what layout is going to be best for the future of how they're going to build certain retail spaces let's say as an example where there's like market research that's going on there so maybe you could elaborate on how you're able to digest all this physiological and biometric data and turn it into real insights and feedback for companies that are trying to get a sense of what's happening that might not be possible if they're just doing it in the physical world without the xr that has all this additional tracking
[00:10:15.373] Tony Bevilacqua: Yeah, for sure. So, you know, it's very dependent on the use cases in terms of like what the customer is ultimately trying to achieve. But to go back to your example, you know, in the CPG retail environment, we might be interested in wayfinding a signage. So a good example would be like, you know, if you're going to put signage within a store, which signage is most effective and what type of placement or what type of content draws the user in to recognize that stimuli and then find the path to the location that we're ultimately after. So you can test a lot of different variations in how you're going to lay out the store or how you're going to put those types of signage and then test the wayfinding to be able to figure out what path did you ultimately take, but also leverage the eye-tracking data to be able to say, okay, yes, they did fixate, they did comprehend this particular stimuli, and then they took action.
[00:11:00.049] Kent Bye: Right. And so what's been the response here at AWE as you're here on the expo floor and as we have over 300 exhibitors here on the floor and the 5,000 people at this conference? And so what's it been like here at AWE for you?
[00:11:12.489] Tony Bevilacqua: We have the largest team that we've ever had here. So we've been super excited spending time going to sessions. We've had tons of people come by the booth. Lots of excitement. I feel like it's a really good year. I think that there's a lot of people that are interested in what happens with Apple. And I think there's some new wind in the sails, per se, in terms of how things are going to move forward. This is probably the busiest AWE that I've seen, and I've been coming since 2016. MARK MANDELMANN- Great.
[00:11:37.667] Kent Bye: So what's next for some of the features that are coming for Cognitive 3D?
[00:11:41.536] Tony Bevilacqua: Yeah, so the big thing for us right now is that we're rolling out our new SDK as well as our new dashboard. And so the new dashboard is focused on consumer-based applications. As I mentioned before, we're really focused on helping developers understand how to build higher quality experiences. We recognize as a business that developers are obviously worried about retention in their applications and making sure that users have a comfortable experience and then they ultimately come back, right? But what we've also recognized is that poor quality experiences that are not comfortable, that offer low levels of performance and low levels of presence also create potential risk for the platform itself and attrition within the platform. And when I say that, I mean people putting the headset back in the box and taking it back to the store. So I feel like this particular medium, the stakes are the highest of any other medium prior to it in delivering on quality. And if you don't meet that quality, it's likely you're going to fail to engage that user.
[00:12:37.404] Kent Bye: And so what are the best ways for people to get involved with if they want to get the SDKs or like what's the pricing plan or how does that work for actually becoming a customer of Cognitive 3D?
[00:12:46.589] Tony Bevilacqua: Yeah, for sure. So the freemium plan that we launched is pretty generous. You get 100,000 sessions per month. So that covers the majority of developers for free. So we're just trying to build better quality content. And by the time you hit that scale, you're probably already winning in your category anyway. So then we can talk about pricing.
[00:13:05.419] Kent Bye: Great. Great. And so finally, what do you think the ultimate potential of XR might be and what it might be able to enable?
[00:13:13.146] Tony Bevilacqua: For me, it's about ubiquity. My first investors were super ventures and they always had this mantra of giving the people superpowers. And so for me, it's about ubiquity in life and giving human superpowers.
[00:13:26.397] Kent Bye: Awesome. Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:13:30.703] Tony Bevilacqua: No, I mean, if you're interested in cognitive, definitely check it out. We're trying to help developers build better content, better experiences to help this market be successful. But I'd also say to folks that are either competing with us in this space, or they're thinking about privacy and data collection, think about Do Not Track, think about XRPF. Maybe it's something that you can implement, respect, and that we could make a safer data collection medium in this platform.
[00:13:56.195] Kent Bye: Just a quick follow up with that XRPF. Does that mean that if you have a consumer device someone puts on the headset, would that mean that the user has the option to say don't track these certain things as they come up? And that it'd be up to the developer to reveal that to the user so that they can opt out of things?
[00:14:11.311] Tony Bevilacqua: I mean, ideally, the platforms would implement it. That would be the best place. So you can kind of make those decisions upfront prior to the app launching. Kind of like when you use a Google Android application, it asks you for what permissions you want to grant. Ideally, it would happen there. But the way we built it is you would implement it inside your application. And then the user would see it. They would make their decision. And then the decision would be respected by whatever data collection methodology you're using.
[00:14:35.239] Kent Bye: Awesome. Well, it's really exciting to see you again on this circuit. I know we've crossed paths a number of times, and it's great to get a little bit of an update on where you're at and where you're going, because I feel like analytics and this data visualization was a part of just kind of having a better sense of what's actually happening in these scenes. It's like a black box a lot of times, so you're helping to kind of demystify that black box. So thanks again for joining me today to help break it all down. Thanks a lot. Thanks again. thanks again for listening to this episode of the voices of your podcast. And if you enjoy the podcast and please do spread the word, tell your friends and consider becoming a member of the Patreon. This is a, this is part of podcast. And so I do rely upon donations from people like yourself in order to continue to bring this coverage. So you can become a member and donate today at patreon.com slash voices of VR. Thanks for listening.
