When Derek Belch was a kicker on Stanford’s football team in 2004, he took a class with the Virtual Human Interaction Lab’s Jeremy Bailenson where he was exposed to virtual reality technologies for the first time. Belch asked Bailenson if it was possible to use VR to train football players, but the technology wasn’t ready yet back in 2004. Fast-forward 10 years with Oculus Rift VR development kits readily available, Belch started a master’s thesis project with Bailenson to study how to use VR to train quarterbacks.
Their pilot program had promising results, but not enough conclusive evidence to be able to say for sure. But the response from football players and coaches was so overwhelmingly positive that they decided to start a company called STRIVR Labs to put their research into practice. They quickly signed up Stanford, Vanderbilt, Clemson, Auburn, Arkansas and Dartmouth as their first official partners to continue their research, and they also started working with NFL teams including the Cowboys, Cardinals, Giants, Vikings, and Jets.
I had a chance to catch up with STRIVR Lab’s Chief Science Officer, Michael Casale, at the Experiential Technology and Neurogaming Conference in May. Casale was brought on by Bailenson to help advise Belch’s master’s thesis on learning transfer and category learning techniques that would optimize the learning process. He’s continued this transfer learning research by working with elite athletes at both the collegiate and professional level at STRIVR Labs since it moved from an academic research project into the real world.
LISTEN TO THE VOICES OF VR PODCAST
Casale was hesitant to report on specific quantitative evidence since there are a lot of proprietary metrics that they’re using internally, but he said that there’s been a lot that’s been reported within the mainstream press. The San Diegeo Tribune reported that Stanford quarterback “Hogan’s pass completion rate jumped from a 65 percent average over the first 10 games of the season, to a 76.3 percent average over the last three games – right around the time he started using the virtual reality trainer to study defenses and make decisions.”
The article emphasizes that correlation is not causation, and establishing learning transfer from VR technologies to the real world is still an open problem. But there’s a strong indicator that VR is having a huge impact when looking at Arizona Cardinal’s record 13-3 season with VR early adopter Carson Palmer telling ESPN that “I think it’s improved my stats. It’s improved my knowledge of our offense.” ESPN speculated that “It might not be a coincidence that Palmer had the best season of his career, throwing 4,671 yards and 35 touchdowns, and finished the season with a career high in quarterback rating (104.1) and QBR (82.1).”
Casale hinted that there’s a lot of value that’s being gained from VR training that might not explicitly show up within the existing statistics that drive fantasy football leagues. Being able to detect an oncoming blitz and dynamically changing the play before the snap is one example of a skill that can honed within VR, but not directly measured on the field. Quarterbacks can also watch themselves from the 360 footage and they can then work on correcting their throwing motion and footwork in the offseason.
A vital part of the training is being able to have more interactive coaching sessions where the quarterback can re-watch different defensive positions and talk about how they would change or adapt their play. Here’s some footage of a Stanford quarterback reading the defense and telling his coach what he sees.
https://www.youtube.com/watch?v=QSJVU7PK660
Rather than translating X’s and O’s of a play from a 2D whiteboard in their mind, quarterbacks can prepare and watch what the field actually looks like from VR reps. Carson Palmer was learning 171 plays in 5 days using STRIVR Labs VR system installed in the comfort of his own home.
VR locomotion is still an unsolved open problem, and so most of STRIVR Labs’ VR training for football, basketball, football, baseball, and soccer is shot using a stationary 360 camera, but they’re looking to be able to move around as well. It’s likely that they would have to move to a CGI environment for that, or perhaps there will eventually be a breakthrough in volumetric digital lightfield capture. But for now, they’re focusing on training quarterbacks, goalies, watching pitches, and shooting freethrows.
One big challenge facing STRIVR Labs is that their sample size for elite athletes at the collegiate and professional level is still pretty small, and so determining the optimal combination and sequence of physical and virtual reps is one of the biggest open questions that they’re still trying to answer. This could explain a big motivation for why they’re considering expanding and scaling into high school training as well.
As the 360 video capture process evolves and becomes more mainstream, then there’s not going be a lot of technological barriers for other competitors to start to enter into the sports training space, but knowing the optimal training combinations and VR production best practices is going to help STRIVR Labs maintain their current leadership position. And just as Sabermetrics revolutionized the ability to more objectively track impactful baseball players, then I expect that STRIVR Labs to come up with their own set of new objective measurements that use VR technologies to track the progress of learning and performance of elite athletes. And given the objective success that VR early adopters have seen, then we can expect that virtual reality sports training is here to stay.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye, and welcome to the Voices of VR Podcast. So virtual reality is a great medium to be able to do training applications, and the big question is whether or not doing something in VR is going to be able to be transferred over into real life. And so there's a VR training company that started as a master's thesis project and then kind of virally spread into going all the way to the NFL and training some of the most elite quarterbacks in the world. So we're going to be talking about Stryver today with the chief science officer, Michael Casale, and I had a chance to actually try out a demo of Stryver at the Experiential Technology and Neurogaming Conference. And so we'll be talking about some of the transfer learning and category learning concepts that are involved within Stryver, as well as kind of their journey into virally spreading into a lot of different major national football league programs. And so that's what we'll be covering on today's episode of the Voices of VR podcast. But first, a quick word from our sponsor. This is a paid sponsored ad by the Intel Core i7 processor. VR really forced me to buy my first high-end gaming PC, and so Intel asked me to come talk about my process. So my philosophy was to get the absolute best parts on everything, because I really don't want to have to worry about replacing components once the second gen headsets come out, and the VR min specs will inevitably go up at some point. So I did rigorous research online, looked at all the benchmarks, online reviews, and what I found was that the best CPU was the Intel Core i7 processor. But don't take my word for it. Go do your own research, and I think what you'll find is that the i7 really is the best option that's out there. So this interview with Michael happened at the Experiential Technology and Neural Gaming Conference that was happening in San Francisco from May 17th to 18th. So with that, let's go ahead and dive right in.
[00:02:10.285] Michael Casale: So I'm Michael Casale, I'm the chief science officer of Stryver. At Stryver we're simply using virtual reality technology to train athletes both professional and at the collegiate level and possibly beyond that. The idea there is VR basically provides an experience that is as close to on the field as you can get, the more exposure our guys can get to being on the field without actually having to suit up and go through the machinations of practice, the better their performance, we believe, is going to be at the end of the day. So it's basically all in the spirit of augmenting performance for our athletes that they wouldn't be able to get access to otherwise.
[00:02:53.057] Kent Bye: And so can you tell the story of how this was founded at Stanford University and how you were able to kind of prove out the technology with the football team there?
[00:03:00.546] Michael Casale: Yeah, so it's kind of a cool story. So a friend of mine, a friend from when I was doing a PhD at UC Santa Barbara, Jeremy Baylinson, who's now a professor at Stanford University, was a postdoc. in a virtual reality lab at UC Santa Barbara. We became friends just by association. So we were in the psychology building together and we've always kept in contact. I was always fascinated by the work that they were doing just from a purely aesthetic standpoint. It was always cool to see what those guys are up to. So I was interested in his work and he was very interested in the social elements of virtual reality. how social interactions are affected and how you can even start to manipulate some of the factors in VR that can produce different behaviors than you would get in the real world. So all that stuff was pretty fascinating to me. Fast forward 10 years, he was advising a master's thesis student, Derek Belch, who was a former kicker on the Stanford football team. and basically approached Jeremy and asked, you know, can we do this to train football players? Jeremy was like, sure, I think, but I'm probably not the best guy to ask. I know a guy who has a little bit of a athletic background and knows something about learning and training, so that's when he contacted me. I helped advise Derek on his master's thesis that he and Jeremy were working on. And this was a really early pilot study, effectively using VR to train these quarterbacks. So very focused situations, we ran them through these scenarios and then tested to see what they knew before their training and what they knew after their training. And if there was any discernible improvement in their performance, it looked like there was. But the design wasn't extremely rigorous and you know obviously as a academic I kind of wanted to go forward and Derek's like great You know that's you know We'll kind of keep in touch as this goes along and we did muse for a little while around Things we could do next to kind of extend that line of research and really understand if it works well in the meantime you know Derek you know has some contacts in the football world and Started just talking and not even necessarily pitching to these guys this idea of you know We want to kind of test this stuff out and they were all over it and before we knew it They were asking us to build out content. They were asking us to use it as part of their real training so it was a kind of this unique situation where before you know, we had all the data and these guys were just wanting to kind of use it already and And frankly, from my perspective, what better way to assess whether something works or not than to actually put it out there in the environment and collect the data. So I saw this as a highly unique and valuable learning opportunity for Stryver. And then, frankly, a commercial value for Stryver as a company. And so Derek, decided that this was something that we could really do at scale, formed a company, and then basically just started creating content for all these teams. And of course, they would inform, you know, what content, how we were going to do it. And technologically, you know, we've learned a lot about what is out there that is going to be valuable, how to create efficiencies and creating the content, that kind of thing. But still, along the way, through these teams using, even just anecdotally and qualitatively, we've learned a lot about what's working and what's not. But as far as the future of Stryver and understanding the effectiveness, I mean, that work continues on from last year or the past year and a half. And we continue to extend Derek's work to really understand, you know, where is it helping these guys? How are they finding value in it? What's the right data to collect to actually speak to that? So in the background, we've been able to kind of form the successful company, along with, in parallel, doing some of the research that really speaks to the effectiveness of Stryver. And that's really how I've been involved, is to guide our science and our research endeavors.
[00:06:50.215] Kent Bye: What are some of the qualitative and quantitative results? And I know that there's been some anecdotal stories of Stanford football team using it and then improving either past completed percentage or even their win-loss record. But looking at the actual data, how are you able to measure the impact of what this is doing?
[00:07:07.799] Michael Casale: Yeah, I mean, obviously, I'm not at liberty to talk specifically about some of the players and that since, you know, they're our clients. But, you know, generally speaking, we have a handful of the quarterbacks who really advocate for situational instances. So what I mean by that is When these guys train during the week, things like correcting their throwing motion and their footwork, those are things that are obviously not only kind of motor in nature in terms of their performance, something that we're not going to be able to work on right now, but those are more generalizable things that they could work on in the offseason and independent of Stryber. Where Stryber really has been beneficial is for specific situations against specific teams, that they can prepare in a more robust way. So again, instead of having to get on the field for hours a day, these guys can jump in, strive, or look at that one play five times in a matter of 10 minutes, where it would take them half the practice to do that. Or sometimes they don't even get to that in practice. So it's really augmenting what they're experiencing on the field that has been really valuable. And guys have come out and even said, and I'm sure you can look at the mass media to get this information, but these anecdotes where players were preparing for a particular type of blitz during the week, and then that blitz came up in the game, and they felt so comfortable. It's that level of comfort and familiarity that they're making the right decision in the right amount of time that they really value. Now, as far as the quantitative data, I mean, that's something that I think is a bit of a work in progress for us. What we really wanted to do during this past season was let the users really define how they found value in Stryver. So we had a thought about, you know, if you use this to practice the plays that you can't get to, or even the ones you didn't quite feel comfortable with during the week, we think that could be a value. But we found all kind of different use cases. Guys studying themselves, so they're just kind of looking at themselves, was kind of an interesting one. And then there's been others as well. But again, it's nice to be able to attack a number on that. So there's a little bit more certainty about how you're possibly improving performance. The unfortunate thing I think that we deal with, certainly in football, and this is probably true of other sports, is really digging into the more nuanced type of learning. And so when you see stats get published, on the web or in any kind of major sports outlet. It's typically around balls and strikes, completions and completions, interceptions, shooting percentage, that kind of thing. And those are tremendously multidimensional numbers. They involve a lot of different variables that we're not directly training. So what we've really done systematically over the past, probably this year, and I would say probably more than in the year past, is really trying to figure out what are the relevant data types we can really use to say that players are performing or not up to where we want them to be performing. And I think even teams themselves are generally interested in this concept. So we've really spent a decent amount of time brainstorming around what are those metrics that we can collect that are not only unique to being in the VR, but more generally, understanding kind of knowledge content. Like, do these guys know when they step up to the line of scrimmage, they see a particular formation? Do they understand that that's formation A, or formation B, or formation C? How do you measure that? And then how do you know that Stryver's helping with that? So that's kind of the spirit of the research we're really undertaking now. So instead of just waiting for the game to come and saying, Well, they threw an interception and two touchdowns. It's very unclear what Stryver had to do with any of that. We're really kind of dedicated to creating more focused types of data collection so that we can really speak to the value of Stryver more intelligently than we can now. But again, last season was more about letting the players and teams and the coaches dictate how they found value in it, and then go back. So it's been this bi-directional thing, which has been great, because I think us guessing why there's value or where there's value is less good than if we let the players and coaches themselves kind of dictate. And then, of course, we can figure out the ways to measure that. That's where our expertise will come in.
[00:11:12.740] Kent Bye: Yeah, the thing that was striking to me and just trying this out was that third person perspective where you could either be looking at the field from that same perspective, but also turn around and look at yourself if you were the quarterback and be able to analyze yourself. But that there seems to also be a strong emphasis of having the coach be there right with you, stopping things and seeing if you're noticing things. So talk a bit about that interactive teaching component that you think you're kind of really building into Stryver.
[00:11:36.003] Michael Casale: Yeah, so I mean we're not really trying to reinvent the wheel here with respect to the training And so as you just alluded to a lot of the best practices as far as what coaches are doing to train Their players in general, but quarterbacks as we've been focusing on specifically. We're trying to incorporate that into Stryver so when you see things like, you know, the quarterback meeting that's a very common thing these guys meet and a couple of times a week to kind of hit home the major points about the team that they're going to be preparing for that week. So it's really about what are the opponents doing, what to look for, where to focus your attention, what's likely to be kind of available in terms of open receivers or the running game, etc. So we don't want to lose that. So we try to also build Stryver with the idea that there could be that kind of interactivity in mind. And so, again, having a guy drop X's and O's on a board, which is something coaches still do, but even if we move past that, it's not much better. It's still two dimensions, third-person perspective. Having you know the player requiring them to do that mental gymnastics to say okay now How does that translate when I go on the field? What does that look like well? We've skipped that what we've hopefully done is actually then Put them on the field so that they're there together the coach and the player and we can do that at any time of the day Because we have the content they can just jump in the headset and recreate that experience And we think that learning is going to be much more effective so when a coach says I look for the safety. Okay, now I know when I'm in that first person perspective on the field tomorrow, I know where to look for that safety instead of, again, remembering the X's and O's and then kind of trying to translate that to what you're actually viewing in the first person. So we think that's going to be minimally helpful training tool. But again, it's the other stuff, too. I think it's that immersive experience that's kind of hitting at a level of depth that's greater than what these guys are seeing even on an iPad or, again, on a chalkboard or even on a video screen. It's more real to them. And indirectly, there's a lot of literature looking at different types of learning and how much more robust and durable that learning is once you start to get kind of that depth of encoding. And we think that's kind of what we're doing. Again, these are hypotheses to be tested, I think. one of the imperatives for Stryver this year is to really start to collect that data to demonstrate that that's what's happening, that we have that level of effectiveness.
[00:13:56.869] Kent Bye: One of the really interesting things that I read is that in the NCAA, they have limits for how long quarterbacks can actually be on the field and practicing, and that there's no restrictions or limitations for how much additional training they could be doing in VR. And so it seems like a real nice fit to be able to do the real actual practice, but also go into VR and to do additional training. So maybe you can talk a bit about that.
[00:14:18.883] Michael Casale: No, and that's a great point. That's certainly where we're going to also see a lot of value is because of those restrictions and limitations and also for cases of injury too. Guys aren't able to get on the field. There's guys who've been playing for a long time in professional football and As the season winds down, their arms get tired. So even if there's not mandates, there's other reasons why guys can't get on the field as much as possible. And so we'd like to have them, again, augment their training. Or even sometimes, I'm not going to go so far as to say we're substituting for repetitions on the field. But if you can get 80%, 70% of what you're getting on the field with VR, that's certainly better than nothing. And so yeah, I definitely think there's a use case for that.
[00:15:04.717] Kent Bye: And also that maybe talk a bit about how this is impacting backup quarterbacks.
[00:15:08.839] Michael Casale: It's another great point and so maybe most critically we're finding a lot of the value in the guys who aren't getting on the field and you can just look at this past NFL season to see how valuable a backup quarterback can be with all the the spate of injuries and that have kind of plagued a lot of the teams this year. And this is probably true at the collegiate level as well. How do you deal with that? How do you get a quarterback ready? Or when guys get traded from one team to another, you're requiring them to learn a pretty intense playbook to be able to facilitate that process more easily with this technology in a way that they can take into their hotel rooms the night before the game, that kind of thing, we think is a hugely valuable thing for the teams.
[00:15:47.451] Kent Bye: So at this point, you're doing a live capture, a 360 video. But I can imagine a time in the future when you would want to move to more of a CG model, where you could actually just program things in, rather than actually being on the field to capture things. So maybe you could talk a bit about where you're going in the future.
[00:16:02.172] Michael Casale: Yes, that makes me smile when you say that, because when we can really nail that down, I think we've really started to feel like we've created a real value proposition in training. I think one of the limitations right now, and this is true for anybody doing VR for any training scenario, is those who can create their content in CG, they're at an advantage, but that typically exists in the gaming world. when you have to do it for folks who are requiring and relying on you to create really realistic renditions. In this case, we're actually capturing live video. You really need to nail that. Like anything that's suboptimal in terms of the realism is not going to be used at all. It's really kind of binary. It's either good to go or it's not useful at all. There really is no in between. And again, one of the limitations we have right now is that we can't create all of the content that a team would want because When they train for the week, they're still looking at video because that's the only format where they can look at opposing teams' defenses. They can recreate some of that on the field in practice that we're capturing, but there's a whole world of content that they're looking at that's critical for them to understand tendencies, to understand week in, week out, how their opposing teams are actually preparing and running defenses, or if you're on the other side of all offenses, And so being able to train with that data, that content also, would be huge for us. Because then that kind of eliminates that side of that training. And I think the teams would want that too. It's just they use the video and that film because that's where they can get information about opposing teams the best. But yeah, the world of creating that content digitally is certainly, I think, going to increase the value so much more of this as a training platform. But it's going to take a lot of work to get there. We've already started down that path, and we're pretty confident that that world will arrive, but it's not quite here yet.
[00:17:54.630] Kent Bye: And so maybe you could talk a bit about your own background and how your expertise was helping create this VR training applications at Striver.
[00:18:02.773] Michael Casale: Yeah, so I mean, there's nothing that anyone's really doing directly looking at the training of quarterbacks and other elite athletes with this technology, since the technology is fairly nascent. That said, there's a lot of learning from the research that's out there on the field of learning, and so that was my background and my expertise. So I have a PhD in psychology focused on visual learning, so a lot of how people learn to categorize objects when they see them in the world, novel objects. And so you can think of a lot of the decision making that we're training for as a categorization problem. You see guys align in a certain spatial formation. How do you know how to react to that? How do you know how to behave to that? And so there are things around how do you deliver feedback about whether they're right or wrong. How do you structure the repetitions optimally so that you're getting kind of the best type of learning? So an example of that would be, you know, one way to train is just to show a lot of repetitions of the same look over and over and over again. But what happens when they step on the field and it's a little bit different? While they're going to be expert at detecting the slight differences in position, the inability to categorize that slightly different formation as the same defense is no longer there. So from the category learning literature, one of the things you'd want to think about doing is creating a broad enough variety of category exemplars so that you can optimally train so that when there's little deviations in that defense, not only can you detect them, but you can also say, that's part of the same category. Or like, no, that's totally different. Got an audible out of my formation, that kind of thing. So it's learning how to behave to things you see in the world. And so there's some relevance there in some of the stuff that I was studying with a guy named Greg Ashby, who's an expert in visual category learning. He has his cognitive neuroscience lab at UC Santa Barbara. So I think there's some overlap with that. I've also worked in industry where we were developing basically experiential gaming platforms. So these were for the therapeutic purposes. So in particular we were developing a game for children that were high-functioning that had autism, spectrum disorder. So these are kids who are typically not that great socially. And then the question is, well, how do you get them better? Well, the typical way that you provide therapy for them now is you send a therapist in the home for many hours a week. It's cumbersome. Therapists have to travel. It's expensive. A lot of families don't have access financially or even geographically. And so what we were hoping to do is to help somehow mitigate some of those drawbacks of the current way therapy is being delivered in person with an automated gaming platform. And so I learned a lot also about what are the ways you have to engage people, certainly even on an emotional level, so that it starts to feel real. Again, if you are a therapist, like any teacher in a classroom, and you're telling your child, hey, this is what you should be doing, don't do this, It doesn't resonate as well as if you can actually experience that and then say, okay, stop, give them that corrective feedback. There's a lot of literature demonstrating how much more effective that is than just rote instruction or even text-based instruction. So I think for us that's kind of also key is, again, we're trying to move to more immersive environments because there's a school of thought that says that learning is better, it's more translatable, it's more durable. And that's kind of the hypothesis we're working on. So those experiences taught me a lot about then how to kind of incorporate and provide the right kind of content in the right way for these training situations.
[00:21:40.648] Kent Bye: And so does that mean that you're doing some sort of like spaced repetition in terms of as people are doing stuff, you have a certain way of knowing whether or not they remember it? Or are you getting feedback from them of how strong the memory is? Or are they being quizzed by the coach before they go through it? Or how do you get feedback or input in order to determine the spaced repetition?
[00:22:00.210] Michael Casale: That's a that's a great question. And so yeah, I mean, so space repetition, also things like desirable difficulties, you know, you want to make things a little bit harder than they have to be, because that also perverts a certain, in some instances, better type of learning. And so I don't think we know exactly how some of those phenomena translate directly onto the training that we're involved with, but we're learning that. So certainly some of those parameters we're starting to explore a little bit more. So as of now I couldn't tell you, you know, across the board this is exactly how you should be training. Because we're also learning from the teams themselves in terms of, you know, how are they finding value, how many times a week do they use it, and what is the spacing like so that they're finding a lot of benefit. And I think, honestly, it's going to be more of a bi-directional thing, encouraging them that, you know, there's an indication that this might be the optimal way to train, but then also have them maybe train that way and figure out, okay, that wasn't the best. It turns out we actually want to reduce the spacing if it's spacing, or we want to increase the difficulty. And we can do that through kind of, you know, maybe speeding up the film or, you know, occluding part of the film. So there's all these techniques. that have come from the learning literature in the research world have shown to impact learning, and we want to be able to start to translate those into our training regimen and our training paradigms. But there's a lot to be learned. I mean, you know, as you probably know, moving from the lab into an applied situation, there's more variables to consider. And, you know, getting the right type of data to validate whether or not that particular training paradigm, whether it's spacing, et cetera, has helped is also something that's a bit of a challenge. So that's a work in progress for us. We have some thoughts about what might be working, but I wouldn't want to commit to that we found the things that are the best practice uniformly across the board. I mean, that's something that we'll probably continue to learn, frankly, as we incorporate new teams and new players, because it's not like we're doing this with thousands of people. I mean, this is on the order of tens, if maybe hundreds of folks. And that's still not a ton of data that we're getting from them. So I think as we go along this path, it's really critical for us to kind of learn, but also to have these hypotheses in mind that come from the research world. So it's not like we're just starting from scratch. There's some indications that, yes, spacing is something that's important. Let's keep an eye on what spacing is working, what spacing is not, and other kind of like learning parameters that we can start to understand how they impact our type of learning. Because again, just because they impact things in the lab doesn't necessarily mean they're going to translate to the things that we're doing in the training room.
[00:24:29.066] Kent Bye: I can imagine that at this point from seeing some of these early demos from different sports that there seems to be an emphasis on stationary perspective. So the quarterback is in one generalized space in the pocket. The hockey goalie is in one space in front of the goal. Someone who's batting is standing in one spot as the pitcher is pitching the ball. And when you're shooting a free throw, you're just standing at the line and you're shooting. Are you getting demand for people to have other positions, but running into the issue of VR locomotion and the simulator sickness that comes when you start moving cameras around?
[00:25:00.489] Michael Casale: Yes. And it's another great point. I think solving that will also, kind of again, provide more value for training with VR. That is a limitation. But what we have done is we started to kind of inch our way forward into doing that kind of translation in the headset. So being able to really figure out what it has to look like so that when they move the scene moves with them kind of thing. It's technically it's a bit of a challenge obviously. But even little, what we're finding, and this is just anecdotal at this point, is even little movements, being able to kind of move a few feet back and have that seemingly move, that's powerful. It creates a level of immersion and, you know, again anecdotally from the feedback that we get from the players themselves, it starts to feel a little bit more like they're living in that world. But there's a whole host of other things aside from, like, the locomotion. Because I think that's just a, we'll see what happens, but that's going to always be an issue, minimally because of the physical space you're in, right? Even if you can figure out how to do that translation well, you're never going to put VR and then be on a football field, right? I mean, maybe. That's not too far-fetched. But I think for now, we're always dealing with the space constraints. And so even if you wanted to run 10 yards, you got to go get a room that's going to be suitable for that. And that's not always trivial to track down. But as far as there's other things, I think that can really augment the experience as well that we're starting to just broach. And these are things like when you move your limbs in space, you can all of a sudden start to see your limbs in space. And I think haptics is another big part, right? incorporating other sensory modalities again all within the spirit of creating that immersiveness so that it really does feel real and then you know down 50 years down the road when it is real basically people won't be able to tell the difference but yeah we're just like I said we're just kind of starting to inch our way toward that you know the things like locomotion and being able to move and have the scene move with you in the headset Because you're right like as a gamer or anybody who's gaming right now with VR That is a big issue when you're starting to move even violently so in the virtual space yet You're sitting still there's that discordance kind of creates not always the most pleasant experience for the users
[00:27:17.138] Kent Bye: So I was really struck and surprised by, you know, the basketball training where it's essentially just shooting a free throw over and over and watching the ball go in. And so what is the value of visualization? What is a concept that has been proven out in the cognitive science literature that if you're able to visualize that that has some impact on training?
[00:27:34.403] Michael Casale: Interestingly, basketball free throw shooting is one paradigm that folks have used. I'm not going to say that it's a hugely robust and certain finding that visualization always helps improve performance. You remember a lot of these studies are done in contrived settings, so it's not like they had these guys visualize and then tested them in the game, which is really what you want to do. things that are kind of like on their way to being in the game, so in a practice scenario, and they do show improvements. Again, I don't necessarily think that the data that's out there is, I mean, there needs to be kind of more of that and like any kind of cognitive phenomena, you know, really doing as much replication as possible to bear out that result. But I do think there's a lot of literature on just the idea of mental imagery in general and what it's doing. And what we do know from the neuroscience literature, there has been a decent amount of work done to look at certainly the brain areas that are involved. So the neural circuits and that they do overlap with what we see when people are actually performing motor movements. I know a lot of this work is done in non-human primates, but even in the human world, we do have some evidence that when you engage in motor tasks, then you imagine those same motor tasks, that we see overlapping circuits being activated. I mean, this goes back to the mirror neuron stuff. But even beyond that, that you're activating some of the motor programs. Obviously, you're not initiating the motor programs themselves. So what we're hoping is happening is that when you're imagining something, or in our case, when the imagining is being done for you, you're hitting on those same kind of practice circuits that then are going to be used when you step on the field and engage in that mode of performance. So that, you know, we heard a lot today from folks who are doing kind of neural priming, that kind of thing. Well, we think we're actually doing that by showing you hopefully as close as possible of a realistic simulation of what you will see when you step onto the field. So you are hitting those same at least kind of like premotor parietal circuits that are going to be involved once you actually have to perform that movement in real life. And again, that's some of the evidence that we're using to support the idea that what we're doing is actually going to facilitate real life performance.
[00:29:46.598] Kent Bye: And from an outsider's perspective from the sports world, but coming from the VR world, I see that Stryver is putting a 360 degree camera on the field and then you're adding a lot of different cognitive science and learning principles in the system that you're building now and into the future. But what's to stop anybody from just putting a camera onto the field and calling it a day?
[00:30:07.954] Michael Casale: Absolutely nothing. And in fact, you know, that's happening. What we are seeing though, interestingly, is folks that don't have much experience in VR or kind of wanting to get in this space, they just don't know how to do it. And so, anybody can put a camera on the field, not everybody knows where to put it, not everybody knows what content is really critical and what to film, and then also how to view it. So, again, these are things that we're gleaning along the way, are kind of those best practices. What's the right way to train with VR? And not that we've completely solved that problem, but from having a pretty good amount of data from just this past football season, and you know, we're broaching other sports as well, but even in the football space, we're learning that there's a certain cadence, and you mentioned things like spacing. I mean, when to do it. Do you want to do it early in the week? Do you want to do it later in the week? How many times do you want to do it? Do you do it right after practice? Do you wait three days after practice? you know, of the thing that you practice. So, for example, if you went through 40 reps of training with Stryver and then went on the field, is that better than going on the field first and then doing 40 reps with Stryver? I mean, these are questions we can start to answer now and that's why we think we're uniquely positioned to really create a better experience. And again, all the trial and error stuff that we had to go through with just even filming the content, camera position, like I mentioned before, and then, you know, I think folks have been putting cameras on people even, and that's created an interesting experience that's not optimal. And, you know, again, if you don't know what you're doing, I mean, and you have a camera, things could go really well, or things can go really poorly. Jeremy Bailenson likes to say, you know, it's like uranium. It can heat your home, or it can destroy the world. And I think we're seeing a lot of people not quite destroying the world, but certainly creating suboptimal experiences. And I think one of the cool things about Stryver is that we just want to do the thing that's good and that works. I mean, we're not interested in and kind of, you know, just selling a technology and we're never going to be a technology company. What we are selling is insight and learning and through careful kind of deliberation understanding kind of what works and what doesn't work, and then building out the features in the technology that we know are useful. We're not just creating, you know, gadgets and bells and whistles for the sake of. I mean, these things have to work and they have to have value for the teams. We're a very results-driven company, and I think that starts with our CEO, Derek. He really is the most no-nonsense guy I think I've met. And that sentiment has certainly trickled down to the rest of our team. And we just want to do something that works. I think that's what we're all in it for.
[00:32:39.792] Kent Bye: And finally, what do you see as kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:32:45.855] Michael Casale: Well, I mean, a lot of scary things. Personally, I think one of the cooler things we can do and what I've always wanted to do with VR is to recreate someone else's experience. I think the ability to create empathy and not just an understanding that someone's going through a hard time or someone has difficulties in their life or someone You know is experiencing whatever they're experiencing but that you get to live through that That you are kind of hitting on those same emotional circuits or at least with enough fidelity that it's real to you To kind of provide a better understanding. I think you know people talk about things like we'll go travel the world. It's really good You get a perspective on life I mean the reason for that is because you can actually kind of experience a little bit of what those people are experiencing less good than that is watching a National Geographic documentary. Less good than that is reading a book about it. And less good than that is not knowing anything about it. So I think again, you know, the more we can understand about how other people think and feel and what their experiences are like, I think, you know, who knows exactly what's going to come from that, but it definitely feels positive. And just from a personal anecdote, I've always tried to do that. Just, I mean, I don't even make an effort to do that. It's almost like an automatic thing. I always try to figure out, you know, where people are coming from. for one reason or another and I see that other people don't have that capability or not always you know in that same train of thought and I'm not saying it's you know gonna solve all the world's problems but I think people's inability to kind of understand other people's points of view is the source of certainly some of our social problems and and I know Dr. Bailenson's a big advocate of that too so I think creating empathy is something that I would like to see happen more of in the world of VR.
[00:34:30.445] Kent Bye: So anything else that's left unsaid that you'd like to say?
[00:34:33.166] Michael Casale: Yeah, I mean, I think it's an exciting world. And I think I'll go back to Jeremy's quote. I mean, this could be a really powerful tool and really do some good things. But I also think that with any technology misused, it can do some bad things. So it's nice to see and come to a place like this, that this technology is at least in the hands, sometimes, in the right people's hands. So I'm hoping that that sentiment continues. I hope we see more positive work coming from these technologies. And anything I can do to contribute feels pretty good. So I'm pretty happy to work in this space.
[00:35:04.820] Kent Bye: Awesome. Well, thank you so much. Thank you. So that was Michael Casale. He's the chief science officer for STRIVR, which is a sports training application using virtual reality technologies. So I have a number of different takeaways from this is that first of all I had seen a lot of different articles about Stryver in the news over the last couple of years and surprisingly a wide range of different coverage from like all the different major sports magazines and Fox Sports and it was one of those VR companies that was getting a lot of huge mainstream press and I was really excited to actually try out some of the VR demos here at the Experiential Technology and Neural Gaming Conference And so one of the things that I noticed that I mentioned is that you're not moving around in any of these scenarios. So you're either a quarterback, or you're the goalie of a hockey game, or you're looking at baseball pitches, or you're watching free throws being shot. So there's a lot of different VR locomotion issues that need to be solved in order to really kind of take it to the next level to be able to do something with the wide receivers. And I'd imagine that something like augmented reality could be something that is useful and helpful to be able to use the real world and perhaps use some augmented reality glasses to be able to actually use the physical motion on the field, but to do some supplemental training and perhaps having the coaches giving some direct feedback. So that was the other thing that I really noticed in the demo that I had with Stryver is that there was a lot of interactivity that was happening with a coach standing there pointing out different things. And so that was one thing that Michael said is that being able to actually have a direct experience of a play and be able to have a coach kind of interrupt and interactively ask you questions and point out different things. And I think that is one of the things that I found really interesting that It kind of goes beyond just sticking a 360 degree camera on the field and then coming to know what to do after that. I mean, one of the big value adds that Stryver is giving is a lot of this research into learning and spaced repetition. So, spaced repetition I didn't really go into too much, but the idea is that whenever you learn something, you are going to forget it at some point, unless you kind of remind yourself. So if you think of flashcards as kind of like the most basic form of repetition, but the thing that you don't want to do is kind of look at cards that you already know. And so the idea with spaced repetition is the moment that you really want to actually revisit something that you are trying to remember is at the moment you're just about to forget it. There's programs like SuperMemo that keep track of how strong of a memory you have of different information and then you're supposed to use these type of spaced repetition things every day such that you will be reminded of things that you're just about to forget. So that's part of the big open question when it comes to training quarterbacks is how often do you do these types of space repetition? What is the balance between actually doing things on the field and real life to get that actual repetition, which according to Michael seems to be the end all and be all like that is in some ways the most effective amount of training, but. People only have limited capacity to be on the field. In colleges, quarterbacks can only be on the field for like 20 hours. And so you can supplement that with virtual reality training. But they're kind of trying to figure out what is the best combination between doing this in real life and also supplementing it with some of these training applications. The other thing that was really striking to me was just some of the metrics that they're looking at in terms of objectively tracking progress. It makes me think of something like sabermetrics within baseball, which is essentially this idea that, you know, growing up we have these baseball cards with batting averages and all these other numbers that are trying to quantitatively describe the performance of baseball players, but yet sabermetrics comes along and tries to come up with a whole set of new objective measures that are trying to describe what would actually make a team or player do well and succeed and win more games. And so there's the Moneyball book and film, which kind of looks at using some of these different sabermetric statistics to be able to start to measure and quantify athletes in different ways. And they had a lot of success with that within the Oakland athletics, which was covered in Moneyball. Now with football, I think there could be eventually some new objective metrics that are used to either help track and train quarterbacks over time, but also help to determine whether or not some of these scouts are going to be able to have the skills that are necessary to be a quarterback. I think. One of the most explicit ones that was mentioned here and has been mentioned in some of the mainstream press is as the quarterback is being able to look at the defense and be able to dynamically change the call based upon the defense that is happening. And so they're supposed to be able to recognize and determine some of these different formations. And so that was what Michael was calling category learning. So in other words, being able to look at the field and be able to categorize what is happening and be able to see slight variations and to be able to know whether or not it's significant or not significant. So in other words, just the distance of one player, does that mean that they're about to blitz or does that mean that it's just like some of their normal formations where they would normally be? And so this is some of the skills that expert and elite athletes are able to cultivate, is this type of category learning. Now, when I was able to actually step into and look at from a first-person perspective what a football quarterback is seeing, I have none of this category learning experience, and so I was really quite lost as to what to look at, what to pay attention to, what's salient, who is open, and are there any things that I'm supposed to be really extra paying attention to. And so I think that having some of these repetitions with a coach there, something like Stryver is going to be able to help not only train the quarterbacks, but I think also help the coaches be able to more dynamically and interactively coach some of their quarterbacks and to be able to stop and give their kind of expert category learning insights into what they should be paying attention to. Also just the fact that a lot of backup quarterbacks who don't get to get that on field time as much as the primary quarterback because you know The primary quarterback is being prioritized to be able to you know Be the one who's getting the most repetitions on the field but if they get injured then the backup quarterbacks need to have a at least some level of awareness of some of these different plays. And so virtual reality is actually playing, if anything, a bigger part with some of these backup quarterbacks to get them up to the level so that they're able to just step in and feel comfortable with being able to call the plays and be able to execute them. Also, I think an important thing to point out is that there's going to be different quarterbacks who have different learning styles. And so I'm not quite convinced that there's going to be a universal way to be able to have one system that works equally for everybody. I think there's going to be different quarterbacks who have different ways of learning, whether or not they're kind of embodied, or they're doing it through their direct experience, or they have a mental conceptualization of what is supposed to happen. So I think it's possible that someone who is a more direct, active, experiential learner will get the most out of doing those repetitions directly on the field. And maybe somebody who is more of a mental or conceptual learner may get more out of doing some of these visualizations within VR. because there's such a small sample size of elite quarterbacks in the world, I'm not sure if they're ever going to find the one perfect learning solution that's applicable equally to everybody. And so it may be one additional thing as people are doing these virtual reality training, maybe we'll start to learn some of the different unique ways that people are learning and how you might have to adapt to the learning regimen to be able to figure out what the space repetition should be, what the variation of difficulty should be, as well as some of the other techniques that Michael mentioned for category learning that include occluding different parts of the film as well as speeding up some of the footage. So the big takeaway for me is that something that Stryver is doing is essentially a training application that the principles of transfer learning as well as category learning are going to be applicable to many different domains that go beyond just elite sports. It just elite sports is the thing that has the most money and is there providing a direct benefit to a lot of these athletes that if you think about it, you know, the existing coaches were drawing X's and O's on a whiteboard and then up to the players to be able to, like an architect, translate that blueprint into what that actually looks like in 3D on the field. You can kind of just skip that step with Stryver where you start to actually record some of these repetitions on the field and then be able to actually directly experience it. So you're kind of getting a more 3D first-person perspective of that, which is much more immersive and I think more effective in terms of actually running a lot of these different plays. I think the next step is going to be doing something like volumetrically capturing the defense and digital light fields and to be able to actually step into a virtual reality experience where you're actually watching some of the defensive plays within a VR experience so that you're actually seeing it from that first person perspective. That's something that I think is going to be technically feasible at some point once we get some volumetric cameras that are able to give that level of realism. I think we're pretty far away from that, but I know that I've talked to Virtually Live, which they had some of these technologies that were able to take optically tracked players on a soccer field and translate the data that's coming from data.com because, you know, there's these websites that are tracking who's got the ball where they at in the field and then they can have the set of numbers and then translate that into a Virtual experience with virtual characters and so I think it's just a matter of time before they're able to do something similar where they're able to track the defensive plays or an offensive plays and be able to put both the defense and offense onto the field and be able to actually experience a the opponents and being a new way of studying them. So instead of doing research within film, I think some of this kind of competitive research is going to be happening within virtual reality technologies once the technology gets to the point where they're able to take live captured footage from previous games and be able to actually put it into a VR experience for them to experience. That's where I kind of expect this to go eventually at some point doing some more CGI dynamic immersive experiences where they're able to do that. I think it's probably the first intermediary step, but I think eventually we're going to be looking at like digital light field capture and having people have a direct experience of some of these plays. And not just some of these professional athletes. I think that there's going to be a demand to be able to be immersed on the field and be able to watch certain replays, certainly throughout all different sports. So, that's all that I have for today. I'm going to be at PAX West this coming weekend, both Friday, Saturday, Sunday, and Monday. I'm going to be roaming around PAX West, looking at all the different VR demos, and so if you have anything there that you'd like me to come and check out, then drop me a note at Kent at Kent Buy, or on Twitter at Kent Buy, and I'll try to check it out. And if you'd like to support the Voices of VR podcast, then spread the word, tell your friends, and become a donor at patreon.com slash Voices of VR.
