Machine Learning is going to revolutionize so many different aspects of our lives, and it’s starting to enter into game development with IBM’s Watson. Developers can integrate cloud-based AI services into their game to dynamically change the game design progression curve based upon a user’s behavior and performance. If the player is zipping through a series of easy puzzles with no problems, then Watson could detect that and more quickly progress the player to advanced levels in order to keep the game challenging and interesting for them.
I was able to get a sampling of how a number of different innovative game designers have started to integrate machine learning resources last week
at an Intel Buzz Workshop presentation by IBM’s Interactive Media CTO George Dolbier. He showed off some code sample of how to integrate Watson with Unity with IBM’s Watson Developer Cloud API and gave a number of different use cases for how to integrate machine learning into VR experiences.
I caught up with George to talk where machine learning networks can add value, the future of interactive narratives with AI chatbots, and conversational commerce and the future of conversational interfaces in the Experiential Age. Ars Technica recently premiered a sci-fi short film that was written by a recurrent neural network, and George and I also talk about how AI systems like Watson have the potential to empower humans to do more of what humans do best with our imagination and creativity.
LISTEN TO THE VOICES OF VR PODCAST
Here’s some of the Unity code that calls the Tradeoff Analytics API as a part of the Watson Developer Cloud.
Unity API call to a #MachineLearning platform inputting 3 variables to dynamically tune #gamedev progression curve pic.twitter.com/zdUVqUYzuu
— Kent Bye (Voices of VR) (@kentbye) June 23, 2016
Here’s a brief marketing video about the Watson Tradeoff Analytics feature that George talks about in the podcast:
https://www.youtube.com/watch?v=lWhRW5TNdGw
I curated a Twitter list of over 100 AI & machine learning experts, and I’ll be tweeting more about AI on a new Twitter account at @VoicesofAI
This is best description I've seen explaining #ML's relation to #AI @kdnuggets' @mattmayo13: https://t.co/LksyYu3Q7k pic.twitter.com/vaLvforYMe
— Kent Bye VoicesOfAI (@voicesofai) June 27, 2016
Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye, and welcome to The Voices of VR Podcast. On today's episode, I have George Dolbier, who is IBM CTO for Interactive Media. Last week, I was at Intel's Buzz Workshop in Seattle, and George was giving a presentation on how to use machine learning for game development. And he was showing different ways to be able to take Unity code and be able to make API calls into IBM's Watson, which is most famous for winning Jeopardy against two of the best players on Jeopardy. So on today's podcast, we'll be looking at how to use artificial intelligence and machine learning for improving your game development, as well as the future of AI and interactive narrative, and how conversational interfaces are going to be the primary way that we interact with experiential age technology. So that's what we'll be covering on today's episode of the Voices of VR podcast. But first, a quick word from our sponsor. Today's episode is brought to you by Unity. Unity is the lingua franca of immersive technologies. You can write it once in Unity and ensure that you have the best performance in all the different augmented and virtual reality platforms. Over 90% of the virtual reality applications that are released so far use Unity. And I just wanted to send a personal thank you to Unity for helping to enable this virtual reality revolution. And to learn more information, be sure to check out Unity3D.com. So this interview with George Dalbier happened at the Intel Buzz workshop at the Seattle Impact Hub on June 22nd. So with that, let's go ahead and dive right in.
[00:01:49.383] George Dolbier: Okay, my name is George Dolbier, I'm IBM CTO for Interactive Media, and that's games, VR, animation, 3D production. And what we've been doing is taking our machine learning platform, Watson, into the games industry, and really looking at solving some of those interesting fundamental challenges in the games industry, like narrative, interactive characters, and automatic game balance.
[00:02:15.727] Kent Bye: Yeah, so we're here at the Intel's Buzz Workshop, and you gave a whole talk about applying machine learning concepts into game development. And one of the things that was really striking to me was just thinking, like, are there going to be computer programmers in the future, or are we moving to this world where everything is going to be trained up by artificial intelligent machines?
[00:02:37.787] George Dolbier: Absolutely not, especially in games and entertainment. But what machine learning platforms allow people to do is actually do what they do better. So as I mentioned in my talk, game development and working with machine learning platforms isn't so much like writing code and thinking about translating what you would do naturally into a mathematically based bottle. It's more like training a child where you collect a bunch of data and you guide a machine to be able to make good decisions. So it's much more like testing or, you know, already games are developed in a very iterative, highly iterative methodology and that's exactly what you do when you're training a machine learning platform. So developing games, especially developing interactive conversational based interfaces is more about testing and testing and collecting data and shaping how a character might respond. And it's much more creative and much more based on statistical models and probability than strict hard math. So an AI and a machine learning platform doesn't produce a deterministic output, which can be good for games. It can really increase the fun factor of games. It can also potentially make games easier to develop, where you're trying to create an experience by training a system rather than describing the experience that you want to create using a mathematical model.
[00:04:17.314] Kent Bye: So it sounds like we're going to be dealing with hybrid situation where there's still going to be computer programmers who are coding, but yet they're going to start to be making these API calls out to a machine learning neural network that's been trained up to be able to have a little bit more capability to intelligently be responsive to the player. So one of the examples that I thought was really interesting was this idea of being able to kind of dynamically choose the progression curve within a game. So maybe you could talk a bit about the problem of game development in terms of a progression curve and how machine learning can help address that.
[00:04:51.000] George Dolbier: So the problem is one of the fundamental jobs of a game designer is to create an experience that is engaging. And in simple games and in complex games, part of that experience is the progression curve. How much more difficult or complex does the experience become as you get deeper into the game? and getting that progression curve right now is a very manual thing. Game designers spend years or decades honing their craft to try to create just that right kind of curve where if you have a game experience and your first level is difficulty 1 and your second level is difficulty 1.1 and the difficulty progresses linearly, that kind of linear progression has been demonstrated over and over to be actually kind of boring and a turn-off and feel very mechanical to the player. Whereas a progression curve where you get some simple tasks and some difficult tasks and maybe an impossible task that you try and try a couple of times and then you finally get it It's those type of experiences that the game designer craft has discovered to be really engaging. So you could think of that as being drawn as a rather curvy wave or a curvy graph. A machine learning system can actually validate the game designer's hunch and the game designer's skill by actually measuring what players are doing, how they're responding to difficulty and challenges and different micro-aspects or micro-design decisions that are made to validate not only the micro-design decisions but also validate or tune that grand design for an entire demographic, for an entire audience or ideally down to the individual A challenge in the games industry is there hasn't been really a technology that allows games to react to an individual. And, you know, this session Peter Menon was mentioned in some of his groundbreaking games that took years of development to try and react to an individual and to be able to elicit emotional or philosophical reactions from an individual player. We now have technologies where Those capabilities are available even to indie developers to create a game that can really emotionally challenge an individual or just respond in a way that allows the individual to get the maximum amount of enjoyment out of a particular title.
[00:07:38.977] Kent Bye: So in the specific case study and example that you showed, there was a puzzle game that had, you know, let's say 100 or 200 different levels, and you're able to kind of train this machine learning algorithm as a person is playing it by quantitatively boiling down their performance on a certain level to, like, say three numbers. Then you're inputting those three numbers into the machine learning, API and then it's kicking back the response of like okay based upon how you performed in this level the next logical level that we think that you might enjoy would be this one rather than having just sort of a Experience that everybody had which was exactly the same
[00:08:18.423] George Dolbier: Absolutely. I'm glad I was able to tell the story well. And that's exactly this very simple use case was Plight of the Zombie from PopCap Games. And it was really our first experience with trying to almost attack this grand challenge. and discover how much data we needed to collect both in, you know, how many interactions or the telemetry we got from the player, how many individual sessions would we need to collect data from, and how sophisticated would that data. Our initial assumptions were we'd have to collect a massive amount, you know, lots and lots of metadata about how long between each players click and how many times they retried each level and it turned out that with an already trained AI and a particular decision support API we were able to use a very small number of elements of data and a small number of actual interactions before the AI was able to say maybe not consciously, but able to determine, okay, this player is having a problem with the mechanic, and so to match the progression curve for this game, we're going to skew the levels we give this particular player simpler, or this player is blowing through these levels, and to match the overall design curve, we're going to skew the levels that we give this player harder.
[00:09:51.574] Kent Bye: Yeah, and it seems like there's always a balance between your own qualitative subjective experience of a game and then things that you can actually objectively measure. So maybe you could talk about those three numbers that you're taking a puzzle level and being able to boil its essence down to these three numbers that you're feeding into the machine learning algorithms.
[00:10:10.730] George Dolbier: Right, so in this case it was the number of retries per level, so the amount of times a player would retry the level, and then the time that it took them to complete the level. So that was kind of the barest amount of data that we could give the machine learning platform. And then we reported that data for every level the player tried. And so that really does seem to be the floor. Theoretically, we could probably make good decisions on one piece of telemetry. But again, this gives the game designer kind of part of their craft is to choose which elements of telemetry to record and to let the machine actually try to make decisions on. So ideally, you would collect every piece of behavior from a player. Again, like how long between touches or mouse clicks, how long they played each level, how many times they retried, when they stopped, when they paused. Every piece of metadata about any interaction that player had with the game becomes fodder for a system that's really good at detecting patterns. The more data you can give it, the better it can make decisions. What we found with a simple test and with a simple game scenario, we can really get to a small amount of data and a small number of data collected. And that means a lot to indie developers and developers that are trying to get products to market quickly, because you don't have to spend a lot of time creating that framework, collecting that data. You can start to see trends and patterns very quickly. And that really goes to the core of some of the kind of industrial problems of the games industry. You know, how expensive it is to make a game, how long it takes to make a game, you know, testing and tuning. You want to invest in to create a really high quality game, but that's really expensive stuff.
[00:12:10.148] Kent Bye: Yeah, and it seems like what you said earlier was that you have an already trained AI decision-making. Does that mean that you took a lot of raw data of people playing through the game from all 200 levels and then trying to extrapolate decisions based upon real people playing the game through playtesting?
[00:12:27.108] George Dolbier: So the particular machine learning platform that Plight of the Zombies used is an AI that's already been trained. It's called trade-off analytics, and it's an AI that's been trained not just with game data, but it's an AI that's been trained to take a set of data and a dilemma, a question, and return a set of answers that are optimal for a particular solution. Another example is another customer that's using the same AI is a company called North Face. They make jackets and on their website you can go and ask, I'm going to go hike in the Himalayas for the first time in June 25th. and the same API that's being used by the Plight of the Zombies has all this data about North Face jackets and boots and tents and backpacking and has data about the weather in the Himalayas in mid-June and will make a set of trade-off decisions about what is the most optimal equipment. You know, you should pick thick-soled boots because the trails are really sharp and rocky. And so it's a machine learning platform that has already been trained to take a set of data and be given a question and to give a ranked list of answers based on a pool of data. So in the game's case, the pool of data is these different levels, and the metadata is the difficulty metric. That's pretty boiled down. And then the dilemma is, I have this player that has exhibited this particular behavior, and then the trade-off analytics API, its job is to match a behavior with a set of data. And so that's for IBM and for Watson, that's one of 25 different pre-trained and configured machine learning APIs that are available. And others include question and answer where you load Wikipedia or a novel and be able to ask, well what did the character do on this particular date? And that's another example of a pre-trained machine learning platform that can take an anonymous set of data and then just manipulate it. Linguistics processing is another one. Specially trained machine learning platforms that can understand English or Japanese, detect not only just patterns, but detect objects and entities within human language. So it's not so much you're taking like a raw comparator, which is a basic type of machine learning process, and training it with something new, you're taking a machine learning platform that's already been trained to solve a certain type of problem, and then based on what you're doing with the game, you collect a set of data, and you use this machine learning platform to give you answers based on the data that you're collecting.
[00:15:30.134] Kent Bye: And so is this trade-off analytics API calls that specifically to Watson then?
[00:15:34.950] George Dolbier: Yeah, so Tradeoff Analytics is one of the many APIs that's available on Watson. If you just go to the Watson Developer Cloud or Google Tradeoff Analytics on Watson, we've got a number of APIs that are freely available for developers to test and use. Another great one that makes a great demo is our Personality Insights, and that's another one I talked about where it can take you know, a body of text and determine your personality, not just your personality profile, but also be able to create new text in your own writing style. Or another one is image analytics, where you can upload a photo and Watson will actually look at the photo and tell you this is a picture of a human Caucasian male of a certain age range, and he's wearing a hat and a sport coat. We've got also a great demo. We've been working with the New York Department of Police for a long time and one of our stellar demos is we can take a customer to New York's police department and police chief can stand up and he's in a room that looks like any war room from Hollywood and just talk to this bank of screens. show me every middle-aged male wearing a red t-shirt within five blocks of Times Square, and you'll get 20 monitors that show 20 different people, and then the chief will say, follow screens one, five, and seven for the next 20 minutes. And our machine learning system can actually do real-time video searches and follow objects as they move through video streams and across different cameras. A lot of these systems have been tested and refined over years now and are available on our developer cloud.
[00:17:27.326] Kent Bye: Right. And so maybe you could just give me a brief little history of Watson. And I know that I've heard of it through the Jeopardy, but I don't know if it predates that. And it seems like now it's evolved into this whole cloud platform that developers can actually interact with.
[00:17:39.953] George Dolbier: Yeah, it is something that IBM is really proud of. In IBM, every decade or so, we have something that we call a grand challenge. So historically, 20 years ago, one of the grand challenges was to create a supercomputer that could beat a chess grandmaster. And Jeopardy was the grand challenge for the years 2000 to 2010. And the grand challenge was to create a computer that could understand human language and be able to, again, understand human language at a fundamental level that it could play a game of Jeopardy. So Watson was that grand challenge. And interestingly enough, this huge research investment by IBM was demonstrated to the world through a game. And the first customer that we ever talked about Watson to was a company called Crowd Control Productions. And this was almost six to nine months before Jeopardy aired. and we talked to the lead developers of Crowd Control Productions. They were in Austin, Texas, where one of the Watson development teams was for Austin GDC. We just kind of shared this idea of this grand challenge that we were working on, and they gave us some just really wonderful use cases. How would a game company use this linguistic type of interface. And it really was enlightening for the development team that was working on Watson and trying to focus on understanding human language and reading Wikipedia and being able to answer a Jeopardy question as fast or faster and get it right. And it really did open up the minds of the development team to some of the other potentials. So the Watson Jeopardy challenge was 2011 and the first use case for Watson really was in the medical industry and the challenge for especially cancer research is So much research is being done and documented that no one individual or no one team could really read it all and understand how a particular genetic marker or drug could be applied to an individual or even a population. And that's the exact same use case that the Watson Jeopardy challenge was is ingest all of Wikipedia and several trivia dictionaries and then be trained, you know, understand that body of knowledge and then be trained to answer trivia questions. So in this case it was read every word that was written about oncology and then look at patient cases, questions doctors asked patients, how patients responded to individual treatments. And the core of Watson was marvelous at understanding human language and then mapping those two different sets of data to produce a real result.
[00:20:53.469] Kent Bye: Is there a grand challenge from 2010 to 2020? I can't talk about it, but yes, yes, there is. Nice. Yeah. So it feels like there's been an explosion of artificial intelligence in the technology wise with the NVIDIA GPUs. From your perspective, what really changed within the last, you know, five, 10 years that really hit the stepping point where, you know, AlphaGo beating least at all is something that people didn't think was going to happen for another 10 years or so, but yet, Here we are, and we're just sort of swimming in all these open AI initiatives. These new algorithms are being open sourced from Google and Facebook, and it just feels like it's just exploding.
[00:21:31.658] George Dolbier: One of the things is business need or money drives development. And what we were finding was with big data, beyond it being just a term used by consultants, most information that is really valuable to make decisions, either whether you're running a company or you're trying to cure cancer, the information that you need to make good decisions isn't just math. It isn't just available to a statistic analysis. It really requires a system to understand human language and to be able to not just understand what you're asking and return a reference to a document, It needs to understand what it's reading, understand your question, and then create an answer. And that's the basic functionality of our Watson platform. Google has driven tremendous financial and industrial value over its artificial intelligence around optimizing search. Facebook, the AI that drives your news feed, is again driving a tremendous amount of just really fundamental financial value for that company and its investors. Recommendation engines that are used by everything from Amazon to Netflix are another type of this AI, and what we're seeing is A, the adoption of cloud, giving you virtually unlimited resources to throw at intense computational problems, and the cost of storage and compute with cloud coming down so that you don't have to throw away data or even think about pruning or backing up data. You can keep everything you want and then search it. And it turns out machine learning platforms, the more data you give them to learn, the better they get. I'd like to think that the Watson Jeopardy challenge really opened people's eyes to the potential. Most people like myself, once they saw a machine being able to read human language, understand the intent of a question, and then come up with an accurate answer, that capability can be applied to stabilizing global financial markets, to, like I said, curing cancer, to designing better products, taking user feedback, and actually having that generate statements of requirements that engineers can go off and build. We also had tremendous support from the university system, whether it was UC Berkeley, UCF, UMichigan creating a whole generation of computer scientists that were steeped in machine learning and knew how to implement them. just a magic trifecta of cheap, inexpensive computational resources, more people that understood these types of systems, and then real business problems that large companies with big budgets could fund research projects and have those research projects turn into commercial products. it was just the right time. Like I mentioned, AI has been around since the 50s, and our grand challenge in the 90s was the Deep Blue system that beat Kasparov. So AI has been around for a long time, but much like VR, I remember my first VR experience was at the Conference on Human Interaction here in Seattle in 1991. And you could see the vision, but it was a VR experience that was running at 7 frames a second and maybe 50 triangles. It was a horrible experience, but now, you know, Oculus getting supported by a Kickstarter campaign and really revolutionizing and lowering the barrier to entry. And I really look forward to, you know, HoloLens and Magic Leap really advancing in that area as well.
[00:25:30.297] Kent Bye: Yeah, and Google just came out with their Google Home, and you're having these conversations with AI entities. And so from my perspective, I really see with virtual reality, we're going to start to have these conversations with chatbots. And one hashtag, conversational commerce, is something that we're starting to see Uber and other companies really start to adopt ways of just having text messages or being having, just through the course of conversations, being able to have some sort of commerce that's derived from those conversations. So for you, what do you see as kind of this intersection between these AI chatbots like Watson and in how that's gonna play a role within virtual reality?
[00:26:09.031] George Dolbier: I'd like to coin a term right here, conversational commerce. You're right on the path. If you think about the demographics that use Uber and are very comfortable with a mobile device, think about if you could dramatically broaden the demographic of people that could use an interface. Even me, someone that is self-described as very technophile, the map interface for a lot of these ride-sharing programs is, to me, really frustrating. I just want to say, you know, I can see the street sign. I want to say I'm at the corner of Olive and Second. Come pick me up. I need to go to my hotel, which is the Seattle Hilton. And for me, that's the ideal interface. And as we all age, we get very good at using verbal communications. and it will be much more natural for games and commerce and all of technology if the interface is actually really conversational. So that echo with what Amazon is doing with their conversational interface is wonderful. Children love it. Any household that gets Alexa and that has children reports the same thing. Kids ask it to tell it jokes. Kids will converse with it. Kids will ask it questions and are more likely to actually ask it questions about, you know, what's the tallest building in New York? You know, how big are the oceans? They're much more likely to ask a very patient, mechanical interface. than potentially even in a socially awkward situation in a classroom or even with their parents. And that's not necessarily a bad thing. It's another interface, it's another way that our lives can become richer, and it's a much more natural way of interacting with technology.
[00:28:02.586] Kent Bye: So I'm really looking forward to being able to hang out with Watson in a VR experience but yet I don't know what type of business model that would require for example for every interaction that I have with Watson or every engagement or question that I ask him is that something that is gonna be charged or is that something that you know you'll foresee licensing that out to different companies say like a high fidelity or are different companies that are then sort of covering that cost and then it's just part of your free service, or how do you see that moving forward with being able to actually engage directly with some of these AI chatbots?
[00:28:34.997] George Dolbier: So this is a really, really interesting question. You know, Facebook and Google have open sourced their AIs. As a public company, we have our fiduciary responsibility. What we have been doing is a couple of things in specific industries like the healthcare industry. We have specific Watsons for the healthcare industry. There's something called the Watson Health Cloud. For industries like manufacturing or advertising or the games industry, we're partnering with companies that understand the industry and understand our technology and will take Watson and tailor it and then provide Watson as a service to that particular industry. So for example, in the medical industry, people that have to sign up for health benefits on an annual basis can use a tool called WellTalk. And WellTalk is a conversational interface that uses Watson to actually understand a company's rather complex health policies. And then you just ask WellTalk really simple questions about what health plan is best for me given that I've got two kids, a wife, and we're of a certain age. And that's really our go-to-market. So for specific industries and specific use cases, we have direct interaction with clients. For industry broad, we work with partners, and so we will license Watson to a partner, and then that partner takes an API and trains up Watson and delivers that to meet a specific industry need. So if you're really interested, you go to the Watson Developer Cloud. Many of our Watson APIs are available free for testing. And then there's like a standard commercial per API call. So if you're familiar with any web service business model, we have that business model as well.
[00:30:34.249] Kent Bye: And it seems like applying Watson chatbots in this interactive AI is going to, in my mind, really change the future of narrative within virtual reality because it's going to be able to kind of dynamically respond. Probably one of the best examples that I've seen so far is Andrew Stern's Facade that was released back in 2005. they kind of had a structure of a narrative that was unfolding and yet you're kind of like engaging and it's kind of extrapolating your intent whether or not you're being friendly or not friendly and either you're supporting the wife or the husband and you know they have it's kind of like an algorithm they have that's giving the structure to the overall 2,000 different dialogue pairs which any given run through you may see 15 to 20 percent of that so It feels like integrating some of this AI technology is going to fundamentally change the interactive components of narrative, but yet it's a little bit of a black box in terms of it's a little bit difficult. It's like peering into somebody's mind to be able to discern what is going to happen. especially if the intelligence is as smart or smarter than a human, then it's just the same problem with being able to really unlock what's going on in someone's human mind. And so to me, I see that there's a lot of this potential, but yet it's this uncontrollable phenomena that, you know, how do you kind of steer it in a way that forms a coherent narrative?
[00:31:54.591] George Dolbier: So I hope that machine learning platforms do transform games. I am a big fan of Telltale games and their narrative-driven games. You're almost asking the question, what is a game? There's narrative and story-driven games and experiences that are based on human literary traditions. Games is a story following cinema and novels. And then there are games that are activities like soccer and golf and chess. And is a game an interactive story or is a game an activity? So for the interactive story part of industry and these experiences, What machine learning platforms should do is be able to allow a much richer narrative, to allow the already very creative teams to be able to have more English majors that are good at writing backstory and illustration and narrative. and not have to map creative literary style to a tree structure, which is a mathematical model, but to actually take that narrative and load it into a software system that understands that narrative. So you're actually focusing on the quality of the narrative and not the machine that is driving the interface so that the machine understands human language and the machine can be told to understand and emulate a personality that you create. And so, again, like modern game design engines like Unity and Unreal allow a creative team to focus more on creating a good game and less on how to create a rasterizing engine We're really hoping that machine learning platforms like Watson will allow creative people, especially in the games industry, to focus more on the depth and the quality of that narrative experience than on so much resource on the mechanics and software engineering that actually drive that interaction.
[00:34:05.406] Kent Bye: Yeah, it was really interesting to hear the process of a character development of hiring a writer to write an entire backstory of an AI and then kind of combine that with some frequently asked questions and then be able to let the AI kind of seamlessly meld the process of owning this personality and backstory that's been constructed by a writer, but then be able to have those dynamic interactive components within the game context.
[00:34:27.637] George Dolbier: It was such an honor to work with Guy and the team that developed The Suspect, and they really were pioneers. Guy had a lot of experience with chatbots and chatbot technology, and this really was, can we throw everything that we can think of You know, video, audio, backstory, freeform text, formatted text, a mind map. Can we throw everything we can think of into describing this personality and allow somebody to actually hold a conversation with this richly created, calling it a chatbot is really over trivializing it, but it really, it's a conversational interface and the conversational interface has a personality.
[00:35:15.355] Kent Bye: Speaking of narrative and AI, just within the last couple of weeks there was a film that was premiered on Ars Technica which input a couple of hundred sci-fi movie scripts and then they used a recurrent neural network to be able to generate a script And then from that script, it was sort of like, you know, sci-fi as a genre has a lot of people that are very confused and they don't know what's going on. And so in certain level, you read it and it's a bit gibberish, but yet they handed it over to actors that were able to kind of imbue this kind of meaningless text in some level with all sort of emotional meaning by their blocking and their way that they're intoning and as well as, you know, how they're kind of acting. And you kind of extrapolate a lot of the meaning of the script that was written by an AI. And so, to me, I see that there's this collaboration that's happening between AI and human imagination and creativity, and that it's able to really augment the things that may be things that we don't want to necessarily be bothered with, but really amplify those components of our imagination and creativity that really shine as humans.
[00:36:20.790] George Dolbier: Absolutely, and that's what we're doing with Watson. It really is to help humans do what they do better. And I hadn't seen that particular Arts Technica, but what you describe is something very similar to something we call Chef Watson. So what we'll do is we'll have Watson read every tweet from the past 24 hours and pick 10 flavors and ingredients that were indicated by tweets, like the flavors of the day. And then Watson doesn't create a recipe. Watson takes those 10 ingredients and then hands them off to a master chef who looks at these 10 ingredients. Sacre bleu, what is this stuff? And then Watson will suggest a particular dish, and then it's up to the culinary expert to take these flavors and create a final dish. And one of the ones I remember was a chocolate truffle burrito with several other ingredients. And when you look at the ingredients, it makes your stomach churn. But then when you look at what the chef took with those ingredients and created, you go, Oh, that sounds really, really good. So it was Watson taking and understanding a massive amount of data and creating a set of suggestions and a context. It's like, OK, here's what everybody is thinking now. And here's the top 10 things. Here, you human, I've read this massive amount of data and come up with this set of choices. And here, you human, go take this and make something really good out of it. And when you make something really good out of it, the odds on that being really delicious are going to be a lot higher than if you trolled Twitter yourself and tried to make it up.
[00:38:11.027] Kent Bye: Yeah, the thing that is really striking to me is I see this shift from the information age into the experiential age. And so we're coming from desktop computers and having an interface where we're saving and storing information. But things like Twitter and Facebook are really kind of like expressions of abstracted and symbolic expressions of our ideas, but yet that's the quantitative information that's being fed into these AIs that are a little bit more into the experiential age, which is things that are not constructed from an algorithmic perspective, but more of a huge set of data that is you're giving an experience to this neural network and it's able to then kind of discern all the meaning from it. So I kind of feel like we're in this transition between information age and moving into a world that's really dominated by these different types of experiences, especially with augmented reality and virtual reality as well as artificial intelligence interactions. But yet it's still fundamentally based upon this older paradigm of information.
[00:39:11.993] George Dolbier: So, I like the way you described moving into an experiential age. Right now, our primary interface with the web and with the rest of humanity are keyboards and monitors and flat screens. and there's a financial barrier that creates a digital divide but then there's also just a fundamental technical barrier that you have to learn to type and you have to learn to run the mouse and you have to learn these user interface paradigms that are part of a 2D screen-based interface and it comes naturally to some and it doesn't come naturally to others but as humans speaking and conversing is something that's natural and I see as conversational experiences, conversational interfaces as a big step to that seamless experiential interface, that seamless experience-driven, economy-driven technology that you're indicating that in 20, 30 years a 2D display will be the lowest common denominator of our interaction with technology, that our primary interface will be talking to things like Alexa.
[00:40:31.695] Kent Bye: And finally, what do you see as kind of the ultimate potential of virtual reality and artificial intelligence and what it might be able to enable?
[00:40:39.182] George Dolbier: So what I think of as the conversational interfaces allow greater access and a more natural access to massive amounts of data and information. So being able to make decisions easier, faster, less expensive, less risky. and then immersive interfaces, whether it be augmented reality or virtual reality, really changing our perspective of what is possible. So I think about being able to get a CAT scan and have a surgeon actually describe to a surgical robot what needs to happen. And, you know, as a surgeon with 20 years experience, even mechanical robots and industrial robots today have dexterity 10 to 100 times better than the best surgeon. And, you know, imagine a surgeon addressing a trauma situation where there are multiple things that have to happen, and the surgeon has the understanding of how to prioritize what has to happen first, and his most effective way of communicating is to say, do this, then do this, then do that. And then to be able to monitor what's going on and say, stop, rearrange this priority. That and having a surgeon be able to be in an AR or VR situation and have that command to somebody that's in Antarctica or a plane crash or somebody that's, you know, he's the world's expert in a particular type of hematoma and he's in Chicago and somebody has a hematoma in Shanghai and he'd be able to react at light speed. resolve a problem until you'd be able to see in 3D in a natural way this patient and to describe actions that need to happen in a natural language and then have them executed by a system that is very accurate and reproducible and dependable and reliable. Awesome.
[00:42:45.650] Kent Bye: Was there anything else that's left unsaid that you'd like to say?
[00:42:48.648] George Dolbier: I want to see AR and VR and machine learning platforms get us off the planet. So I'm one of those that really wants to colonize Mars and live on the moon and invent a real faster than light technology and explore the universe. So that's what I'm really looking for. Besides, I want a new body.
[00:43:07.272] Kent Bye: All right, great. Well, thank you so much. You're welcome. Thank you. It's been a pleasure. So that was George Dalbier. He's the CTO of Interactive Media at IBM. And so there's a number of different takeaways that I had from this interview is that, first of all, it's really interesting to see some of the first examples of how to integrate game development into a machine learning cloud platform like Watson. To me, it made me really want to start to understand these interfaces and to be able to figure out what's going in and what's coming back. In this case, you have to really start to think about how to input different quantified metrics into this API and then take whatever number that is coming back and then be able to put that into your game development somehow. The Progression Curve is a pretty interesting example, and we're really trying to tune the game to make it the most fun for people to play. So I think it'll be interesting to see whether or not this type of artificially intelligent mediated game design is going to take off and be able to really be universally fun for everybody to play, even though the AI algorithm may be giving someone a completely different experience than someone else who is playing the game. Also, game designers and coders are not going to be going away. I think there's a lot of room for human creativity that artificially intelligent machine, I think has a long time before it really figures out what's fun and interesting within game design. So I think game developers are pretty safe here. You know, doing this interview really set me off onto this path of really trying to understand all the different types of machine learning architectures that are out there. And at this moment in time, those architectures are really exploding what seems to be exponential relative to any other time in history. Within the last couple of years, there's been the open sourcing of some of the tools at Facebook AI Research as well as Google. Facebook released their Torch modules back on January 15th, 2015, and Google released TensorFlow back on November 9th of 2015. And then at one of the White House AI summits, I had a chance to talk to a NVIDIA representative who talked about the importance of GPUs and parallel processing in terms of being able to train these neural networks. So that's a big factor as well as the availability of cloud computing and cheap compute resources We just have this explosion of tools that are available and papers that are posted on archive so just a lot of really intense innovation that's happening in the artificial intelligence field right now and I I actually just started a new Twitter account at Voices of AI, and you can check that out if you want to hear more of my process of learning more about this AI and machine learning field. Deep learning and all these neural nets I think are going to actually make a big impact in terms of virtual reality. There's going to be a lot of different cloud resources that are available from Google as well as IBM and Watson, but also if you wanted to start to learn more about it and try to train your own networks. I think one big insight that I had in talking to an artificial intelligence engineer at the Rothenberg Ventures Field Day is that he said, you know, part of the reason why Facebook and Google feel free to kind of open source all these tools is because it requires a lot of processing in order to actually train up these neural networks. And so it's not just something that one individual can be able to create on their own. However, at the same time, there's all sorts of massive amounts of data sets that machine learning researchers have available. And there's certain huge repositories of images and examples of handwriting samples for computer vision training, but also just the internet. And Project Gutenberg, which has scanned in millions of public domain books, as well as just individual ebooks. And I think that we'll start to see a lot more of specific AI neural nets that are trained to do very specific use cases based upon what input information that has been fed into it. I think a big challenge moving forward with AI is going to be the issue of bias and being able to have a diverse enough representation of the input data, whether that's from a good mix of gender and race and various different political beliefs, but also a mix of quantitative and qualitative data. You know, I think there's actually a pretty strong bias towards quantified data. And the qualitative aspect is something that I think is going to start to be difficult to try to feed into these machine learning programs. So yeah, check out the Voices of AI on Twitter. And who knows? I may kick up another podcast if things keep going well here at Voices of VR. And so just to kind of wrap things up, I just wanted to thank you for listening. I just over the weekend happened to check my iTunes and had a lot of really great reviews and comments from people. So if you do enjoy the podcast and would like to tell others about it, one good way is to go leave a public review on iTunes. It's, I think it's going to be kind of fun to look back as almost like a historical document of who was listening to the podcast way back when at the beginning. So. Go make your mark and leave a review for others to help discover what I'm doing here at the Voices of VR podcast. And if you also would like to financially support the podcast, then please do consider becoming a contributor at patreon.com slash Voices of VR.
