Indira Thouvenin teaches Virtual Reality at the University of Technology Compiegne. Her research focuses on informed interaction in virtual environments, and how knowledge representation and sensory motor knowledge are linked within these environments and can be used for training, designing, and collaboration within virtual worlds.
Here’s a bit more context about informed interaction:
Informed interaction is a concept connecting 3D interaction in virtual environment and knowledge representation. An ‘enactive’ approach to cognition or ‘enaction’ inspires our work with the concept of ‘action guided by perception’. This notion originates from the reflections of Francisco Varela, a neurobiologist who, following his work with Maturana in Chile, sought an alternative to computationalism and connectionism as approaches to understanding cognition. Varela’s attempts to introduce concepts from the biological cognitive sciences and his research in neuroscience lead to the concept of ‘embodied cognition’.
Our objective is to understand how human experience can be at once reinforced,capitalized and re-exploited in a virtual environment. We are particularly interested in activities centered on training, design and collaboration for which on one hand knowledge engineering provides a pertinent level of abstraction and on the other hand virtual reality offers modes of interaction with sensory feedback which are beneficial for the users involved (trainers, students, designers, remote workers, etc.). In this vast region of investigation, we have chosen to focus on a single area: the relationship between interaction and knowledge within informed interaction.
In this interview, Indira talks about a specific application of VR training by talking about how VR is used to train pilots of barges. VR provides a safe learning environment as well as the opportunity to learn through failure.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:12.093] Indira Thouvenin: I'm Indira Tuvna. I'm working in the field of informed virtual environment. I'm professor at the University of Technology Compiègne, where I teach to computer science and mechanical systems students how to design a virtual reality system, how to design an interaction, and how to use virtual reality interfaces. My research is focused on new models about knowledge-based interaction, like, for example, evolving environments for training, for communicating, for collaborating. So the situation is, for example, a user in a virtual environment, if he's a beginner or if he's an expert, the system will adapt automatically to him. and will give him metaphors of visualization and interaction but also multimodal guides in order to let him experience and do, for example, errors and correct by himself his errors.
[00:01:22.878] Kent Bye: Great, and so what's some of the research that you've done and some of the results that you found in terms of really optimizing a virtual reality environment for training or learning?
[00:01:32.817] Indira Thouvenin: We have been working for example on boat simulator to train pilots to understand what is piloting on a river because it is very difficult when you have a long boat, a barge, And the barge is difficult to pilot. Why? First, because you cannot stop it quickly. It takes a long time. You have to anticipate. And how to anticipate? Nowadays, pilots are learning on real rivers and on real barges, which is very expensive and also difficult and dangerous. So we give them more and more complex situations. and we give them also some virtual guides. So the core of the project was decision module taking into account the uncertainty because uncertainty means that when you are piloting in fact You have to interpret the data, you have to be able to understand fully what will be the next step in half an hour, not immediately. So the system is giving, for example, a virtual road on the river and explicitly gives what will be the collision maybe that you will have if you take the bad direction. So this is an anticipation for the user and he can understand how to orientate his boat. This is one of our results. So our decision module was based on belief functions and these approaches are knowledge-based approaches.
[00:03:25.915] Kent Bye: Do you think it's important in a learning context to be able to fail and to learn from the failures? Obviously, when you're piloting a barge down a river, it's not feasible to learn from your mistakes. But there is a process of learning from your mistakes in order to learn. And so in virtual reality, you can do that a little bit more. And so I'm curious of that trade-off between having guides to kind of show them the right way versus allowing them to experiment and fail and then learn from those failures.
[00:03:51.934] Indira Thouvenin: Yes, it's a very, very important question. In fact, it's a subtle question, because when you do the transposition between the real and the virtual, there are a lot of possibilities, but you can do very bad environments. An example of a bad environment would be to do a realistic environment with very complex objects, complex processes, complex situations and you can learn in fact with very simple things. In fact at school there is a reduction of the real situation and you guide the student with small steps. So you are doing first this, then this, then this. So that is a temptation in the virtual environment to do very complex things. And it would be a very dangerous way to teach. So what we have learned, we have learned from the scientific experiments, it's that the transposition can give a believable environment which means that the user engages himself in a virtual environment, but the situation is real. He is perceiving with his body, with his complete perception, sensorial perception, and the emotions are real, even if the life is artificial if the environment is artificial. So the first step is to be able to design believable environments, giving a few elements to guide the learner. Then what is more important and very difficult in fact to model is to represent the uncertainty and the maybe not very clear situations. Sometimes the information that you get in the real world are not so clear. And if you give very simple and basic situations, in fact, you do not learn. So this point of uncertainty is difficult to model. And then, yes, you're right. It's important to fail. It's important to learn from failures. But you have also to, again, give small failures and possibilities to understand why it was a failure and why You did not take the right decision or the right gesture. For example, we are working on handwriting system with augmented and mixed reality. And handwriting for adults, not for children, means that you understand the result of your gesture with the 2D trace. But what is happening in air? For example, for Chinese calligraphy, you need to have a very good posture, you need to be very concentrated, to have an attention. And how can we give all these very subtle things in the virtual environment and give the feedback to the user that he has the good attention, the good gesture, Maybe giving a transposition of the handwriting in 3D, which is maybe too complex for the user. So the 3D trace in the air could be a new way to learn handwriting.
[00:07:32.744] Kent Bye: You had mentioned emotion, and I've heard from a couple of other people in terms of virtual reality simulation trainings, especially in terms of safety training, is that they try to create a sense of real danger or fear, and that fear can actually help improve memory. In the case of handwriting, I don't imagine that fear would be necessarily helpful in learning the handwriting. So I guess I'm curious about using emotions and generating fear, and when is it appropriate to use fear, and are there other emotions that are important in terms of creating learning experiences?
[00:08:05.809] Indira Thouvenin: Fear is maybe an emotion that you can feel when, for example, you are in a situation where there is a fire or there are some terrorists coming and you are in a very, very big stress. But in fact, the basis in teaching is to be confident. And if you are learning handwriting, you have, in fact, you have the impression that you are bad, you are not doing the right gesture. So to be confident, to liberate your gesture, you need to have some support and to understand that what you are doing is okay. So how can we give some feedbacks about, for example, the hesitation, the intention, the attention? It means that you know the context, so you have a modelization of the activity, of the cognitive activity, but also the body activity. But we need also the context and the level of the user. Is he beginning to draw something or to design a new letter? And how can we give him this confidence? Maybe using a game or maybe giving to the user more confidence in himself because we give him different levels, easy things to do and maybe some feedbacks like sound or vibration or maybe funny things.
[00:09:41.497] Kent Bye: Yeah, and you'd mentioned both the VR interactions and the multi-modal components of learning, and I'm curious how you tie both of those into learning experiences in terms of having limbs tracked or what type of things that you're doing in a VR environment, but also what you mean by the multi-modal element.
[00:10:02.458] Indira Thouvenin: Multimodal can be the classical feedbacks like haptics, tactile, sound, etc. But the multimodality means also that you can augment the gesture. For example, you are doing a drawing on a tablet, but some senses are catching your gesture and interpreting it. So you can draw, I don't know, some objects while doing your gesture, or you can draw and have the music, and have the level of the sounds, or you could have the avatar of your hand and your arm. Or another feedback would be to have the third person, yourself, being drawing.
[00:10:50.683] Kent Bye: And yeah, I'm curious about your VR lab at your university in terms of if it's known for training or if there's any kind of specialty that the laboratory at your university has.
[00:11:03.198] Indira Thouvenin: We are a lab doing a lot of things and our team is more specialized in knowledge and interaction. So we are doing for example autonomous virtual characters for the question of the risk management. We are also modeling new environments for giving adapted assistance while driving, for example augmented reality for the intelligent vehicle. or we are also trying to work for manufacturing virtual environment like interactive technology for production. And we have a lot of models able to connect virtual reality, machine learning, artificial intelligence, knowledge representation, and also we have the support of cognitive science lab.
[00:12:02.730] Kent Bye: So, yeah, a lot of things around learning, it sounds like. And when you mention knowledge, are there insights into knowledge that you've had in terms of like, what is the best way to capture and distribute knowledge? Or, you know, what is it about knowledge and this sort of learning that you're really focusing in on?
[00:12:21.892] Indira Thouvenin: Knowledge for us means knowledge representation like ontologies, like agents, multi-agent systems, like what we can see in artificial intelligence about reasoning, rules, things like that. But it can be also for our colleagues the fact that when you take an object you know what it is. So you have the concept of inaction in learning, which means that when you act in the virtual world, if the virtual world is answering to you, you perceive the virtual world. For example, if I open a door in the virtual environment, I will understand that the object, the virtual object, is a door because it is possible to open. So you construct the knowledge from the objects of the virtual environment and from the interaction, which is the concept of action perception. and we focus on the concept of inaction. But you need also to have semantics in the virtual environment because you do not have to manage only polygons or 3D objects with the geometry. You need this semantic information.
[00:13:46.507] Kent Bye: And has there been any of the students or papers or talks here from your lab that have been presenting anything here at the IEEE VR? Maybe you could tell me a bit about what they were talking about.
[00:13:56.800] Indira Thouvenin: In fact, my PhD student presented at the doctoral consortium his work on the handwriting system.
[00:14:06.071] Kent Bye: Okay. And are you looking at virtual humans at all in terms of like learning environments and the importance of having other representations of social situations with virtual avatars and what you've kind of found from that?
[00:14:19.389] Indira Thouvenin: Yes, my colleague Domiti Lourdeau is working on this kind of situation. So we work together. I'm more involved in the informed interaction and she's more involved in the autonomy for virtual characters. So for example you have a lot of people and there is a problem of terrorism and one of her project was about how the panic is propagating. So there is a very original concept where you can be contaminated by the panic or you can be isolated from the panic and the behaviors are changing with artificial intelligence models.
[00:15:02.993] Kent Bye: Ah, interesting. It sounds like you're working very closely with cognitive psychologists and other psychologists. I'm curious how that plays into your research in terms of working with the psychologists and whether or not you're able to get more insights into psychology or if you're more taking insights from psychology and applying it into virtual reality.
[00:15:22.803] Indira Thouvenin: We use the results of scientific results coming from psychology and cognitive sciences and we try to put it in the computer.
[00:15:35.863] Kent Bye: And finally, what do you see as the ultimate potential for what virtual reality may be able to enable?
[00:15:43.210] Indira Thouvenin: Maybe the best would be to get transparent interfaces, but I'm not very familiar with the next generation interfaces like having in your body implemented systems. I think that We should have a mixed reality with systems able to understand the user but giving to the user the opportunity to evolve. I think that our brain is plastic and we can adapt very quickly to different technologies and the technology would be like our keynote was saying on the first day of the conference. Mark Billinghurst, he was talking about empathic interfaces and maybe the concept of evolution, adaptation in mixed reality could be a natural way to go further.
[00:16:40.526] Kent Bye: Yeah, I definitely think that from this conference, the neuroplasticity implications of using VR, to me, is like the killer app. But I don't know if you've seen any other sort of either implementing neuroplasticity insights or the implications of using VR to kind of rewire our brains, to me, I think is probably some of the most exciting things.
[00:17:01.009] Indira Thouvenin: Well, you can use the plasticity with external representations and let the human evolve by himself, but it could be possible, I think, in the future to use interfaces like, for example, brain implants or things like retina implants and things like that, with the problems that you can imagine, like juridic problems or health problems or technological, classical, basic problems. and maybe there will be a split between these two directions, the very, very instrumented, like the post-human and the trans-human, and also the natural human evolving and having a lot of levels of evolution by working, collaborating, playing, and being connected with others.
[00:18:07.345] Kent Bye: Yeah, I definitely think that there is a possibility of that bifurcation between the transhumanists that are going all in with putting all this technology inside of their body and the sort of the more maybe luddite or more natural humans that are deciding not to. And yeah, there could be an evolution and difference of, you know, split into the species, you know, evolutionary. But it could also, I personally get afraid of, you know, the long-term implications of embedding technology in your body. Yeah, there just seems to be a lot of moral and sociological implications and do you think there will be kind of like this bifurcation split between those two?
[00:18:43.427] Indira Thouvenin: Yes, and I think that it is our responsibility to be able to anticipate this and to give some interesting perspectives and consequences of the splitting.
[00:18:55.758] Kent Bye: Great, well thank you so much. Yes, thank you so much. And thank you for listening! If you'd like to support The Voices of VR Podcast, then please consider becoming a patron at patreon.com slash voicesofvr.
