Architects Anna Mill & Luke Jones spent eight years creating a graphic novel called Square Eyes that explores the speculative design of augmented reality technologies and interfaces. They see this project as an in-depth paper architecture exploration of a sci-fi future where the digital and real worlds have been seamlessly combined, which allowed them to explore the dynamics of the potential future through the lens of a narrative critical dystopia.
I talked with Mill and Jones at the end of the Symposium on the Architecture for the Immersive Internet where we talked about their inspiration for Square Eyes, why spatial computing interfaces caught their imagination as architects, and how they used the conceit of depicting physical reality in black and white and digital reality in saturated colors. We also talk about some of the other implications of immersive technologies onto the practice of architecture, what type of insights they came away with about the fundamental affordances of AR after their eight-year exploration, and some more information about Jones’ podcast About Buildings & Cities that he produces with George Gingell.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
Here’s Anna Mill’s presentation at the Architecture for the Immersive Internet
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye and welcome to the Voices of VR Podcast. So continuing on my series on the future of immersive architecture, there is a symposium that happened at the Architectural Association on March 1st, 2019, where brought together all sorts of different architects and different backgrounds and At the end of that day, I had a chance to catch up with a couple of presenters that were presenting about their graphic novel that's called Square Eyes. And so for eight years, going all the way back to 2010, they started to think about augmented reality and the future of these spatial interfaces. And they wanted to explore, through this form of a graphic novel, some of the potentials of what the implications of augmented reality might be and how it changes our relationship to the world around us. Who's in control? What do we see? What do we not see? And so they created this whole fictional story called Square Eyes. It's a graphic novel. It's 256 pages. Thinking about the future in the very early nascent days of augmented reality and as they were working on this project together, Anna Mills and Luke Jones, they were basically seeing the evolution of these immersive augmented reality technologies. And as they were thinking about the speculative future, you actually have the emergence of these virtual and augmented reality technologies. So they were trying to come up with these different interfaces, different implications, but taking a very architectural approach, meaning that they were doing all this world building and designing of both the spaces, but also these social dynamics and interactions, and using the method of paper architecture to be able to explore these different concepts and ideas, not in the sense that it's being built, but to be able to explore the more conceptual aspects of the future of these immersive technologies. So I had a chance to talk to both Anna Mill and Luke Jones about their process of working on this project and some of the takeaways they have about the affordances of augmented reality through the form of speculative design and paper architecture. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Anna and Luke happened on Friday, March 1st, 2019 at the Architectural Association in London, England. So with that, let's go ahead and dive right in.
[00:02:26.843] Anna Mill: So my name is Anna Mill. I'm an architecturally trained designer and artist. And in the space of VR and AR, I've been working for eight years on a graphic novel, which is about an augmented reality saturated city. And it was just published in October. It's 256 pages long. So it's been a long investigation of this particular situation.
[00:02:52.459] Luke Jones: My name's Luke Jones, I'm an architect and I'm Anna's co-author. I broadly did the words and the story side a bit more and she is responsible for the artwork and then also we collaborated on most of the story. And I guess that my involvement really is in the dimension of the book which is kind of the speculative design of AR as a technology which really for us began at a point when it really was very, very speculative. So the book took eight years to complete and in its gestation it even started a little bit before that. So it really was starting to take shape in like 2009, 2010 as a set of ideas when AR had appeared as this very intriguing set of kind of demos and things which you could do if you had an iPhone or you know with your computer webcam or with these sorts of things but which otherwise was a purely speculative medium and so because of the length of time that the book took we've kind of gone through this cyclical paranoia of whether Technology would catch up with and make fools of all of our speculations in the end that never quite happened And actually I think that there's still some really quite interesting things which we've managed to get in there Which perhaps still have a sort of relevance to the future Yeah, I'm wondering if you could talk a bit more about the catalyst that made you from an architectural lens Start to look at something like augmented reality
[00:04:19.379] Anna Mill: Yes, so I think at first it was that we were in search of a story I think. So how it first came about was that we did this four page short comic book which was for a competition and we were really looking for just any topic to base it on. It coincided with the time when we were living with a friend of ours who was working in interactive design and spatial interactive design was the specialism he was working in. So he had been very enthusiastic about these new possibilities of AR and he had shown us some demos and things like that. And I think we realized that there was an interesting way in which augmented reality itself could feed into the fictional layers of a narrative in a way that seemed really interesting. And because we were obviously already considering architectural space, what happens when there are these kind of imaginary layers that you can put on top of the physical?
[00:05:20.010] Luke Jones: The kind of intuition which I remember starting it all off was this idea that there would be this very interesting, productive and kind of, in a way, a little bit crazy mapping between the city as a really existing urban space and the internet as this amazing playground which we're both very into as finding interesting websites and exploring. but that somehow between those, there was the potential for what felt like was going to be this totally surreal condition and experience. And I think that that was the initial, you know, we started off from these sorts of speculations, which were super obvious at the time. Like, you know, back then you still had like pop-up ads and these kinds of things, like what was going to be the equivalent of that when it became a fully spatialized kind of three-dimensional. thing, and obviously that's sort of a comically antiquated way of thinking about these things now, but the narrative potential of thinking about the technology in that kind of way is what started us off, and then the more we thought about it, the richer and more interesting it became. A key dimension which really developed is that we were thinking about the tension between the glittering technological virtual world and the material world, which is mapped on top of, and all of the potential for those things to mismatch and to have an imperfect relationship, but one which is narratively interesting. And to talk a little bit about the story, one of the main arcs within the interaction of the narrative and the setting is that that is to some extent a bit of a conspiracy in the sense that the city as it appears is not quite the way it really is and the ability to create a sort of illusion is sort of explored as a way of creating a kind of a conspiratorial distortion of the city at a societal level. There are lots of other things going on as well but that's kind of one of them.
[00:07:20.303] Kent Bye: Well, one of the mechanics of communication that you're using is to depict the so-called real world as it is in black and white and to use color as a form of telling the audience that this is being augmented in different ways. And so maybe you could talk a bit about that, how you were using that as a construct to explore this tension between the digital and the real and how those don't always align.
[00:07:44.998] Anna Mill: Yes, so I think there's two different ways in which it answered this design challenge for us. The first is in which the black and white seemed to be almost a better way to celebrate the kind of physical materiality and the detailed textures and the crumbling of the physical world and the physical city. And we needed some sort of graphical device in order to let the reader know what was virtual and what was physical because the way that I was representing it in the book was all in kind of traditional hand drawing so even the virtual is drawn with a pencil outline so we needed some way to designate what was virtual and what was physical when actually the method of representation was more or less exactly the same. The other thing, I guess, was the choice to make the digital layers full color. In some way, trying to have this great vividness about the digital layers that are on top of the grayscale physical layers was also, in a way, trying to speak to the way in which, because it's a comic book, It's a series of still images, and in this AR-saturated city, we can imagine that everything is sort of shifting richness, which was hard to depict in this sort of series of still images. So making it chaotically colorful was a way to try and speak a little bit to some of the movement that couldn't be depicted in the comic book itself.
[00:09:22.934] Kent Bye: Yeah, and I'm curious to hear a little bit more about the process of creating this story in the midst of the unfolding of this modern resurgence of both virtual and augmented reality, both with Ready Player One that released a couple of years after you started your project in 2011, and then Oculus Rift as you're writing this story and seeing all the zeitgeist exploding, if you are also like involved with experiencing and having an embodied experiences of this technology as you're writing the story and how that may have helped shape or inform or just to be in this cultural zeitgeist while you were doing this deep dive in this topic in the midst of it emerging into a whole industry.
[00:10:16.957] Luke Jones: I mean, one of the decisions that we made early on is that we weren't actually going to show the technology by which any of this stuff is perceived at all. That we thought that one of the things which would really clearly date it would be if it was a glasses or it was a lens or whatever. We would, you know, we thought that there were a sufficient number of plausible ways in which the interlacing of the visual field could happen, that we would just not show how it was done at all, and we would just present this as being a fact of everyday life. And I think that that's one of the things which actually I think just works quite well, and which back at the start when we were proposing the project, for a certain number of people it was quite hard to get their head around, but which now, viewing it, people I think have really no trouble at all with understanding that that's exactly what's being shown. Did you want to...?
[00:11:02.044] Anna Mill: Yeah, so after we'd done the short four-page story, in order to get the publishing deal, we did what ended up being a 100-page proposal, which was where we had to explain the world and the story and the characters. And actually, the most involved section of that was about the world and the technology. And because it was so early on in the emergence of the technology, we had to go into this sort of minute detail about what it was and what it meant. And so there are in fact scenes at the beginning of the book which were intended originally to be a way in which you could start to immerse, slowly immerse the reader in what this technology is, what the affordances of it are, before they get right into the full submergence of the city itself. So there's a scene at the beginning where someone is returning to the city and they're arriving slowly gradually coming into the city on a train and there's a sort of trying to organize it so that it's a casual conversation it was almost like what you see now in VR where there's the sort of teaching experience sometimes at the beginning of the actual creative experience that someone wants you to understand so we felt like we had to do that but luckily Because it took so long for us to make the book, the technology has now become much more familiar to people. So we didn't have to spend so long on this kind of initial chapter of this introduction to what it is, what it means for people who were completely unfamiliar with this space and the technology.
[00:12:35.211] Luke Jones: I mean at the same time one of the I guess you could think about the sort of speculative design things which we really took a lot of pleasure about doing in the book was trying to invent a paradigm of like interface and a way in which people I mean one of the things we had to do is we had to invent what we thought was a sort of plausible looking way that people might code in the future or like something that would look completely different from the way they do it now and the simple idea was that it's become more abstract it's kind of gone further up into the sort of layers above where it is now. And those kinds of things, it was helpful not to look too much at existing technology for those kinds of things, but actually to look for design references, which in these cases came from natural drawings of plants or came from clockwork mechanisms. And we're much more about trying to think about the types of design models and kind of analogies which might be adopted several generations in the future. rather than the things which are working for the technological models of the present.
[00:13:38.297] Kent Bye: We just got done with the day-long symposium on the immersive architecture of the internet and we had like a two-hour discussion and debate about some of these topics and one of the topics that came up was paper architects and the role of speculative design and You had actually mentioned that you consider this project as a form of paper architecture where you really immersed yourself into this world and really trying to explore these different concepts and ideas. And so I'm wondering if you could connect the dots a little bit to the lineage of paper architects and how you see what you were doing in this graphic novel as a form of paper architecture.
[00:14:14.783] Anna Mill: Yes, so I mean in the tradition of paper architecture, I mean Luke might be able to define it a little bit better than me, but my conception of it is that it's something that is not necessarily intended to be really physically built. So it's got that freedom of creative exploration without being limited to the practicalities of what can really happen. And those kind of practical limitations might be physics or society or all kinds of ways in which the architect's perfect dream is bounded by having to build something in real physical environments. So for me this is a sort of long and sequential version of paper architecture in that it's covering a whole city and a city with different situations in different areas but then covering a sort of different shifting cultures within that city as well. And all of these things, I think, fit into the tradition of paper architecture, but this just happens to be a project that is on 256 sheets of paper and was explored over a much longer time. But the thing that I found interesting in this discussion today was that there was some debate about whether paper architecture could be a valid experience. There was a discussion about whether a paper architecture project is completely different to a virtual architecture. And for me, I don't think there's necessarily a line. I think you can, even if you're working on traditional paper and with a pencil, you can develop enough richness into it that you're forming these spaces literally within your own brain even if you're not seeing it actually with your eyes in the way that you do in virtually imagined architecture.
[00:16:04.599] Luke Jones: So paper architecture, and I think this is particularly true more recently, I think maybe going back into the 18th century or whatever, this would be less the case, but there is also, it gives you the capacity to stand a little bit of a critical distance from kind of authorship or the question of whether this is actually a good idea. So the paper architecture can take on a little bit of the character of what you might call a critical dystopia or a critical utopia or whatever. So it's a model which presents itself as being produced or taking advantage of existing technologies or conditions or the sort of affordances of them, but which perhaps has a somewhat ironised relationship to the material or to the sort of social conditions. And so the narrative form of a comic book means that you can explore all sorts of different things, many of which are actually, they're not things that you would say are good for them to happen. But some of them have to be. I think things which are actually for use, like when we had to design things which sort of look like interfaces. I don't think that those things can really, you can't really do that without thinking that actually this would be a nice and pleasurable and kind of enjoyably sort of frictionless way to interact with large amounts of data and lots of different things going on. But other aspects like the city itself can present themselves much more as a bit ambiguous in that sense.
[00:17:33.199] Kent Bye: It reminds me of, in terms of storytelling analog, of something like Dungeons & Dragons where you're using the theory of the mind to, in the moment, in the conversation, be able to explore concepts or ideas but not be limited by whether or not it's pragmatic or practical at all because it's just you're speaking and you're imagining it. With that freedom gives you this ability to explore all sorts of interesting concepts with this world that you've created. You're now all of a sudden able to explore different design potentials and the outcomes of those. And so what were some of those insights that you got into the fundamental nature of augmented reality as a medium as you're looking at it through this lens of speculative design?
[00:18:13.585] Anna Mill: I think it wasn't necessarily a particular insights into what the potential could be. I mean, we didn't explore those insights. I feel like I have some thoughts about it. And there's a part in the book where you get a tiny glimpse of what this augmented reality future meant to our main character before she underwent this particular crisis that has occurred at the beginning of the book. So just to give a little bit of background, the main character is someone who had been living a life of a sort of tech entrepreneur of some sort and had a seamlessly AR integrated life and was living this kind of was developing a technology that meant that she could create instantly and seamlessly and it was this kind of beautiful process and I think that's one of the things that feels exciting to us about this is the the way in which the time that it takes between conceiving of an idea and kind of manifesting something that you can show to some other people or something that's a sort of prototype, it's just shrinking and shrinking and shrinking. And that's a thing that in VR and AR feels really exciting. But the other thing that I accidentally found interesting about AR was this, what I was talking about a little bit today at this symposium was this idea of, particularly in AR, I can foresee this way in which there could be accidental beauty in the way that the AR and the physical city interact together with kind of other complex systems like the weather and the population and it's all of these different systems and agents kind of human or non-human that are interacting together which can create this unpredictable complexity and these moments of accidental beauty which was what I found particularly exciting about this
[00:20:05.513] Luke Jones: One of the things which I think, narratively, the sort of thing which we're much more investigating is what is fear and what is desire in this medium? And in the context of the story, you know, fear is this conspiratorial kind of paranoid fear that you're being manipulated and that your reality is not being presented to you in an honest way and that there are all sorts of things going on in which which you aren't able to be aware of. And then desire, I think, is this thing which is presented as the main character's invention, which is this seamless way of your thoughts. You can kind of think things and instantly turn them into objects in the sort of objective consensus reality of AR, basically, is the idea, and that she's invented this extremely effective way of doing it. And that seems to be the kind of dream of AR is the dream of creativity and the dream of this totally fluid and natural creativity. In terms of what's possible to explore in the form that we've been working in, It's difficult actually, you know, AR has a sort of solipsistic reality and it has a consensus reality. There's the reality which is just the reality of individual people and which can be arbitrarily different. And there's the consensus reality which is real space full of virtual objects which we can all see at the same time. And the latter is much easier to explore in this kind of form than the former, which is obviously really, really difficult because you're having to show the same, a different thing at the same time. It's an insoluble, problem in those kind of narrative terms. We've tried to do it a little bit, but I think that that is the great kind of unexplored dimension of the technology for us actually in this particular artifact.
[00:21:45.595] Kent Bye: Yeah, because I mean, we already have filter bubbles that have been created to have social networks and algorithms that are creating a bubble of a consensus reality that's shared amongst those people, but to then go down to individual people's perception of what they're seeing and how that matches up to what other people are saying. I mean, I think that the philosophy of postmodernism and the pluralistic approach of allowing each individual to have the truth of their own perspective is the natural end game of that is to have this split where the literal worlds that are being created around this could be reinforcing these individualistic perspectives. So, yeah, it sounds like you're thinking about that, but it's difficult, I guess, to explore that in a narrative. But was there any ways to explore these deeper questions of what is truth, what is reality in the context of this story?
[00:22:31.981] Luke Jones: I think that Yeah, I mean, that's a big one, isn't it? In the end, the characters actually, the kind of solution is that they sort of just have to find a way of getting on with their lives. These enormous questions are not satisfactorily answered, are they? They're answered partly by this sort of, well, maybe I'll backtrack a bit. There's a critical moment towards the end of the book in which the characters break through this kind of wall of illusion. which is represented traveling through the spine of the book. They break out of the frame and travel through the white space of the spine and that sort of represents this otherwise unshowable moment of kind of moving through, out of the reality which they've been in. And then they enter this world which is the kind of hidden part of the city, which is the city in a state of savage and dramatic recycling and reconstruction in which it's being ground up by all of these big machines and it's being reconstructed to the needs and requirements of the new order. Yeah, so there is a moment of revelation which takes place at that point and which is glimpsed in this moment of crisis. But they can't live there, people can't live in that state. Actually what really the resolution is much more about finding a way of living with the technology and that is much more ambiguous and the ending of the story is deliberately much more kind of ambiguous and potentially up to the reader to resolve a little bit what they think has really happened.
[00:24:05.523] Kent Bye: Yeah, and I'm curious if you could expand a little bit on coming here to the Architectural Association and to be speaking about this work to a room full of fellow architects who are also thinking about immersive technologies. Like, what were some of the major points that you were trying to get across to them today?
[00:24:23.769] Anna Mill: I think I actually I was a little bit worried almost about coming because it's a room full of architects and I feel like I know sort of a certain amount about architecture having trained in architecture and I'm teaching architecture now but architects who are specifically working with questions about the virtual realm which is something that I've been thinking about continuously for eight years but actually have only kind of limited experience of because we don't actually have any equipment so we've mostly it's been entirely this kind of further distance speculation that we've been doing So I think in some ways I was quite glad to find out that actually a lot of the things that we were thinking about were some of the same things that people who are really trying to create in the virtual environment are thinking about as well. So even though we're not practically working in this environment, it seems like the considerations are really similar. So I suppose what the things that I particularly wanted to present as my thoughts on the subject, I think I'd sort of come down on the side of the conclusion that it feels like there are all of these ethical and psychological questions that we need to be considering as we're thinking about designing these new spaces, but that it's more something that we sort of need to keep in the back of our minds and be aware of rather than trying to kind of lay down stylistic or behavioral guidelines. And I think I really feel like because it's a whole new exciting realm to design in, I feel really like I don't want to restrict it by trying to fearmonger too much about what could happen. And so even though when people read the book, they might think that it is quite dystopian in some ways. So we tried to strike a balance between the creative and exciting possibilities and the possible problems. But again, they're all the sort of same things that everyone's kind of distilling around these ideas of who's in control and are you manipulating the space or are you being manipulated by it? And I think it sort of feels like these are the sort of same questions that are emerging over and over again. But I definitely feel like these are things that as long as lots of people are able to work on these questions and think about them, it's better for us all to just be moving forward with an awareness of what the problems might be rather than trying to kind of lay down rules about what we should do.
[00:27:09.572] Luke Jones: Yeah, I mean, I don't think we would have done the book if we weren't fundamentally, like, really excited by the technology. You know, I think even in this particular scenario, which is a gloomy one in some ways, it wouldn't have been possible to put the mental energy in to realise it if we weren't fundamentally very excited by the potential of these kind of VR, of what these things could be and how they could transform our sort of experience.
[00:27:38.729] Kent Bye: Well, I guess what I find interesting is that both virtual and augmented reality are, in some sense, like this melting pot, where it's requiring the experiences from many different domains and disciplines. And everybody has something to contribute to this medium. And so I'm curious if you could describe to me what is an architect, how you define what an architect actually is, and what you think architects have to contribute to either virtual or augmented reality as a medium.
[00:28:10.281] Luke Jones: I think that obviously there's space, isn't there? And architects have a long history of thinking about these questions of not only space but the way in which people move through it. We had an argument during the day about architects who are doing narrative-based works, as a lot of things in virtual reality are, shouldn't they try and learn a little bit more from actual writers and from screenwriters? And I do sort of agree with that, but I think that there is a kind of moment in between, which is this moment of sort of staging, the movement through space, which is kind of spatial, but which is also about maintaining a kind of meaningful direction and a certain kind of orientation, a certain sort of awareness. which I think is definitely something which you can contribute to the meaning. I think also there is an attentiveness to detail and there is an understanding of this kind of connection between the affective seeming of things and the imagined like material consequences which might have caused them to arise. I think as an architect you're quite used to looking at materials and seeing them as a result of quite a long process and that there's an interesting imaginative thing that you can bring to virtual environments which is endowing all of these materials with quite a long story of how they came to be the thing that they are rather than just being. like the thing which they ended up being, which I think is enriching and I think, yeah, also allows spaces to be more, not just more detailed, but kind of more like thought out in all of their relations.
[00:29:50.478] Anna Mill: Yeah, I wonder, I mean maybe you're already touching on this, but the way in which architects are manipulating space and materials and light and shadow are all ways in which they are controlling people's experience of that environment and sometimes it's trying to control an atmosphere and sometimes it's trying to control a behavior or sometimes it's trying to direct people through a space. So these are all things that feel like they have a direct way in which that useful knowledge can be translated into the virtual as well.
[00:30:25.930] Luke Jones: Yeah, I mean, there's a sense in which buildings are this example of these multi-directional narratives, aren't they? As the architect, you have the idea that when you do walk up the stairs, you see through this particular window the view of the tree or whatever. That's a kind of point in a narrative, but it's not one which has an absolutely sequential position within a single arc. It's something which is contingent on moving through a space in a particular way. And I think that there is... those sorts of moments, as well as just a kind of general attentiveness to the quality of experience and the way in which it's built up of all of these different cues to do with space, views, light, sense in which the space in which you're in like affects your bodily condition, which is a thing which if you imagine like the difference between being in a normal room and being in like a kind of gothic cathedral or whatever, is the way in which having this enormous space above you does something absolutely physical to your sense of yourself as a body, like kind of transforms and draws you up. And yeah, all of those things I think transfer over in a very meaningful way.
[00:31:32.158] Kent Bye: Well, in talking to other architects as well as game designers, the thing that I took away is that there's not necessarily a singular unified theory as to how you build a space or have a series of interactions and how that's going to get translated into a human experience. and that in some ways you have to cultivate your own design intuition just by building it and seeing how you feel and then seeing how other people experience it and then just do that enough times that you can know how to create these spaces. But the thing that I find interesting is that there's still architectural schools, there's still philosophies and theories and so like how do you make sense of like the fundamental theory of architecture and how that gets applied to the practice of architecture and how VR or AR could look at some of those theoretical concepts that may be coming from architectural education and potentially gain some insight for what it means to do this type of general experiential design, but specifically through modulating space to modulate experience.
[00:32:31.407] Luke Jones: I mean in a way I think that actually architecture as it's taught in universities has already started to anticipate the virtual in the sense that the types of projects that students often produce in architectural schools are already extremely narrative because like a narrative building in which each room has a very clear relationship to some sort of overall like narrative reasoning and Each step in the narrative produces a different sort of experience and that then dictates a series of more or less extreme design approaches for that particular moment in the building. It's already something people do a lot because it works as a satisfying paper project kind of approach for an architecture school because it gives you a lot to work with. In terms of architecture in the real world, what's the theory? I think that the theory in a way is like budget and materials and context and it's all sorts of other things which actually just don't transfer over to the virtual at all and in a way what's liberating about the virtual is the ability to explore spaces in this purely effective dimension and that you are freed from all of those sorts of things and you can get back to the question of what's at stake or all of the things which potentially you could do.
[00:33:51.282] Anna Mill: I'm in a sort of strange situation where almost the entirety of my architectural design work at the moment is in representing architecture. I'm not really doing any kind of practical architectural practice, kind of construction practice right now. So mostly I experience it through teaching students, but also I do a lot of drawings for other architects. So I do their presentation drawings and things like that. So weirdly, I sort of actually experience architectural design almost through a virtual simulation, which I'm drawing. So that has sort of become the process through which I consider architectural design as well, is that, as Luke was touching on, I always think about architecture through the minutiae of human interactions in that space. So I might be given just a box that's a room and the requirement is for me to fill it in such a way that it looks like an interesting and desirable space. And I do that through really thinking through all of the possibilities of the inhabitation of that space. but without actually being able to physically go into it because it doesn't exist yet. So I have to do this kind of virtual simulation in my own head and then translate that into a drawing on the 2D piece of paper and then that becomes the way in which I've been engaging with design and then that also is the way that I think about design for myself and the way that I encourage students to think about design is more through this imagined inhabitation of space and the feedback loop that comes from imagining what might be able to happen in this first iteration of a space and then changing the architecture in order to kind of affect the experience of that space and then drawing it again in such a way that there's this kind of imaginary feedback loop that's happening, but it's actually happening entirely in drawing.
[00:35:59.613] Kent Bye: Yeah, there's a lot of talk about this metaphor of control Z of the undo where you're designing and then you're maybe failing fast but then you want to revert and so you undo that and so I think within the context of software development we already have this iterative process by which that we can do these experimentation if we don't like it we could back up because there's a whole archive and history of each of these steps and so we can revert it in that way. But there seems to be this aspect of the virtual representation of architecture where you're actually able to get this full spatial experience of what you're designing rather than just having a 2D representation of that. And to me, I feel like that is going to potentially start to move architecture away from the way it is now, which is very much a waterfall. You design it, you build it, and then you kind of move on and you're done. but that into a much more live and dynamic iterative process by which that you're able to more quickly get yourself into these virtual environments that you're building and potentially even have other people experience those before it's even fully baked, but you're able to integrate that feedback. But then once you launch it, then think of it more of as a dynamic process that's still growing and evolving and changing rather than it's something that you're done with and you're kind of moving on. To me, that seems like it could be a huge paradigm shift within architectural design to move away from the slowness of the physicality of buildings, and then the virtual actually is going to potentially do this whole revelatory change in how architecture is even done as a design process.
[00:37:30.720] Anna Mill: Yes, I think that there's going to be an initial problem though which is that people will need to learn how to understand the sort of the half-baked architecture that's presented to them because at the moment members of the general public if you do a public consultation using a kind of virtual representation of a proposed piece of architecture they treat it as if it's a finished piece of architecture And because that's their experience is that they are only usually ever experiencing a finished piece of architecture. So the way in which they often engage with these kind of virtual representations of a proposal is by interrogating it as if it was finished rather than that it's a kind of in progress design. So there will need to be a kind of a way in which people learn to understand something as not a finished piece of architecture. before we can make use of this new process of designing.
[00:38:29.296] Luke Jones: Historically, iteration is something which has been very stressed in architectural design, but it's something which happens purely within the design process of a particular stage. So the technical design of the building, or the concept design of the building, or whatever, will have a strongly iterative process, but it's kind of sandboxed within that particular, otherwise necessarily sequential route that the project goes through. And yeah, in a physical building, yeah, you can't really, there's no like 0.1 release version of a building because people will fall down through a hole in the floor and die and that kind of like, I mean maybe that would be a better way of doing it, but our sort of legal and the political system isn't really able to deal with that as an approach. Yeah, obviously, the virtual does completely change that and turn it on its head. I think that there is enormous value in being able to subjectively experience a building as a kind of virtual proposition, but I'm resistant to the idea that it's necessarily going to be the only or better one, and that, you know, we have these very well-established forms of abstract representation of buildings and spaces, which are, you know, these like plans and sections and things, which in their own way are also an extremely powerful way of understanding relationships which are invisible to the human eye. And I think that one of the, for me, very interesting possibilities of virtual reality is for those kind of experiences which are neither a kind of naturalistic human impression of the world, like experience of the world, nor a kind of already existing, like, representation of the world, but something in between, a kind of inhuman virtual reality experience of, like, complex three-dimensional objects, I think is potentially really powerful.
[00:40:07.273] Kent Bye: Yeah, well, I think one of the other things that the virtual represents is this power to do dynamic architecture that is changing and evolving over time. And so, yes, currently we have a very fixed idea about what a piece of architecture is, but I don't think that we've seen very many examples of how we go into a virtual space and it's dynamic and changing and constantly moving and flowing. And so that to me feels like there's another element of what dynamic architecture is and what that means. And I don't know if there's an established precedent for dynamic architecture and being able to actually have something change over time, or if that's something that you think is going to be something that's going to be completely new within the realm of architecture.
[00:40:45.853] Luke Jones: I mean, for the historical perspective, I think that that is kind of a dream of the 60s in the kind of a literal real world sense that you would have these buildings which would be reconfigurable, which would be sort of imagine like a big empty kind of Amazon shed, but which then can reconfigure itself on the inside and can become whatever sort of building it wants to be. Intriguingly, some of those ideas have been brought out of cold storage and reintroduced as if they were new by some of the designs for the Google headquarters and these sorts of things. They've been rediscovered. I think that none of those things ever actually happened though. They were technically too difficult to pull off. And it was never really clear that they were that much better than doing things the old-fashioned way. And I think that it's the sort of zero weight, zero cost quality of the virtual. which means that those things actually will happen, that actually we will have a dynamic architecture because it's just as easy to do that as it is to do a static architecture, like there's absolutely no difference, so we will finally kind of make the discovery of what the potential of these things actually is.
[00:41:59.678] Anna Mill: There's something that I've been finding quite interesting recently, which is these spaces for VR experience when you're not looking at them through the headset. So you have these environments that are created almost out of these elemental architectural pieces, architectural experiences. So you have something which is a corridor and it can be overlaid to become any sort of differently clothed corridor and you have things that are a column and it can be clothed to become any kind of different column and it's almost like in order to create these VR experiences architecture is being distilled down to the kind of elemental parts that you need in order to be able to then reconfigure them to become any kind of space once you've clothed them in these digital layers.
[00:42:51.643] Kent Bye: Great. And so for each of you, I'm wondering what are some of the either biggest open questions that you're trying to answer or open problems you're trying to solve?
[00:43:04.680] Luke Jones: I mean, for me, the really interesting one is that I think we are only just starting to discover what these kind of models of interaction and creativity are going to be like in the virtual. And I think that those are the things where, at the moment, they're super fun and really imprecise, these sort of tilt brush techniques, which are kind of lovely. But I think it's something which can be glimpsed a tiny bit in these really interesting experiments which are starting to go on at the intersection of like hand tracking and AI and all sorts of different things that there is going to be a new way of interacting in a 3D way, which is going to be incredibly natural, incredibly fluid. and also will have kind of incredible depth and what comes out of that I think for me is the really exciting potential which I can just see now that it's going to happen but I can't tell what it's going to be like.
[00:44:03.746] Anna Mill: Yeah, I absolutely agree. It's the sort of creative potential and the sort of democratization of creative potential when there's not this material cost to creating things. And yes, at the moment, on the occasions where I do get access to some equipment, I still find it incredibly frustrating because obviously I'm very used to having a method of creating which I have pinpoint control over and I know exactly how to use it. and I can be extremely precise and the thing I have in my head can come out exactly as I imagine it on paper. And then when I go into a sort of virtual drawing program it's incredibly frustrating because I can't get anywhere near that level of, you know, the thing that I want it to be and actually creating that. And obviously there's something interesting that happens when you have these kind of constraints of a medium as well. suggesting that that can't be a kind of rich and interesting translation of your old and tired techniques once you develop them into a new space. But I think it needs to go another couple of stages of evolution until it becomes the tool that I have good faith that it will be and that I'm excited about discovering.
[00:45:28.285] Kent Bye: And I understand that you have a podcast of your own and maybe you could tell me a bit about like what you cover in your podcast as an architect
[00:45:35.985] Luke Jones: Yes, so I have a podcast called About Buildings and Cities. It's me and my friend George. We cover topics in the history and theory of architecture. Also a little bit about the future which we normally explore through science fiction. So quite a lot of futures of the past. Like we've recently recorded a lot of material about the way in which people in the late 19th century thought about the potential of the machine, machine utopias. And I mean, I think the title is kind of self-explanatory, but understood in the kind of broadest and most interesting way. You know, we kind of explore everything sort of from the earliest reaches of history to the sort of dimly imagined future is our kind of line.
[00:46:18.716] Kent Bye: Interesting. Great. And finally, what do you each think is the ultimate potential of virtual and augmented reality and what it might be able to enable?
[00:46:31.753] Anna Mill: I think, I'm afraid I'm probably just going to end up repeating myself. To me it's this sort of, it's what I can foresee as this amazing creative tool and that's what feels most interesting to me. It's not necessarily, although I'm sure it'll be very interesting to explore kind of new environments other people create, to me it's how I'll be able to use it to create for myself.
[00:46:58.368] Luke Jones: For me, one of the things which I've really remarked on in the projects which other people have been showing work today, and we've seen some really very creative VR-located installations, and the way in which these projects are able to take people into a space of almost primal behaviour, where they go back to first principles, they seem to reason like a child. And also they make these very, very strong and immediate like social connections with other people in the virtual spaces. And I don't know if that's just something which is going to be around for the first few times you use it, but there does seem to be a real like tenderness in the way in which people relate to each other in VR, certainly in these early stages, which I think is very touching in a way and like has a very hopeful sort of potential.
[00:47:46.468] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the immersive community?
[00:47:51.233] Luke Jones: If you want to find out more about the comic book, it's at squareeyescomic.com.
[00:47:56.618] Anna Mill: Yeah, and I think it's just, to me, it's very exciting to hear about all of these amazing ideas that people are generating. it really does feel at the moment like a collaborative effort and that's reaching kind of all around the world. And that makes it feel potentially very wonderful. And yeah, I'm excited. I'm just very excited to see where it's going to lead.
[00:48:21.651] Kent Bye: Awesome. Great. Well, thank you so much.
[00:48:25.412] Anna Mill: Thank you very much.
[00:48:26.585] Kent Bye: Thank you. So that was Anna Mill. She's a architecturally trained designer and artist who spent the last eight years working on designing and drawing this graphic novel called Square Eyes, as well as Luke Jones, who's an architect and co-author of Square Eyes. So I have a number of different takeaways about this interview is that, first of all, Well, I had a chance to read through Square Eyes before I produced this episode. At the time of the interview, I didn't have a chance to read through the graphic novel yet. I had just been given a copy of the book by Lara and Frederick. And so I enjoyed reading through the story and being able to see the different elements of paper architecture. So from an architectural perspective, looking at both designing the buildings and the world that they're in, but also the different social dynamics and mostly this graphical interface that they were kind of fleshing out in a certain way. It's very interesting to hear that they had taken the approach of not really focusing on the actual technology at all, but just to see what the augmented reality interface would be and some of the different user interaction designs that they're kind of fleshing out through this process of the graphic novel. Kind of imagine all the ways that we use our phone and be able to see messages and locate where we are with maps and what each of those different elements would look like with this pure spatial interfaces and how would you navigate it. So just as a conceit, being able to have the black and white be describing what the material reality would be, and then everything that's augmented or these virtual digital objects are in these highly saturated colors in order to show you what the digital and real and how they're blending together, but also to try to impart some sort of dynamic motion from this highly saturated colors that they were using. So there was a debate that had to come up a little bit earlier in the discussion. There was like a two hour discussion and debate, and there was this kind of question as to whether or not what you experience within a virtual experience within an architecture is fundamentally different than what you experience within paper architecture. for Anna she sees it much more similar to a virtual architecture because her experience is that she's going into these imaginal worlds she's exploring these different concepts and for her they're very visceral very real but there's something about the virtual reality being able to hijack all of your sensory experiences and actually give you a sense of that embodiment within these different spaces and so there's just kind of like this exploration of the lineage and practice of paper architecture and kind of compare and contrast it to what virtual reality technologies are going to be able to afford. And so I think there's a little bit of a difference in terms of phenomenological experiences, but in terms of Anna, in terms of her own creative process of kind of tapping into her imagination, she sees that it's very similar to other type of virtual representations of architecture in that Because she is an architecturally trained designer and artist. She's not actually building these things that she's Imagining and so for her this method of paper architecture into this more abstract representations for her it feels just as real as any other expression of these concepts and ideas and and that she has this kind of imaginal feedback loop cycle where she's able to imagine something in her mind, she's able to draw it out very quickly and easily, and then from that be able to continue to explore these different concepts and ideas. And using the graphic novel as a format to kind of expand and innovate what this kind of speculative design or paper architecture lineages and traditions are able to do within the medium of a graphic novel. So just listening to Luke, he was talking about how these traditions of paper architecture allow these architects to be able to explore these concepts and ideas without the pressure of them needing to be built. You don't have to worry about the limitations of physics or the cost of material reality or some of the different sociological limitations to be able to implement it. And so there's a freedom to be able to explore these different concepts without all these other constraints and limitations. that they're able to look at these either critical utopias or dystopias and be able to kind of project out the role of technology in relation to humanity and the world around us. And so this took a little bit more of a dystopic turn in terms of fleshing out the way that where technology is now and kind of porting out a lot of the digitally mediated environments that are controlled by other corporations and entities and what's it mean when you start to blur the line between the virtual and real and how's that impact somebody's life as they're navigating this fusion of the virtual and the real with who owns and governs different aspects of what can happen within these virtual worlds and as a result within the real world. So for me, it's just fascinating that they had been working on this project starting back in 2009, 2010, and then for eight years were exploring these different concepts at the same time as there was this whole renaissance and explosion of the very same technologies that they were exploring within their book. And I do think that there are certain aspects of what they're exploring that do start to look at these deeper relationships between our relationship to the virtual and the real and how much we use the technology to be able to create this mask to be able to hide the messiness and ugliness of some aspects of material reality. And what are the implications of that? And also it was interesting to hear Luke talk a bit about how it's easier to kind of talk about these digitally mediated realities if it's a consensus reality. There seemed to be ways in which they were sharing these different virtual objects with each other and being able to have them as the center point for different conversations. But he also said that maybe a little bit harder thing to explore is the more solipsistic way of like, what does it mean when everybody's in their own reality? They're perceiving different layers of reality overlaid on the reality. And what's it mean when people are into their own worlds, so to speak, which is already kind of happening in certain respects. But what's it mean to sort of amplify that even more to have these different layers of virtual representations onto reality? And how could that either separate us even more or potentially find ways that it could connect us if we're able to share these different virtual representations? There seem to be different ways of lowering the barrier between what's happening in your imagination and being able to actually express it within these virtual worlds. I know that Anna said that her own personal experience is that as an artist, she's able to draw something very quickly, take what's in her mind and her imagination to be able to express it and have this very tight feedback loop cycle. and that she doesn't necessarily feel like that it's at that same level within these immersive and spatial computing technologies, but that a lot of what they're exploring in Square Eyes is imagining a future where that is possible. To take your different concepts and to have very little limitations to being able to express different aspects of your imagination and creativity within these different virtual objects, and what would it mean to have this magic-like quality to be able to manifest any sort of spatial objects in reality without any of the constraints of material reality. And to me, it's just fascinating to be able to talk to not only Luke and Anna, but other architects over this one day symposium of the immersive architecture of the internet and to get these different perspectives of what is an architect? What does it mean to be able to design these different spatial experiences? It is interesting to hear how architecture has already been adapting in terms of being able to fold in new layers of different narrative and stories and going back into the history when it comes to like these gothic cathedrals that had stories embedded within the actual buildings. And I think with the printing press, it kind of, you know, took this turn into really focusing on the individual. And then maybe with these immersive technologies, kind of going back from the individual back into these different collective spaces. And what does it mean to be able to mediate different social dynamics when it comes to these architectures that are built within these virtual realms? And when I was at the architectural association, there was a lot of talk about these concepts of the corridor. And I think the night before we had our immersive architecture symposium, there was a whole discussion talking about architecture of prisons and how prison architecture, that is very much about, you know, isolating people into their individual cells. And I know Frederick and Lara were talking about how The invention of the corridor was also creating these individual rooms that are cut off, they're private, and then there's like this amplification of the individuality. And I think the trends that I'm seeing is just like this shift towards like these more communal open spaces that are getting away from us just being as individuals, but to get away from a lot of this isolation and loneliness and to find ways that we can connect to each other in new and novel ways within these virtual realms. And then if you're interested in hearing a little bit more of these different architectural breakdowns, you can definitely check out Luke's podcast about buildings and cities, where he's been taking a look at not only these speculative sci-fi worlds, but also exploring the earliest reaches of history and the dimly imagined futures with these different science fiction architectures and whatnot, and having these different walkthroughs and breakdowns of different concepts and ideas and really expanding upon them within these extended discussions on their podcast. So if you're interested in the process of graphic novels and speculative design, paper architecture, I'd highly recommend picking up Square Eyes. It's a beautiful piece of art and work, and there's a lot of really interesting conceptual ideas that are explored within this project. And I found it very satisfying, especially with blurring the lines between the physical reality of the grim nature of reality versus, you know, what are the implications of having these different aspects of Augmented reality and what is this going to sort of evolve over time and how is this relationship between the virtual and real Going to continue to evolve and this is I guess one potential future that on the back It says that square eyes is a graphic novel about a future where the boundaries between memory and dreams and the digital world Start to blur. It's a kaleidoscope mystery story, which asks in a city built on digital illusion. What really holds the power? What is weakness and when is it most dangerous? So that's all that I have for today. And I just wanted to thank you for joining me on the Voices of VR podcast. And if you enjoy the podcast, there's a number of things you can do. Just spread the word, tell your friends, leave a post on social media, and consider becoming a financial contributor to the podcast. This is a podcast that's supported by listeners like yourself. And so if you enjoy the podcast, then consider becoming a member. Just $5 a month makes a huge difference and allows me to continue to bring you this type of coverage. So you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
