August in San Diego 4. Memory, Wayfinding and Design
Neuroscience For Architecture, Urbanism & Design
Michael A. Arbib
This is the fourth of a series of nine posts on the A«Nblog reporting on the “Neuroscience For Architecture, Urbanism & Design” Intersession held at NewSchool of Architecture & Design in San Diego on August 12-15, 2019. The individual posts range in length from 1300 to 3000 words. The first post provides an overview of the series, along with a Table of Contents with links to each of the posts. A PDF of the whole series may be found here.
A major concern of this post is with memory systems linked to specific brain networks, whereas the next looks more at general learning mechanisms at the synaptic level and their translation into artificial neural networks for Artificial Intelligence (AI).
Core Neuroscience: Episodic Memory, Place Cells, and Navigation
The hippocampus has long been a focus for study of brain mechanisms of memory. One key finding came from neuropsychology, the study of effects of brain lesions and neurological disease on human cognition and behavior. A young man named H.M. had such severe epilepsy that his surgeon decided (perhaps too drastically) to remove a massive bilateral area of brain including the hippocampus. Although banishing epilepsy, the result was drastic: HM lost episodic memory. Although he remembered events from before the surgery, he could not recall subsequent ones (Scoville & Milner, 1957). If you interacted with him, he might appear normal – his working memory seemed intact – but if your interaction were interrupted while you left the room, he would have no memory of you when you returned. The big surprise, though, was the discovery that H.M. nonetheless had procedural memory – he could acquire new skills. (Check out the movie Memento if you do not already know it.) If you played a new game with him on one day, he would not remember playing it the next. Any yet, after playing it again and again (each time believing it to be for the first time) he became increasingly skillful. These observations reinforce the points that (i) memory takes many forms, and (ii) that the brain is a collection of diverse regions making distinctive contributions to the patterns of activity and memory that cohere to make a self. We have already noted that John Zeisel has developed architectural strategies to help Alzheimer’s patients – and Alzheimer’s is a progressive disease whose initial impact on memory is associated with damage to the hippocampus.
The other key finding about the hippocampus came from the use of single cell recordings from rat hippocampus as a rat navigated around a simple lab environment. O’Keefe and Dostrovsky (1971) discovered that certain cells fired not according to what the rat sensed or was doing but rather as to which region in its cage or maze it was located. They referred to these as place cells. Building on this, O’Keefe and Nadel (1978) viewed The Hippocampus is a Cognitive Map while noting that there is also a taxon navigation system– we might paraphrase the latter by noting that we can navigate by referring to available affordances when we are in an environment for which we have no prior knowledge. My own group developed a computational model of these systems and their interaction, though we argued that the hippocampus does not of itself support a cognitive map but rather makes crucial contributions to an overall system for map-based navigation (Guazzelli, Corbacho, Bota, & Arbib, 1998).
Wayfindingin and out of Virtual Reality
Dane Clemenson uses Minecraft to study navigation and even creativity. The computer can explore moment by moment how people explore a rich Minecraft environment over multiple weeks, and he now has a setup to have people play Minecraft in fMRI (results are preliminary). He can also explore creativity by having subjects start with the same flat world in Minecraft and build a different world.All this raises the question of how well spatial ability translates between real and virtual environments. Using a videogame in which locations are based on Los Angeles, he studies whether there is transfer between real-world locations and experience of these locations as represented in a virtual world. [Certainly, many of us can do this by studying street maps to create an ad hoc cognitive map that guides actual navigation.]There was transfer between an object location task in VR or reality [RR: real reality?] with pre-exposure in the other mode, but only in some conditions not others.
We may navigate space egocentrically following landmark-by-landmark directions for a specific route, or allocentrically, having a cognitive map that can support finding different routes with different destinations. To a first approximation, the former involves the striatum while the latter involves the hippocampus but, as already noted, these are embedded in a larger system. In an object location maze, Clemenson looked for transfer between navigating a virtual and an actual maze and found differences between use of egocentric versus allocentric strategies that could be assessed via analysis of errors — but note that we get much more feedback when navigating the real world, in part due to proprioception. In summary, a range of studies explored the refined measurements of behavior that can be made in VR, while complementary studies demonstrated the extent of transfer between the VR and RR version of the tasks. One can learn from assessing this transfer. Another study, with Microsoft, studies active vs passive navigation. Relying on Google Maps does not yield a good cognitive map [as indeed is consistent with my group’s model cited above since using Google maps is a form of affordance-based navigation], and they are exploring the impact of using Microsoft Soundscape to generate auditory cues as to the direction to the next target.
Q recalled the notion of aboriginal songlines. Aborigines can navigate across Australia by following a path associated with a song by repeating the words of the song which describe the location of landmarks, waterholes, and other natural phenomena. However, if not knowing the language in which the song was sung, “the melodic contour of the song describes the nature of the land over which the song passes. The rhythm is what is crucial to understanding the song. Listening to the song of the land is the same as walking on this songline and observing the land.” Might this suggest ideas for infusing buildings and towns with auditory signatures?
Design– from Devices to Buildings
Don Norman lamented that there is unfortunately little interaction between Design and Architecture faculty in schools of architecture and design. He sees design as seeking a match between technology and people. Looking attractive is just one of these. His focus is on interaction design and user experience, emphasizing design in the small of devices and services rather than buildings. But perhaps these are not as disjoint as he suggests. For example, in designing a waiting line for Disneyworld rides, the solution was to add an anteroom to convey the story of the exhibit to distract from the fact that one is still waiting in line to get to the ride. In 1987, while still a cognitive scientist at UCSD, his group published a book User Centered System Design (UCSD!). After years in industry (including Apple) Norman returned to UCSD to found the UC San Diego Design Lab. The Lab takes a broad view far beyond the scope of Norman’s talk that extends to major societal issues such as large-scale education, automation, and healthcare. Design is viewed as a way of thinking “focusing upon how people interact with complex systems and technology, and upon ensuring that we solve the right problem, the root issues that define the true needs of the people and groups work with us, that we serve.” Clearly, such a system approach is relevant to the Urbanism thread of the Intersession, though this was outside the scope of the talk.
Norman views science is a procedure for conducting repeatable experiments. Theories can guide, and are tested by, experiments – with incremental modification far more common than paradigm shift. Evidence may be hard or soft, but has to be replicable by independent people who do not share a common bias. In making devices, one can make prototypes and test them before going into mass production. Science requires precision, but for much design work we don’t need the same precision. Not what is the temperature, but is it comfortable?
He noted that the Hong Kong Polytechnic has a new building designed by Zara Hadid. It has given the department great prestige — and yet it is a pain to work in. This is a conflict. How do we resolve the visual impact of prestige building versus people come first?
Norman sees the Bauhaus as ignoring people, but I wonder if this is correct. Margarete Schütte-Lihotzky’s Frankfurt kitchen was a design to reduce the time spent in housework, so that women had more time for factory work or to pursue other interests. Among her influences was the small kitchen of the Bauhaus-designed Haus am Horn that had specific storage and drawers for specific items – a manifestation of the more general time-and-motion studies of Taylorism and a precursor of the workflow analysis that helps guide kitchen design today.
A key observation was that with small devices design can be iterative: Build, test and modify. Norman is skeptical about human-centered design at the scale of architecture since one cannot iterate a building. He noted that after much iteration with unsatisfactory releases of Microsoft Window, Microsoft Windows 10 finally got it right. But I wonder whether this in some sense contradicts Norman’s point. Each release of Microsoft Windows was a massive system that was then tested by millions of users. Perhaps, then, the issue is not one of device size but rather the extent to which lessons learned from one form of a device or a single building or development can yield feedback that goes into the design of later devices or windows. Post Occupancy Evaluation (POE) is only carried out (according to one audience member) in 20% of major buildings. While this may or may not support upgrades to a specific building, it does add to growing expertise that is transferable to other projects. As Norman said, “Architecture is a social art, reflecting and changing culture” but it does not follow that “Architects design in a vacuum.” Experience and materials science research with glass yields new materials and new strategies for exploiting them in future projects. Moreover, research can be done as part of designing a building without resort to a full-scale prototype. At a talk in Taliesin West in 2011, Jeanne Gang discussed how her firm constructed scale models of protypes of the Aqua tower for Chicago along with the nearby buildings for wind tunnel testing that balanced the distinctive sculptural form of the balconies with the need to stop the winds of the Windy City making the balconies unusable. Today, designers of large buildings can call on experts in air conditioning (or its ecological avoidance) and “elevatoring” to combine science-informed engineering with cumulative architectural insights (and yet, see the next section, the result may be “new”). And Autodesk now has programs to prepare designs that architects can choose between to satisfy a set of specifications and constraints.
Elena Pacenti observed that although we do test use, we don’t test social impact or its effect on the brain. She is intrigued by the rise of Uber and other ride-hailing services and stresses the need to understand its social impact at the level of urbanism – will they reduce traffic or not? – and the impact on taxi drivers both in the impact on memory and the downgrading of their profession. Norman commented on the impact of working at home, a long way from laboring in textile mills. The work shift is societal. Global business also changes work habits as people interact across time zones with very different 9 to 5. Meanwhile, office design has shifted from private space to open space and now may be swinging back again. He quoted the dictum of Herb Simon (a pioneer of Artificial Intelligence and a Nobel Laurate in Economics) that we are always predicting the future for that’s how we make decisions. This is, of course, a principle of animal behavior that guides our studies in cognitive neuroscience, not just for modern Organization (Wo)Man. Yogi Berra (the baseball philosopher) noted that prediction is hard, especially of the future.
Neuroscience of the design process
Michael Arbib considered the possibility for a neuroscience of the design of architecture by quoting a “challenge” fromPeter Zumthor’s A Way of Looking at Things: “When I think about architecture, images come into my mind. When I design, I frequently find myself sinking into old, half-forgotten memories …. Yet, at the same time, I know that all is new and that there is no direct reference to a former work of architecture … “
In addressing this he recalled the linkage of hippocampus to episodic memory, noting that “There are striking similarities between remembering the past and imagining or simulating the future – and a common brain network underlies both memory and imagination,” quoting from “The Future of Memory: Remembering, Imagining, and the Brain” (Schacter et al., 2012). However, this quote is somewhat misleading since there are differences in brain activity between remembering the past and imagining the future, with different regions, as well as common regions, involved in the two tasks. There are now a number of empirical relevant papers such as “Anterior hippocampus: the anatomy of perception, imagination and episodic memory” (Zeidman & Maguire, 2016). My own work (outside the scope of the talk) is addressed to creating a theoretical framework for addressing such data that builds on my work on How the Brain Got Language (Arbib, 2012, 2016)yet links to the development of a neuroscience of design. The key idea is to see each of vision, episodic memory, and imagination as constructive processes. What we see is based on an assemblage of schemas that interpret different parts of the scene and the relations between them, but (as Tom Albright also emphasizes, see the eighth post) this reflects top-down influences of our prior experience and current motivation and tasks (recall the action-perception cycle) as well as current sensory stimulation. Such an assemblage is not a mere labelling of regions of the scene; it captures aspects of shape, texture, location and more. An episodic memory is then a recoding of the assemblage, but the recollection of this memory will be only partial and in parts inaccurate, perhaps distorted by the task at hand – but it can again be based on a schema assemblage. Imagination then builds on diverse such memories, taking parts of different assemblages and distorting them as they are reassembled, with the schemas acting as dynamic entities to create new narratives – and these may be dreams unconstrained by current sensory input, or imaginations contributing to and constrained by an ongoing design process. The worked out details will be published in my forthcoming book When Brains Meet Buildings.
Arbib, M. A. (2012). How the Brain Got Language: The Mirror System Hypothesis. New York & Oxford: Oxford University Press.
Arbib, M. A. (2016). Towards a Computational Comparative Neuroprimatology: Framing the Language-Ready Brain. Physics of Life Reviews, 16, 1-54.
Guazzelli, A., Corbacho, F. J., Bota, M., & Arbib, M. A. (1998). Affordances, Motivation, and the World Graph Theory. Adaptive Behavior, 6, 435-471.
O’Keefe, J., & Dostrovsky, J. O. (1971). The hippocampus as a spatial map: preliminary evidence from unit activity in the freely moving rat. Brain Research, 34, 171-175.
O’Keefe, J., & Nadel, L. (1978). The Hippocampus as a Cognitive Map. Oxford: Oxford University Press.
Schacter, D. L., Addis, D. R., Hassabis, D., Martin, V. C., Spreng, R. N., & Szpunar, K. K. (2012). The future of memory: remembering, imagining, and the brain. Neuron, 76(4), 677-694.
Scoville, W. B., & Milner, B. (1957). Loss of Recent Memory After Bilateral Hippocampal Lesions (Reprinted in J Neuropsychiatry Clin Neurosci 2000, 12, pp.103-113). J. Neurol. Neurosurg. Psychiatr., 20, 11-21.
Zeidman, P., & Maguire, E. A. (2016). Anterior hippocampus: the anatomy of perception, imagination and episodic memory. Nat Rev Neurosci, 17(3), 173-182. doi:10.1038/nrn.2015.24
 https://en.wikipedia.org/wiki/Songline. Even though I grew up in Australia, I first learned about songlines from Bruce Chatwin’s 1987 book The Songlines, though the book has been criticized for its over-reliance on non-Aboriginal informants.
About Michael A. Arbib
Michael Arbib is a pioneer in the study of computational models of brain mechanisms, especially those linking vision and action, and their application to artificial intelligence and robotics. Currently his two main projects are “how the brain got language” through biological and cultural evolution as inferred from data from comparative (neuro)primatology, and the conversation between neuroscience and architecture. He serves as Coordinator of ANFA’s Advisory Council and is currently Adjunct Professor of Psychology at the University of California at San Diego and a Contributing Faculty Member in Architecture at NewSchool of Architecture and Design. The author or editor of more than 40 books, Arbib is currently at work on When Brains Meet Buildings, integrating exposition of relevant neuroscience with discussions of the experience of architecture, the design of architecture, and neuromorphic architecture.