DeepMind, AI and Spatial Memory: an interview with scientist Talfan Evans

Neuroscience and artificial intelligence have a long and symbiotic history. In recent years, both fields have flourished. Many discoveries in one domain have influenced the other. Taking just one example, DeepMind’s use of experience replay to train Deep Q-Networks to superhuman level in Atari games was explicitly inspired by the neuroscientific observation that neural activity in the hippocampus is ‘replayed’ after experiences.

Where might the next advance in this intertwined history arise? We spoke to DeepMind scientist and neuroscience researcher Talfan Evans to find out. Talfan has a fascinating perspective on this question. Having originally studied mechanical engineering, he did a PhD and postdoc in the neuroscience of spatial memory and its application in robotics before joining DeepMind.

I ask Talfan about this trajectory, and how he learned the skills and knowledge that got him to where he is today. I also asked him for the 3 books he would take with him if he was stuck on a desert island. I hope you enjoy this conversation as much as I enjoyed chatting with Talfan.

---

Matthew:

If you had to describe what you do, what would you say your role is and what that involves?

Talfan:

Well, I come from a background of doing pure research in academia in machine learning and cognitive science. Research at DeepMind works very similarly.

So most of my time is spent developing new machine learning algorithms. And there are lots of ways that you can do that. Because we’re generally using neural networks you can, for example, change their architecture, or you can also change the objective of what you're trying to train on so you can try to teach the network to do a different thing. Most of the time, I’m either writing down the math from first principles or coding it up when I already have that theoretical framework. And then when I've done those things, I run experiments to test things that I don't know the answer to ahead of time, and we analyse those just like you would a science experiment. So you look at the results, you plot them and you try to figure out what's going on.

M:

So when you have a blank canvas in front of you and you're picking what you're going to do next, how does that process work for you? How do you pick what you're going to do next?

T:

Well, I guess you could break it into three things. So one is long-term intuition and interest that you've built up over years of getting used to a particular field. You will have things that you just feel intuitively interested in or want to test.

And then there's number two, there's what's hot right now. For example, if there have recently been a lot of interesting innovations in a particular field that you think you'd like to test out and build on top of, that's another way to think about planning projects.

And number three, often things come directly from the project you were just working on. So there're certain times where you'd be trying to do one thing, you'd have an idea for a side part of that you don't have time to explore at the time. And usually those are divergences that you queue up for later.

M:

Yeah, that makes complete sense. I know you've done a variety of different roles like a PhD and you've done machine learning research. Would you say that there's a through-line between all of those things that you're aiming towards? And if so, how did you come to that as a thing that interested you?

T:

Yeah. So there have been a few things that I've been interested in, throughout my PhD and then in my postdoc, and now my work at DeepMind. So it's easier to start at the beginning because I developed a lot of those ideas when I was doing my PhD.

I guess I was interested in this idea of hierarchical representation. So in my PhD I was studying how the brain might map the real world. And part of that problem is, if you learn something new about some object in space, you want to integrate it into your map in a way that doesn't take too long. Now, you can imagine that this could be difficult if your map was previously incorrect. For example, you only ever travel on one route from home to work. And in your head you’re taking these right turns and left turns through a mental map until you arrive at the office. You’ll store the office at a location on your mental map based on estimates of the trajectory along which you’ve walked, which might be imperfect.

Now, what if one day you actually take a different route and you think “oh, well, I know that the office is right at the end of the street and I take a right”, because you're using a version of that map in your head to plot a new trajectory, and then you take the right turn and it's actually nowhere near you. You have to reintegrate what you've previously thought about where you stored the office on that map. So you've got to slot in this fact that actually, the office is another street and another left away on the map from where you’d previously placed it.

Doing these sorts of corrections is also a problem in robotics, especially when you have a big map. But there may be some ways around it. One thing that I was interested in finding out about was whether you can make this correction process faster if you represent space in a hierarchical way. So instead of having a flat map where everything is, say, measured in units of metres, you actually represent it as a hierarchical map. In this case, you’d have one scale of representation which says the office is in the same state or country as your house…and then another map of a different scale which says the office is in the same county as your house…the same town as your house…the same, you know, square kilometre or something…all the way down to the finest detail. And there's a lot of evidence that something like this is going on in the brain. And these multiscale representations have to a certain extent been worked into state-of-the-art systems for robotic mapping. But it's still unclear what the best way to do that is, so people are still working on this problem.

Now, I don’t work specifically on space in the sense of Euclidean space, but I work on memory models of more abstract concepts. But I still think about how your memory might represent those abstract concepts in a hierarchical way; at a large fuzzy scale, all the way down to some fine details. I think about whether that's actually good for an abstract memory system.

M:

So I think that's absolutely fascinating, the generalisation of the idea from spatial memory over to conceptual memory and abstract concepts. Can you talk me through why having a hierarchy helps at all? What's the benefit?

T:

It completely depends on the structure of the task that you have. But a good example would be this. Let's say that I give you a task to change some PIN code for a lock from some initial integer to a random integer between 0 and 1000. Let’s start from e.g. 0; you would have to wind the wheel on this mechanical object all the way from 0 to 600. And then you know, if you want to change it from 600 to a new random integer, say 10, you've got to wind the wheel all the way back, right? It takes a long time to traverse through that space when you’re moving in units of single integers.

A more efficient way to do it is actually if you represent the integer 0 to 1000 not on a single scale (like on a single wheel), but instead you have three wheels. So you have one wheel that tells you 0, 1, 2, 3… and then another wheel that tells you 10, 20, 30 and a third that does for 100, 200, 300 etc. Because you've got 3 wheels, now it becomes much easier to change from 0 to 600, because I can just dial 6 on the 100 and I'm done. And I can do that if it's 666 because I dial 6 on the 100 and 6 on the 10 and then 6 on the 1. So I've only got 18 turns versus 666 turns in the previous case. If you think about how to efficiently move around in this space, you get some intuition about why it might be easier to move around objects on e.g. a spatial map, if you represent that map across different scales of detail.

Also, when we think about planning maps, we don't think ‘I'm going to go from here to the office, I'm going to go right and left and then forward 10 metres, etc’. No, you say something like, ‘I'm going to get on the bus, which will take me to town, and when I get off in town, I'm going to go to this district.

M:

Yeah, that’s exactly how I think about navigating. I think ‘we're going to go in that direction and then I'll figure out the details when I get close’.

How does that then map onto how space is represented in the brain? What's the analogy there? You mentioned there's evidence that this happens in the brain. What does that look like?

T:

So specifically there are a couple of areas in the hippocampus, which is a part of the brain, that're associated with representing memory and space. There are a few different kinds of neurons which respond in very specific ways to where yo’re in the physical world.

So first of all, people discovered that there were these cells called ‘place cells’. These are cells, or neurons, that if you pick one and then you observe it as you move around in space, you'll find that it'll only be active in a specific location. By active, we mean that the neuron does this thing called ‘spiking’, where it produces an electrical impulse. So what that means is it only starts spiking when you're in the top left corner of a room, or it might only spike when you're right next to a particular chair.

Now, after that, there was another finding of a type of cells called grid cells. They're a bit like place cells except instead of firing in one location in space, they fire in a periodic grid across space. So they would fire where you're currently standing and then again if you walk 10 metres forward, again at 20 metres and so on and so forth. And then you can imagine the same thing in the right and left directions. So you get this grid pattern if you visualise where they fire and they don't fire across physical space. But the interesting thing is that you find different cells with different spacings in between those grid points. So some will fire every metre, some will fire every 10m and some will fire every 100m.

And actually, there's been some really cool experiments to show that bats have these cells with periods that span over really long distances. So there’s an idea that grid cells work together to represent space on different scales.

M:

And then if I'm right, this has also been linked to conceptual spaces as well?

T:

Yeah, that's some really cool work. So exactly there was a paper by Alexandra Constantinescu from Tim Behrens’ lab in Oxford, where they had this task where you had to look at a cartoon bird on a computer screen and make the bird more like a different reference bird. And you could do two things: you could stretch its legs and stretch its neck.

They figured out that if they measured the spatially responsive neurons whilst people were doing the task, they would find these kinds of specific sensitivities in firing when people stretch the different components of those birds, even though this was not spatial in the sense of location in space. It's clearly got some scale variation, and it seems that the same cells might also be responsive to that.

M:

It's absolutely fascinating. I'd like to just change it up a little bit. So when you started your PhD, what sort of skill level and knowledge level would you've sort of said you're at and how did you know how have you progressed since then?

T:

Upon entry I’d done an engineering degree, so I came into it with a fair understanding of being able to do math, the traditional math like differential equations and matrix algebra. But I really wouldn't say I was particularly good at anything machine learning relevant, because I didn't do any machine learning when I was in my undergrad. I didn't really study stats much. So that was something that I really had to learn during my PhD.

The second thing was coding. We did some during my degree, but even though it was an engineering degree, we really didn't do that much coding. So I didn't know anything other than a bit of MATLAB. So my PhD was about coming up to speed with Python. But I've done most of my deep learning, engineering, and coding skills, since I came to my phd, then my postdoc and now at DeepMind.

M:

And how did you go about doing that? Was there a process you went through? Or was it trial and error?

T:

I mean, from the perspective of somebody who's trying to learn how to code, I hear so many examples of people saying ‘I really wanted to learn Python, or you really wanted to learn JavaScript. And I tried for a while, but it wouldn't stick.’

I don't think there's anything better than having a project that you need to complete that really requires that you have to be able to code. And it was the same for me. I got to my PhD and there were certain things you just can't do without being able to run simulations.

M:

And so how would you then go about actually learning to do it? You've got this problem in front of you, what's the next step?

T:

It's really hard to begin with. I have to say a lot of the cognitive difficulties I found are like… understanding how specific you have to be depending on the language that you're working with. Something like Python is much more forgiving than C++, for example, because you don't have to tell it the data type ahead of time. It'll kind of understand you, and you’ve got all the shortcuts for writing arrays and things like this. So part of the frustration is thinking things will work when you write them and then they don't.

M:

It’s the ‘I told the machine to do a thing and it's not doing it’ feeling.

T:

The hard lesson is really just learning that you have to be super careful about everything that you put in. And always check your work, don't assume, because you'll spend a long time debugging.

So how do you learn? Component by component, doing the easy thing to begin with. And a lot of time debugging…

M:

Yeah, that sounds right. I have two more questions. One, on your website, I saw you mentioned that you admire Michael Nielsen and Andy Matuschak. Why did you pick them out? What shines out about them?

T:

When I was setting up that website, I was doing a lot of reading into tech and progress in tech. So, specifically, Michael Nielsen wrote this really, really influential (on me) essay called ‘Thought as a Technology’.

And the idea was this: Pre-paper, people used to memorise everything and they would never write things down. They would only exchange information in the form of speech. Maybe there were primitive symbols. But his point was that once somebody invented pen and paper, i.e. people invented a written language that you could use to express your thoughts and record them, which allowed people to really augment their ability to think about things. Because now you have this temporary memory store. You can actually perform logical operations on things that might be really hard to think about, like math. And then you can arrive at answers that you wouldn't be able to if you didn't have this pen and paper, and his point is that you can think of paper pen and paper as some tool for augmenting our thought. And it's a good one. It's been around for a while.

But what is the next thing? What are other tools that we can use to augment the way that we think about things or do cognitive processes? I really like the way he thinks about things.

He wrote another essay with Andy Matuschak called quantum country. So they had this idea to build what they called a mnemonic medium. The idea being that what if there was something which was  like pen and paper, but which would react to the way that you were interacting with it. So he set up this experiment where they tried to teach people quantum physics. And normally we just have textbooks that have all these equations in them. Their idea was: can we somehow record people's progress as they are reading these equations, and then test them as they're reading?

So they would set up little interactive widgets in this online paper which would test their understanding of the stuff they’d read, and monitor how people were doing., Then they would prompt people periodically, e.g. .Sif they came back to it every couple of days, with questions that they that those people would be bad at. So here, what is the thing beyond pen and paper? Is it pen and paper that actually has an understanding of how well you're reading the stuff on the paper.

M:

Yeah, it's super interesting. I love that work so much and their writing is great. It's really good.

Final question. I'm going to put you on the spot. If you were going to be stuck on a desert island and could take three books with you, which three would you take?

T:

Okay, let me let me dig around. I guess the first one would be this book, Glass Bead Game by Hermann Hesse. So this is a classic book for people I think especially like you and I who have a natural interest in math. And I guess for me also music and not to the same extent, but I love the idea of the structure in something like chess.

Now, this book is about a fictional world in which there is this college where people go to study music, mathematics and games. And there is this thing called the “glass bead game”, which is the highest form of this kind of reasoning that the author is suggesting is present in all those three things. And it's just such a cool book to read, because you just think, ‘oh, the author is tapping into something that I felt was maybe like…quite personal’ before.

The second one that I really liked reading when I was a kid was Bram Stoker's Dracula. I don't know exactly why but I just love the kind of noir feeling of it.. It's dark and gothic and wet, especially at the start when Jonathan Harker is arriving at the castle.

The third book is called Starmaker by Olaf Stapledon. It's this sci fi book which is only about 100 pages long, from the 1950s or 60s I want to say. It's not a book that many people have heard of, but it's just an amazing book for its imagination.

So the idea is that there's a guy sitting on a hill in London somewhere and he's looking out to the sky and all of a sudden he feels himself leave his body. And then he finds himself drifting through the cosmos. Not in any particular form, just drifting past planets and seeing the different kinds of civilizations that might exist somewhere in the universe. And then at some point, he figures out that he can control to some extent his motion. He can actually start inhabiting members of different civilizations. And then the middle part of the book is just describing his travels and how he felt. What it was like to be these weird man type creatures? And also creatures that are definitely not man like. The imagination he has in describing these civilizations is amazing. Not just through space, but also through time, so it progresses eventually to very, very far into the future of the universe so that these creatures go through different stages of their civilizations.

M:

Amazing choices, Talfan. I love the Glass Bead Game as well, and will definitely check out the others. Thanks so much for your time, I think this will be really useful to people.

T:

My pleasure, thanks for having me.

---

Links and references

Talfan's website, where you can find his previous academic work: http://www.talfanevans.co.uk/