A THEORY OF LEARNING AND MEMORY:
POPULAR ACCOUNT Copyright 1996 Edmund Furse

INTRODUCTION
Edmund Furse has been working for over ten years trying to understand how university students understand pure mathematics. He has chosen this rather obscure subject because his first degree was in maths with physics, and he did a PhD in pure mathematics, so has some personal experience, before he went on to study Psychology, Artificial Intelligence (AI) and Cognitive Science.

Whilst at Warwick University's Psychology department as a research fellow funded by GEC, Furse taught part of a master's course in Cognitive Science, and gave the course on machine learning. He was very dissatisfied with the very small scale learning that was modelled by all the different approaches, and felt it did not do real justice to the human condition. Hence, began his search for a model of learning that dealt with learning over months and years, rather than just a few seconds or minutes, and that could handle the learning of thousands of facts, or even millions.



MODELS OF HUMAN LEARNING
The problem with human learning, is that so much that we learn is in terms of what we already know. This makes obvious sense. For example, we learn that Paris is the capital of France, but could not really learn this if we did not have some previous idea of what a capital city was, or a country. This, so called learning of facts, is known by psychologists as "declarative learning" to distinguish it from "procedural learning", a distinction made by amongst others the American Cognitive Psychologist, John Anderson.

Anderson built a large model of human learning, memory and problem solving known as ACT (Adaptive Character of Thought), and it has had many different versions. But, he models the way we improve our learning, and do tasks faster, namely how the things we know become proceduralised. For example, when you first learn someone's telephone number, you dial it very deliberatively, one digit at a time. But with practice, this gets faster, until the skill is completely automatic (automatized is the technical term), and then when you think of the name, you can immediately recall the number and dial it.

Interestingly, this is what Freud would have called it becoming unconscious. Indeed, the unconscious is essential to us as human beings. We could not possibly do all the things we do if every thing had to be done under conscious deliberation. Of course, for Daniel Dennett and other philosophers of consciousness, there is no strict distinction between conscious and unconscious. Rather, there is a continuum between conscious and unconscious experience.



HUMAN AND MACHINE LEARNING
One problem with most work in artificial intelligence (AI) is that much work on trying to model human thinking, e.g. in problem solving, does not make this distinction between different levels of conscious processing. Therefore, in most AI models, in some sense, all the thinking is done at the same level of conscious deliberation, and as a result, is ridiculously slow, and frequently the models do not scale up from small scale toy problems to real world problems. Some people are addressing this difficulty, especially Stuart Russell, who uses the notion of "anytime algorithms" which must deliver a result whenever the conscious deliberation of an agent interrupts the lower level processing which is "doing the thinking".

Humans however, are fairly slow to learn, but fast to decide, and to act. As Alan Newell pointed out many years ago, there just cannot be many synaptic jumps from one neurone to another in the 2 seconds or less that we often make a decision. This is part of the reason for the popularity of connectionism as a philosophy for cognitive science, and neural networks as an approach to understanding the nature of human mental processes.

However, many neural network approaches, especially the most popular, known as "back propagation" take a ridiculous amount of time to learn even the most simple of tasks. These models frequently take thousands of trials to learn to classify some simple examples that people would do with just a handful of examples. Furthermore, most such models just learn to classify examples. But, there is far, far more to human learning than just classifying examples.



LEARNING FACTS
Furse, then is pioneering a new theory of learning and memory, known as the Contextual Memory System (CMS). This is essentially a model of how people learn facts. It is a model of declarative learning, and also a simple model of both perception and attention.

The difficult problem in trying to understand the nature of the learning of facts, is how can one possibly learn something new? This is a very old problem going back to the Greeks. Meno's paradox, the 'learning paradox' derives from the ancient Greek sophists who argued that truly novel learning was impossible in that "novel knowledge cannot be derived completely from old knowledge, or it would not be new. Yet the transcending part of it cannot be completely new either, for then it could never be understood."

This problem was revisited in the 1950s by Arthur Samuel when whilst working for IBM he developed a model of how people learn to play the game of draughts. He built a model with a fixed set of 32 features, and the system learned which subset of features of the board was most useful, and what weight to assign to them. But he searched in vain for a method of creating the terms (features) from scratch, and called this problem "the problem of the creation of new terms". This has remained a central problem in machine learning with workers such as Tom Mitchell highlighting its importance in the 1980s.

Furse proposes that the solution to the problem is that the features must come from the environment itself, rather than inside the agent. In a prize winning paper at the British AI conference of 1993, "Escaping from the Box", Furse argued that nearly all models of learning tended to pre-characterise the space in which learning was to take place, a sort of set of pigeon holes decided in advance. Then, learning was to remember which pigeon hole the new information was to go into. This, he argued was a totally inadequate account of the human experience of learning. We do not decide in advance all the different characteristics of birds before we start to learn about them; rather we pick up our knowledge of birds and their features in a haphazard way dictated by our interests and experience of the world.



THE CONTEXTUAL MEMORY SYSTEM
This model of learning and memory, the Contextual Memory System, (CMS), starts with no features and no items in memory. It thus starts as a complete tabula rasa. However, it does have built in perceptual MECHANISMS which given an object in the outside world, it can build very large numbers of features of the object. Thus, the ACTUAL features that are built are purely a function of the objects that the agent encounters in the world. If the agent spends a lot of time looking at birds and rabbits, then he will naturally acquire many features relevant to birds and rabbits. In contrast, if he spends his time studying the business news, then he will build many financial features.

The trouble with testing such a grand theory of learning and memory, is this problem that what we know affects how we see the world, and therefore there is an immense problem in ever getting a computational model working. Historically, Skinner and the behaviourists banned all theorising about what went on inside peoples' heads as unscientific, because it could not be tested. These dark ages of psychology which ran from about 1930 to 1960 and even beyond were eventually supplanted by modern Cognitive Psychology, and, in particular, Cognitive Science.



COGNITIVE SCIENCE
Cognitive Science, in contrast to Artificial Intelligence (AI), is interested precisely in what does go on inside peoples' heads when we think, and learn and do all the other intelligent activities that make us human beings. The method that Cognitive Scientists largely use is to devise theories of what they believe people do in performing some task or other (e.g. playing chess), and then to model this theory as a working computer program. If the program then performs the task IN A SIMILAR WAY TO HUMANS then it is considered to be psychologically plausible, and a likely candidate for the actual mechanisms performed internally by humans. Some researchers, notably John Anderson, go further and do detailed comparisons of the behaviours (protocols) and timings performed by people in ensuring that the model is genuinely psychologically plausible. But, in many cases the tasks are so complex, that just getting a program to perform them is worthwhile for its own sake.



THE MATHEMATICS UNDERSTANDER (MU)
Furse's solution to the problem of testing his theory of learning and memory was to build the CMS model as a computer program, and to use it within a large computer system to learn university level mathematics from special texts. This model is provocatively known as "The Mathematics Understander" (MU), surely a red rag to Roger Penrose, who argues that machines can NEVER understand mathematics. The texts were hand written in a very expressive but formal notation, known as the Formal Expression Language (FEL), and attempted to be faithful to the author's original mathematics textbook. Some preliminary research shows that students find it easier to read FEL texts than the original texts. But, more interestingly MU can learn from these texts too.

The Mathematics Understander (MU) learns pure mathematics by the reading of a text written in FEL. The level of concentration of the agent can be modelled by altering parameters of the CMS, such as the number of features to use when encoding a result. Even the level of "scattiness" of the student can be modelled by adjusting the ratio of old features known already to the system to brand new features created on the fly. Thus the person who is slow to pick up new knowledge can be modelled by using many old features, and few new features.

MU learns the definitions of mathematical concepts as it encounters them, for example the notion of a prime number, like 5, 7, 11, 13 is expressed by an author of a mathematics text as:

Definition
x is prime
if and only if for all n if n divides x then x = 1 or x = n

and would be expressed in FEL as:

Definition 1.3 of prime
x is prime
iff forall n (n divides x => (x = 1 or x = n))

where one assumes that MU has already learned the concept of "divides". Notions such as "iff", "=>", "=" and "or" can also be learned, but so far MU has to have "forall" built into its processing.

When MU reads a theorem, it just remembers it together with various specialisations (operationalizations) that it generates, so that it can be subsequently used in understanding other theorems. If the theorem is followed by a proof, then the proof is checked line by line, and MU gives an explanation of each step in terms of its existing knowledge of mathematics. MU can also solve simple problems, but in general its proof checking is better than its problem solving.

MU has successfully read texts in branches of pure mathematics known as "Classical Analysis", "Group Theory" and "the Calculus", and will shortly be made available over the Internet for downloading for other research groups to use. Furse has published papers in international conferences (including the prestigious IJCAI, and a video) describing his work, and more recently has submitted a lengthy account to the journal "Cognitive Science".



OTHER AREAS OF LEARNING
Furse has also been working on other areas of human learning, including board games, computer programming, foreign languages, and music. Again, the same methodological approach has been taken to study a genuine human learning task from an educational area, and to largely look in detail at the text books used to teach the subject. Then, his students build complex computational models which attempt to be psychologically faithful to the real human learning task. Needless, to say much of this work is in its infancy, but one student, Graham Beven, has submitted his PhD on a system that is able to learn, in principle, any board game within a certain class of games, with little prior knowledge.



LEARNING IS A BIG SUBJECT
Furse's interest in learning continually challenges the more limited notions of learning to be found in the literature. Examination of AI and machine learning conferences and journals shows that the vast majority of the papers are about one small aspect of learning, namely learning from examples and their classification. Yet, this is a small part of the human learning experience. Humans probably have at least 40 different kinds of learning mechanisms, many of them shared with animals through evolution.

Furse argues that Cognitive Psychology and Cognitive Science have an important contribution to make in trying to understand these various learning mechanisms, ultimately linking up with research from neuroscience, so as to provide theories and models that work all the way down from a model of the task down through higher level mechanisms, through large assemblies of neurones, and ultimately to the detailed neural processing. There is a great need for communication between the Cognitive Science community and the neuroscience community, and much of this already exists in the modelling of small parts of the brain. But, Furse argues that this co-operation needs to also be done with higher level mental processes, and here his theories of learning and memory are of great relevance.

Furse is currently working on newer theories of learning aspects of everyday life, apart from the learning of facts.



PSYCHOLOGY AND RELIGION
In parallel with Furse's work in Cognitive Science, he works as a member of the Catholic Psychology Group (CPG). The CPG is a group of Catholic Psychologists who meet twice a year to discuss a wide range of topics of common interest. Furse is a former chairman of this group, and currently on the committee. As a Catholic he has applied his knowledge of Cognitive Science and Psychology to our understanding of what it means to be a person, and in particular, to the nature of religious experience.



THE STRONG AI THESIS: INTELLIGENT ROBOTS
In common with other workers in AI and Cognitive Science, Furse believes in what is known as the "strong AI hypothesis". Simply put, this states that one day, some time in the future, there will be intelligent robots which can do any intelligent task currently performed by humans. This thesis is defended by philosophers such as Daniel Dennett (at Tufts University in America) and Aaron Sloman (in Birmingham, England), and opposed by philosophers such as John Searle and Dreyfus. More recently the British mathematician Roger Penrose has entered the debate with his books "The Emperor's New Mind", and "Shadows of the Mind". Penrose takes a strongly anti strong AI position, and argues in the more recent book that it is impossible for a machine to understand mathematics. Therefore, if a machine cannot understand mathematics, it cannot be a mind. However, Furse's MU program is a simple demonstration that Penrose is probably mistaken.

Whilst the belief in intelligent robots in the future is well established among philosophers of the mind, and probably also among the young who watch Star Trek, it is less well known among the general public, and, in particular, by theologians. Thus, the man in the street still believes that "a computer can only do what it is programmed to do". "False", writes Furse in recent letter in the Catholic International "The Tablet", because MACHINES CAN LEARN.



LEARNING, THE PERSON AND RELIGION
Thus, learning comes centre stage in arguments about the possibility of what it means to be a person, and about arguments about consciousness, and arguments about strong AI and robots.

Most controversially, Furse argues that it is perfectly acceptable for a Christian to believe in strong AI, and even a complete computational account of human nature, without being inconsistent with his Christian faith. Here he builds upon work by Margaret Boden at Sussex University who has argued extensively and clearly that free will and mechanistic accounts are not incompatible.



THE THEOLOGY OF ROBOTS
Furse went further when in 1986 he published a paper in the Dominican journal "New Blackfriars" entitled "The Theology of Robots", and argued that at some point in the future it would be possible to build intelligent robots, and some of these would have religious experiences and pray. Interestingly, Alistair Hardy, who conducted extensive research into the nature of religious experience at his unit based in Oxford, believed that there was a part of the brain responsible for this common human experience. Hardy, and later David Hay at Nottingham, showed that about 48% of the population of Britain had religious experiences of some kind or other, even though few of these people go to church.

Furse, recently whilst at the 7th Mind and Brain Symposium in London (The Science of Consciousness: the nature of religious experience) took this a stage further in the discussion (he was not a speaker), and argued that it would be possible ultimately to understand the workings of Hardy's "religious experience centre" in the brain, and to build it into robots as part of their brains. Then, with suitable learning through their upbringing, there would be Christian robots, Buddhist robots, and atheist robots. In short, Furse argues that robots will pray!



CONCLUSION
Learning is vital to understanding the human condition. Freud believed that dreams were the royal road to understanding the unconscious. Furse argues that understanding the nature of human learning is the new scientific road to the understanding of the mind. This understanding will, in time, encompass a broad range of human experience, from the mundane to the sublime.

Top Level
Edmund Furse's home page