Why did AM run out of steam?
Edmund Furse
Lenat's program AM, A Mathematician (Lenat (1976)), is the best known program which exhibited creative behaviour in the field of mathematics. From a knowledge of 110 basic concepts of set theory, AM was able to discover numbers, prime numbers and a number of important conjectures from number theory, such as Goldbach's conjecture that every even number can be expressed as a sum of two primes. Much has been written about the reasons for this impressive performance, in particular Ritchie and Hanna's 1984 paper analyses the question of whether AM really works using the general heuristics described by Lenat in his papers. Nevertheless this paper is not concerned with this question, but rather the question why it was that AM did not make more discoveries, and when run always made the same discoveries. Thus the paper addresses the question of why did AM run out of steam?

The answer given by Lenat himself, is that AM ran out of heuristics, and needed new heuristics in order to make more discoveries. Lenat pursued this argument with the program EURISCO which did discovery learning in the domain of heuristics, and was successful for example in discovering a new three dimensional transistor structure. In this paper I want to put forward a different argument as to why AM ran out of steam, arguing that whilst AM does capture an important element of mathematical creativity, I think it did not make more discoveries for a number of reasons, all relating to AM's lack of modelling the real activity of creative mathematicians, in particular AM's ignorance of proofs.

There are four reasons, I believe, why AM ran out of steam:
1. AM ignores the social environment of mathematics.
2. AM has no problem solving or theorem proving ability.
3. AM only runs on one machine, rather than a network of model mathematicians.
4. AM has no ability to transmit or receive proofs to other systems.


Fundamental to AM's lack of realism in modelling the creative mathematician, is that it ignores the social framework of mathematics. No mathematician works in a vacuum, but rather mathematicians work on problems that are part of the mathematical culture, as argued for example by Kuhn (1962). It took several hundred years for the community of mathematicians to discover Goldbach's conjecture. It is unrealistic to expect a human mathematician to discover centuries of mathematics from scratch. Nevertheless there is always the problem that it is difficult to set up AM's prior knowledge to a pre-Goldbach environment, since many of AM's heuristics in manipulating functions rely on built in mathematical concepts of much later origin. Thus AM may already have much of the knowledge built in, in some implicit form, as Ritchie argued. However it remains the case that the creative mathematician works with other mathematicians, and does not dream up problems out of nowhere. Even Gauss, possibly the greatest of all mathematicians, despite not publishing important results sometimes for twenty years, worked on problems which ultimately had their origins in the work of other mathematicians.

Mathematics is thus a social activity in which mathematicians communicate with one another. This communication involves both determining what are the socially accepted interesting areas of mathematics (admittedly a potentially contentious issue), disseminating proofs, solutions of problems, counter-examples, and discussing problems which no one knows how to solve. Central to this activity is the notion of proof and problem solving. It allows one mathematician to spend weeks discovering a proof which he then sends to other mathematicians, who can understand the proof in a matter of minutes. If every mathematician had to discover all the theorems he wishes to use, it would take an absurdly long time.

It is clearly unfair to criticise AM for not having a theorem proving or problem solving ability, since Lenat was concentrating on the program discovering new concepts of interest to it, and for the program also to be able to prove its hypotheses would have involved considerably more work. However Lenat's argument that AM ran out of steam because it needed to discover new heuristics needs further examination.

Central to the argument of the importance of theorem proving to mathematical creativity, is the notion that by trying to prove hypotheses the mathematician pursues hypotheses that he is likely to prove. In particular, in trying to prove an hypothesis, the mathematician will generate further hypotheses which may be generalisations of the given hypothesis or subgoals. These further hypotheses are then investigated, to see if they can be proved or falsified by counter examples. Thus because the mathematician is trying to prove the hypothesis, the very activity of trying to prove the hypothesis generates new hypotheses. Furthermore these new hypotheses are interesting to the mathematician precisely because they help the mathematician in proving the original hypothesis.

It is not clear that in trying to prove an hypothesis necessarily needs new heuristics. For example, the author's MU (Maths Understander) (Furse (1990)) program has solved a number of simple problems in group theory using only the heuristics:
1. Expand a definition
2. Break compound statements into parts
3. Simplify expressions

Thus even with a fixed set of heuristics, a program that was capable of generating new hypotheses from an original hypothesis would be able to solve some, if not all, of the new hypotheses. However, work in analysing texts in classical analysis (See Furse...), shows that many proofs use specialised heuristics, often specific to a particular proof. These heuristics may have been discovered by the original discoverer of the proof by discovery mechanisms, but the bulk of mathematicians will have learned these specialised heuristics by reading the completed proof. Furthermore the original discovery of the new heuristics is likely to have taken place within an environment whereby the search space is considerably reduced, due to the examination of several related proofs, and the combining of existing techniques. Many analyses of creativity, eg PoincarÚ, Hadamard, have stressed the importance of the contiguity of ideas in the discovery process, but some, if not all, of these ideas have come from the external environment, eg other people, rather than from within the creator's head. Lenat is probably right that there has to be some discovery of new heuristics by discovery mechanisms, but even in this process it is often driven by ideas from other mathematicians. Nevertheless, most new heuristics are learned from other mathematicians by reading their proofs.

Because AM runs on only one machine and there is no mechanism of communication, AM inherently models the solitary mathematician, whereas as we have argued, mathematics is a social activity. AM therefore runs out of steam because it spends time investigating concepts which other mathematicians could have told it have no future, and more importantly does not pursue concepts which other mathematicians think are interesting. One could argue that if other mathematicians find a concept interesting, then surely AM will also. However imagine a network of computer programs each running the AM program, with the extension that when any of them finds a concept of sufficiently high interestingness, it sends a message to some of the other machines about it. Over time, subnetworks would concentrate on sub-areas of mathematics, and inevitably due to this specialisation more progress would be made. Furthermore, when one machine got bogged down, a new concept or conjecture would come in over the network which would introduce new concepts to be investigated. The mathematical culture has sub-disciplines, and it would be unrealistic to expect a human mathematician to work in all sub-disciplines, unless he or she was a modern Gauss. The same will have to apply to machine mathematicians.

AM is severely limited by not being able to transmit or receive proofs for a number of reasons, some of which have been touched on above. Of fundamental importance is the role of previous mathematical results in solving problems. Polya, Carbonell, Anderson and many others have stressed the importance of using knowledge of the solutions of previous problems in solving new problems. For human mathematicians, nearly all these solutions come from other mathematicians, rather than from the person themselves. Of course there are hypotheses that have been discovered by the human mathematician that are used in subsequent proofs, but the basic groundwork and methods have nearly always come from the mathematical culture.

Thus the ability to receive new proofs both provides new mathematical results and new problem solving methods, both of which may be needed in solving problems. It clearly takes much longer to discover a proof than to explain it to someone else, and by being able to transmit proofs from one mathematician to another, progress is much faster. However, it is common experience that the published proof does not resemble the original proof. The author has simplified and tidied it up, sometimes to the extent that the original heuristics used to discover the proof are unrecognisable. There are many examples of mathematics proofs which produce an expression out of a hat , but no indication is given why this particular expression has been chosen. It was chosen because the original creator tried a series of expressions, and eventually found the one that worked. But this trial and error process, and what led up to it, is not communicated to the mathematical community.

Suppose now we consider a revised AM program, that can investigate concepts and hypotheses much in the way AM works, but with the following qualifications:
1. The program tries some of the time to prove hypotheses.
2. The program runs on a network of computers, and sends messages to some other machines of hypotheses it has proved, or concepts it finds interesting. It could for example send the information to machines which expressed an interest in them, or all results could be published on a bulletin board for other machines to browse.
3. The program reads proofs of interest to it, learns the new mathematical results, discovers in what situations they are applicable, and learns the new problem solving heuristics used in the proofs.

My hypothesis is that such a community of machine mathematicians would run out of steam much later than the original AM, and might possibly never run out of steam. Nevertheless the development of such a program is likely to be some time in the future, since the theorem proving capability would have to be capable of generating new hypotheses in any branch of mathematics it was pursuing, a task beyond current theorem proving methods.

References
Carbonnell J.G, (1983), Learning by Analogy: Formulating and Generalizing Plans from Past Experience, in: Michalski R.S., Carbonnell J.G. and Mitchell T.M. (eds.), Machine Learning, an Artificial Intelligence Approach, Tioga.

Furse E, (1990), The Mathematics Understander, submitted to The International Conference on AI and Mathematics.

Hadamard, J., (1945), Psychology of Invention in the Mathematical Field, Dover Publications.

Kuhn T.S., (1962), The Structure of Scientific Revolutions, Univ. of Chicago Press.

Lenat, D.B., (1976), AM: An artificial intelligence approach to discovery in mathematics as heuristic search, Ph.D. Thesis, AIM-286, STAN-CS-76-570, and Heuristic Programming Project Report HPP-76-8, Stanford University, AI Lab., Stanford, CA.

Polya G., (1957), How to solve it, Doubleday.

Ritchie G.D. and Hanna F.K., (1984), AM: A Case Study in AI Methodology, Artificial Intelligence Vol. 23, No. 3, pp 249-268.

Furse E., (1990), A formal expression language for mathematics, Technical Report no. CS-90-2, Department of Computer Studies, University of Glamorgan.

List of Papers