ch3.txt

Chapter 3
Autonomous Agents
Some wellspring of creation, lithe in the scattered sunlight of an early planet, whispered something to the gods, who whispered back, and the mystery came alive. Agency was spawned. With it, the nature of the universe changed, for some new union of matter, energy, information, and something more could reach out and manipulate the world on its own behalf. Selfish? Yes. But how does matter, energy, information, and something else miraculous become selfish? From that miracle grew a biosphere = and, we must surmise, from that grow other biospheres, scattered seeds and gardens across the cosmos.
Pregnant in the birth of the universe was the birth of life. Agency may be coextensive with life. Life certainly burgeons nowhere without agency. We all act on our own behalf. In the Kantian form: What must something be such that it can act on its own behalf?
I will hazard again my tentative answer, baldly now, then return to it: An autonomous agent must be an autocatalytic system able to reproduce and able to perform one or more thermodynamic work cycles.
The thrashing E. coli swimming upstream in a glucose gradient is an autocatalytic system able to reproduce. In its swimming, it is carrying out one or more thermodynamic work cycles. Can I deduce this tentative definition from some more primary categories? I do not know how. At the beginning, I jumped = more realistically, I struggled = for weeks searching for a constellation of properties that seemed promising, in order to articulate something I sensed but could not yet say. It has taken four more years to understand more of what I still can only partially say in these investigations.
Definitional Jumps and Circles
What of definitional jumps in science? 
So a pause to look at such jumps, to new clusters of concepts that, whole cloth, change how we carve up the world. Consider Isaac Newton and his famous F = MA, force is equal to mass times acceleration. It all seems sensible. If a log lies on an iced pond and I push it, it accelerates. If I push harder, it accelerates faster. If there are two logs, one on top of the other, and I push, the two accelerate less rapidly than if there were one log.
“I make no hypotheses,” stated Newton. Yet Poincaré, two centuries later, argued that Newton’s brilliant move was definitional. The three terms, F, M, and A = force, mass, and acceleration = argued Poincaré, admit no independent definitions. Acceleration is independently definable by the metric concepts of distance, motion, time, the rate of change of position with time = velocity, and the rate change of velocity with time = acceleration. But force and mass are, said Poincaré, joined at the hip, codefined, one in terms of the other. Mass is that which resists acceleration by force. Force is that which induces acceleration when applied to a mass. 
The equation F = MA is a definitional circle, according to Poincaré. Many physicists disagree with Poincaré. I tend to agree, but then I am not a physicist, so take care.
On the other hand, the great twentieth-century philosopher Ludwig Witt-gen-stein said something similar in his majestic Philosophical Investigations. Witt-genstein came to his revolutionary investigations painfully, ultimately rejecting his own earlier triumph in the Tractatus. Indeed, part of why I have so blatantly borrowed Wittgenstein’s title, without my presumption to similar intellectual stature, is that there is a parallel between Wittgenstein’s abandonment of the Tractatus and growing awareness of knowing as living a language game, and a central theme of this Investigations, that there may be a limit to the way Newton taught us to do science and a need to reformulate what we do when we and other agents get on with living a life. As we will see, in both cases, it appears that something profound is going on in the universe that is not finitely prestatable. Life and language games seem persistently open to radical innovations that cannot be deduced from previous categories and concepts. I will only be able to discuss these issues more fully after we have encountered autonomous agents and their unfolding mysteries.
In his early Tractatus, Wittgenstein had brought to conclusion the mandate of logical atomism from Russell. Logical atomism sought a firm epistemological foundation for all knowledge in terms of privileged “atomic statements” about “sense data.” The idea, flowing from Hume to Berkeley to Kant then back to British empiricism, was the following: One might be mistaken in saying that a chair is in the room, but one could hardly be mistaken in reporting bits and pieces of one’s own awareness. For example, “I am having the experience of a brown color”; “I am aware of the note A-flat.” These statements were thought to record sense data and to be less susceptible to error than statements about the “external world” of chairs, rocks, and legal systems. Logical atomism sought to reconstruct statements about the external world from logical combinations of atomic statements about sense data. The Tractatus sought to bring that program to fruition   .   .   .    and nearly succeeded.
It was Wittgenstein himself who, twenty years later, junked the entire enterprise. Philosophical Investigations was his later-life revolution. His revolution has done much to destroy the concept of a privileged level of description and paved the way to an understanding that concepts at any level typically are formed in codefi-nitional circles. Newton was just ahead of his time.
You see, even in the case of the physical chair, the idea that we can reconstruct statements about chairs as equivalent to combinations of statements about sense data fails. The failure is deep and will have unfolding resonances with the mysteries of autonomous agents. The program to replace statements about physical objects with sets of statements about sense data requires “logical equivalence.” Logical equivalence means that wherever a statement about a chair occurs at a “higher level” of description, it must be possible to specify a list of statements about sense data whose truth are jointly necessary and suYcient for the statement about the chair to be true. For example, the statement, “There is a Windsor rocker in the living room,” must be implied by, “I seem to be seeing a brown surface with a given shape,” “I feel a hard surface,” and also, “When I push the top of the shape, I observe an oscillatory motion of the brown surface.”
The trouble is, it appears to be impossible to finitely prespecify a set of statements about sense data whose truth is logically equivalent to statements about chairs. What if the observer were colorblind, or blind, or deaf, or not in the room but might enter the room wearing hobnailed boots, fall on the chair, and break it? Could we finitely specify the list, “If I were to enter the room, I would have the following sense data; and if one who is red-green colorblind were to enter the room he or she would have the following sense data; and if the rocker were to be on a soft rug, the sounds that would be heard by one who is A-flat tone deaf would   .   .   .   . To achieve this end, we would have to be able to finitely prespecify something about a set of statements concerning atomic sense data statements whose truth would be necessary and suYcient to the truth of a statement about the Windsor chair in the living room.
The problem, briefly, is that there appears to be no finitely prestatable set of conditions about sense data statements whose truth is logically equivalent to any statement about a real physical chair in a living room.
Wittgenstein invented the concept of a “language game,” a codefined cluster of concepts that carve up the world in some new way. Consider, he said, legal language, and try translating it to ordinary statements about human agents without using legal concepts. So consider, “The jury found Henderson guilty of murder.” We understand this statement but do so in the context of law, evidence, legal responsibility, trials, guilt and innocence, jury systems, bribing jury members, appeal processes, and so forth. Now try to translate the statement into a set of statements about ordinary human actions, “A group of twelve people were seated behind a wooden enclosure for several days. People talked about someone having died of poison. One day, the twelve people left the room and went to another room and talked about what had happened. Then the twelve people came back and one man stood up and uttered the words, ‘We find Henderson guilty of murder.’”
There is no finitely prespecifiable list of statements about ordinary human actions whose truth would be necessary and suYcient for the truth of the statement that the jury found Henderson guilty of murder.
Another example concerns the United Kingdom going to war with Germany. There is no finitely prespecifiable set of ways this must happen. Thus, the queen, one day after tea, might betake herself to Parliament and cry, “We are at war with Germany!” Or the prime minister might in Parliament say, “Members, by the authority of my oYce I declare that we are at war with Germany!” Or ten thousand irate football fans from Leeds might shoulder picks, take to barges, funnel in fury up the Rhine, and attack football fans in Koblenz. Or   .   .   .   . In short, there is no prestatable set of necessary and suYcient conditions about the actions of individual humans for the United Kingdom to manage to go to war with Germany. (And may neither nation again find a way to make war on the other in the future.)
Wittgenstein’s point is that one cannot, in general, reduce statements at a higher level to a finitely specified set of necessary and suYcient statements at a lower level. Instead, the concepts at the higher level are codefined. We understand “guilty of murder” within the legal language game and thus in terms of a codefining cluster of concepts concerning the features noted above = law, legal responsibility, evidence, trial, jury.
Useful new concepts arise in codefining clusters. It is fine that F = MA is circular, Poincaré might say, for the codefining circle is at the center of a wider web of concepts, many of which have reference to the world; hence, the web of concepts touches, articulates, discriminates, and categorizes the world.
So too did Darwin jump to a new definitional circle with his concept of natural selection, heritable variation, and fitness. Many have worried that natural selection and fitness, which is defined as an increased expected number of oVspring, are a definitional circle. Like F = MA, the definitional circle is virtuous.
So too I jump to the tentative definition, “An autonomous agent is a system able to reproduce itself and carry out one or more thermodynamic work cycles.” Actually, at this stage my own tentative definition is not circular because “reproduce itself” and “work cycle” are definable independently. But as we delve further into the concept of an autonomous agent in succeeding chapters, cyclic definitions arise concerning “work,” “propagating work,” “constraints,” “propagating organization,” “task.” I hope that the circle is virtuous and brings us toward a new understanding of “organization” itself.
In short, unpacking this definition will lead us into odd territory. Part of the oddness is the question of just what is the proper mathematical form to describe an autonomous agent. Is it a number, hence, a scalar? A list of numbers, hence, a vector? A tensor? I think not. An autonomous agent is a relational concept. In a later chapter I will suggest a spare mathematical form for an agent derived from category theory. This attempt is better than nothing, but I am not persuaded that eVort is satisfactory, for the mathematical mappings of category theory are themselves finitely prestatable and a biosphere, I both fear and celebrate, is not. Somethings new are needed.
Autonomous Agents
Well, let’s have at it. I begin with the cornerstone of thermodynamics, the Carnot cycle.
Sadi Carnot was a French engineer writing in the 1830s. The French had recently lost the Napoleonic Wars, Waterloo had come and gone, a certain famous statue to Wellington had been erected in London. Carnot, like others at the time, realized that part of the success achieved by the English had to do with early industrial economic power. The Brits had large numbers of steam pumps that were able to clear water from coal and iron mines, allowing miners to work under bad but relatively dry conditions and extract coal and iron more eVectively than could their French counterparts. The coal and iron in turn were worked into the machinery of the Industrial Revolution and cannons and also, notably, more steam pumps that could keep the mines in working condition. I find it fascinating that the British system was an autocatalytic technology, iron and coal fueled the manufacture of pumps that abetted the mining of iron and coal.
Carnot set about understanding the fundamentals of the extraction of mechanical work from thermal energy sources. The result of his eVort was an analysis of an idealized device to extract mechanical work from heat, the Carnot cycle. Carnot titled his work “An Investigation on the Motive Force of Heat.”
A second point of fascination is that Carnot carried out his crucial analysis just as steam was becoming established as a source of mechanical work. It is almost certainly not a coincidence that the impulse to investigate the central nature of autonomous agents arises just as we are on the threshold of creating such molecular systems. Science, technology, and art tumble into the adjacent possible in roughly equal and yoked pace.
Figure 3.1 shows the essentials of Carnot’s idealized machine. There are two heat reservoirs, one hotter than the other, T1 > T2. Between the two heat reservoirs is a cylinder with a piston inside. The space between the top of the piston and the head of the cylinder is filled with an idealized “working gas,” which can be compressed and can expand. As a real, and certainly an idealized, gas is compressed, the gas becomes hotter; as the gas expands, it becomes cooler.
I will modify the Carnot machine in one central way, which does not alter but rather makes explicit an important feature of the actual working of the Carnot machine. I will attach a red handle to the cylinder, as shown in Figure 3.1. The Carnot cycle begins with the piston pressed up near the top of the cylinder, the working gas is compressed and hot, indeed, it is as hot as the high temperature T1. You will operate the machine. You pull on the red handle, sliding the cylinder (frictionlessly) into contact with the high-temperature reservoir, T1. You then let go of the red handle and allow the working gas in the cylinder to expand, thereby pushing the piston downward away from the head of the cylinder, in the first part of the power stroke of the Carnot engine.
As the power stroke initiates, the working gas expands and starts to cool. However, thanks to the fact that the cylinder is now in contact with the hot thermal reservoir, heat flows into the cylinder from T1 and maintains the working gas almost at the constant temperature T1. Indeed, if the Carnot engine is operated slowly enough, the temperature remains constant. Such slow operation is said to be reversible. If the engine is operated more rapidly, hence irreversibly, the temperature is held nearly constant during this part of the power stroke, thus this section of the power stroke is called the “isothermal expansion portion” of the Carnot work cycle.
It is not only convenient but central to understanding a Carnot engine to plot the state of the system in a Cartesian coordinate system in which the x-axis corresponds to the volume of the working gas, while the y-axis corresponds to the pressure of the gas (Figure 3.2). The work cycle began with the piston near the top of the cylinder, the working gas hot and compressed. As the isothermal expansion phase of the power stroke happens, the pressure falls slightly, while the volume increases appreciably. The corresponding segment of the work cycle connects the starting position in Figure 3.2, position 1, to position 2 by a line segment representing the simultaneous values of volume and pressure during the isothermal expansion part of the power stroke.
You initiate the second part of the power stroke by pushing on the handle and moving the cylinder out of contact with the hot T1 reservoir to a position between the two heat reservoirs, touching neither. You immediately let go of the handle. The working gas continues to expand, pushing downward on the piston and moving it away from the head of the cylinder. However, because the cylinder is out of contact with T1 and the working gas is expanding, the working gas gets noticeably cooler, so pressure drops appreciably while volume increases slightly. This part of the power stroke is called “adiabatic expansion.” In Figure 3.2, the adiabatic expansion step carries the system from step 2 to step 3, the end of the power stroke, a point where the pressure is at the lowest point, and the volume of the working gas is at the highest point of the work cycle.
Now in order to return the Carnot engine to its initial state, 1 (in Figure 3.2), such that the working gas can again expand and do mechanical work on the piston, work must be done on the engine to bring the piston back up to its position near the head of the cylinder and to recompress and reheat the working gas so that its temperature and pressure values, or state, correspond to state 1. If the work done on the Carnot engine simply retraced the exact pathway from step 3 to step 2 to step 1, at least as much work would have to be done on the Carnot engine as the engine released in the power stroke in going from 1 to 2 to 3. If so, no net work would be carried out by the Carnot engine on the external world.
Instead of retracing the power stroke pathway, the Carnot engine, and all heat engines, use a simple trick. You will do it. At step 3, the end of the power stroke, you push on the handle, pushing the cylinder into contact with the low-temperature reservoir, T2. Indeed, you have arranged things such that at the end of the power stroke the working gas is itself at this lower temperature, T2.
With the cylinder in contact with T2, you now run around to the base of the cylinder, where a stout handle is attached to the piston and extends beyond the cylinder’s base. You push on the handle, pushing the piston upward in the cylinder, thereby compressing the working gas. As you do this work on the piston, the compressing gas tends to heat up. But thanks to contact with the low-temperature reservoir, the heat generated by compression in the working gas diVuses into the cool T2 reservoir, holding the working gas only slightly warmer than T2. Thus volume decreases appreciably, while pressure increases slightly.
The point of this trick is that it requires less work to compress a gas that remains cool than one that heats up. Because the working gas is held at nearly a constant temperature T2, this part of the compression stroke is called “isothermal compression,” and it carries the system in its volume-pressure state space from position 3 to position 4 in Figure 3.2.
At the end of the isothermal compression stroke, you do your part again. You pull on the handle, pulling the cylinder out of contact with the cool T2 reservoir to a position between T2 and T1, touching neither. Then you return to push on the handle connected to the piston, further compressing the working gas. Due to the compression of the working gas and the fact that it is not in contact with the cool T1 reservoir, the working gas heats up, so the pressure increases appreciably while the volume decreases slightly, as the gas is compressed until the initial state of hot compressed gas, step 1, is achieved. The work cycle is now complete. The connected set of four lines in Figure 3.2, from step 1 to 2 to 3 to 4 to 1, depicts the sequence of pressure volume states of the working gas around the work cycle.
I have emphasized the role of you and the red handle in carrying us through the work cycle. In a real engine, of course, the role of the red handle is carried out by various gears, rods, escapements, and other mechanical devices that serve an essential role: The handle and you, or the gears, rods, and escapements, literally organize the flow of the recurrent process. I will return to this organization of the flow of process in a machine or an autonomous agent. The Carnot cycle is involved with the organized release of thermal energy in achieving recurrent mechanical work. The organization of work is essential = and will be central = to thinking about what occurs in an autonomous agent. Indeed, part of what we need is a way of characterizing the organization of real processes in the nonequilibrium world. I do not think we have, as yet, an adequate concept of organization.
The first question I want to raise about the Carnot cycle is this: Why is it a “cycle”? There is a clear sensible answer. The Carnot cycle operates in a cycle, as does a steam engine, gasoline engine, or electric motor, precisely because at the completion of one cycle the total system is returned to the same initial state as at the start of the cycle. Because of this, the organization of the process, achieved by the actual gears, rods, and escapements that coordinate the relative motion of the parts of the engine, has returned to the initial configuration from which, with no further ado, the system can again perform a work cycle. Thus in the case of the Carnot engine, the system performs mechanical work on the piston in a work cycle that, in net, transfers heat from the hot reservoir to the cold reservoir. At the end of the cycle, the Carnot engine returns to the initial state with the piston raised and filled with hot compressed gas at the temperature of the hot reservoir. The Carnot engine is all set as a total organization to receive another input of heat energy and perform another work cycle.
Suppose that there were no cycle? For example, a cannon fires a cannonball that hits a standing steel beam that is knocked over and hits a lever arm tossing a ball on its opposite end into the air which then falls to the ground and stops. To get the contraption to do it again would require that somehow the cannon be reloaded, the steel beam be reset on end, the lever arm to be reset with a ball on its other end. Where is all this organization of processes and events to come from? I will return to these issues in the next chapter, but notice for now that the cyclic organization of processes in the Carnot, the steam, the gas, and the electric engines achieve the requisite organization precisely because the system works as a cyclic process. 
A second issue concerns a well-known feature of the Carnot engine that is very much worth comment here. If the sequence of steps is run in the reverse directions, so that the engine is run from state 1 to 4 to 3 to 2 to 1, the Carnot engine is not performing as a pump at all. It is, instead, performing as a refrigerator. Run in the reverse direction, the Carnot engine uses mechanical work to pump heat from the cool reservoir, T2, to the hot reservoir, T1, making T2 cooler.
Now, heat does not spontaneously flow from cooler to hotter objects, making the cooler objects even cooler. So two points leap to attention here. First, the same device, the Carnot engine, can be a pump or be a refrigerator, depending upon the order of the operations. In fact, I am cheating a bit, for the organization of gears and escapements would diVer in achieving the sequence of states 1, 2, 3, 4, 1 versus 1, 4, 3, 2, 1. But essentially the same machine can perform two very diVerent functions, or tasks, pumping in one case, cooling in the other.
The third point, of course, concerns spontaneous processes and nonspontaneous processes. It took more than fifty years from Carnot’s work to really begin to understand thermodynamics = the first law, conservation of energy; the famous and mysterious second law and its handmaiden, entropy; the third law concerning a zero temperature at which molecular motions stop = and for the invention of statistical mechanics to connect thermodynamics and Newtonian mechanics.
Some processes occur spontaneously, some conceivable processes do not. One of the most obvious cases is that if a hot gas is in contact with a cold gas, the two will in due course come to be the same temperature. Heat spontaneously “diVuses” from the hot to the cold object, cooling the former, warming the latter. In statistical mechanics we think of “hot” as corresponding to atoms moving rapidly, hence, with high kinetic energy. When these high-kinetic-energy atoms interact with slower-moving sets of atoms, the collisions transfer kinetic energy to the slower-moving atoms, speeding them up and slowing down the faster-moving atoms. Eventually, the atoms in the two sets come to have the same statistical distribution of motions, hence kinetic energy and hence temperature, where “temperature” is now understood to correspond to the average kinetic energy of the particles in the system.
The apparently mysterious second law of thermodynamics states that the entropy of a system is either constant or increases. The modern understanding of entropy can be stated roughly with the help of the concept of a 6N-dimensional phase space. 
Consider an isolated closed thermodynamic system, say, an ideal gas in a thermos bottle. Let there be N gas particles in the bottle. Now, each particle is in motion in real three-dimensional space; therefore, we can pick an arbitrary three-dimensional coordinate system, with length, width, and height (x, y, and z). We can note the current position of any particular particle in the bottle at any moment on each of the three positional coordinates. In addition to having a position, each particle may be in motion, it may have a velocity and an associated momentum in some direction in the bottle. Using Newton’s vectorial composition of forces rules, we can decompose the motion of the real particle into its motions in the x direction, the y direction, and the z direction. The momentum in each of these directions is just the mass of the particle times its velocity in that direction. Newton’s vector composition rule says that we can recover the motion of the initial particle by constructing the obvious parallelogram that adds the x, y, and z velocity or momentum vectors back together.
So for each particle we can represent its position and momentum in three spatial directions by 6 numbers. We have N particles in the bottle, so we can represent the current positions and momenta of all N particles at any instant in time by 6N numbers. DiVerent combinations of positions and velocities now correspond to diVerent sets of 6N numbers. And, as the N particles in the bottle collide and exchange momenta, bouncing oV in new combinations of directions with new combinations of velocities according to Newton’s three laws of motion, the 6N numbers representing the system at each moment change in time through some succession of 6N numbers. If we think of all the possible values of the positions and velocities of the N particles in the bottle, that set of possible values is the 6N-dimensional phase space of our system. The system starts at some single combination of 6N numbers, hence a single state in the phase space. Over time, as positions and momenta change, the 6N numbers change and the system flows along a trajectory in phase space.
Now here is the heart of the second law: Some of the positions in the 6N-dimensional phase space correspond to states in which all the particles are more or less uniformly spread throughout the bottle and are moving with more or less the same velocities. Other positions in the 6N-dimensional phase space correspond to unusual situations, in which all the particles are located near the top of the bottle, are along the walls of the bottle, are moving in the same direction in the bottle, and so forth.
Consider mathematically breaking up the 6N-dimensional phase space into a very large number of tiny 6N-dimensional “cubes” that together add up to the entire 6N-dimensional phase space. If the cubes are small enough, then each cube corresponds to a quite similar set of states. One cube might correspond to all the particles flowing downward in the bottle, another cube might correspond to a near uniform distribution of positions and momenta.
It is easy to see intuitively that there are many more combinations of positions and momenta that correspond to nearly uniform positions of the particles and their motions scattered in all possible directions than there are combinations of positions and momenta that correspond to all the particles being located in a specific small region of the bottle. In order to quantify this intuition, statistical mechanics counts the numbers of small 6N-dimensional cubes corresponding to each such “macroscopic” state of the gas. Vastly many more small cubes correspond to the “macrostate” in which particles are nearly uniformly distributed and are moving in all the possible directions with velocities bunched around an average velocity = hence kinetic energy = than for any other macrostate, such as the macrostate corresponding to all the particles being located near the top of the bottle.
We are almost home free. We need one more premise = the famous “ergodic hypothesis.” This hypothesis asserts that the trajectory of states leading from the initial state in a long stringlike “walk” will, over time, spend as much time in any small 6N cube as in any other cube. In short, the ergodic hypothesis asserts = indeed, assumes = that the system will wander around its phase space such that after a long time it will have spent as large a fraction of its time in any one tiny cube as any other cube.
But now we are home free. Since vastly many more small cubes correspond to the nearly uniform distribution of particles, moving in all possible directions but bunched around an average velocity, it follows from the ergodic hypothesis that the system will have spent most of its time in this “equilibrium” macrostate.
The second law, in its modern understanding, is simply the statement that an isolated thermodynamic system will tend to flow away from improbable macrostates = corresponding to very few of our tiny 6N-dimensional cubes = and flow into and spend most of its time in the equilibrium macrostate for no better reason than that that macrostate corresponds to vastly many small 6N cubes in the entire 6N-dimensional phase space. The increase of entropy in the second law is nothing but the tendency of systems to flow from less probable to more probable macrostates.
The physical concept of entropy of a macrostate is understood, since Ludwig Boltzmann in the last century, to be proportional to the logarithm of the number of small 6N-dimensional cubes that correspond to that macrostate. The increase of entropy in spontaneous processes is then the tendency to flow from macrostates comprised of a small numbers of 6N-dimensional cubes, or “microstates,” to macrostates comprised of a very many microstates.
Our next step in thinking about autonomous agents requires us to consider again the concept of a “catalytic task space” and the character of autocatalytic sets in the context of catalytic task space. In the first chapter I described the basic framework, due to Alan Perelson and George Oster, of an abstract shape space, where each point would represent a molecular shape. Shape space has at least the three spatial dimensions of length, height, and width, but in addition it has properties reflecting the features of clusters of atoms that contribute to an eVective molecular shape, that is, to those features that collectively might be “recognized” by another molecule that bound the shape in question. Such additional molecular features may include electric charge, dipole moment, hydrophobicity, or other features. At present, it is a reasonable guess that shape space is between five and seven dimensional. If true, this would not mean that there are five familiar features, three spatial and two others, that constitute the dimensions of shape space. Rather, some odd combination of several physical properties = partially charge, partially dipole moment, or partially hydrophobicity and partially dipole moment = might be the dimensions that matter.
In any case, we are to consider a bounded shape space with maximum and minimum values for each axis. A shape is a point in shape space. Thus, a molecular feature on a virus antigen, an epitope, is a point in shape space. An antibody might bind that epitope and a family of similar shapes filling a ball in shape space. As remarked above, very diVerent molecules can have eVectively the same shape, so endorphin and morphine both bind the same endorphin receptor. A finite number of balls will “cover” shape space, and the immune repertoire of perhaps a hundred million antibodies may well cover shape space.
Catalytic task space, you recall, simply applied the concept of shape space to catalysis. A point in catalytic task space now represents a catalytic task. A given chemical reaction constitutes a catalytic task. As in shape space, similar reactions constitute similar catalytic tasks. As in shape space, diVerent reactions can constitute essentially the same catalytic task. An enzyme covers some ball in catalytic task space, comprising the set of reactions it can catalyze. And as noted before, according to transition state theory a catalytic task corresponds to a catalyst binding the distorted, hence high-energy, molecular configuration corresponding to the transition state of a reaction with high aYnity and binding the substrate and product states with, in general, lower aYnity.
In terms of catalytic task space, what is a collectively autocatalytic set? Consider a simple case. Two peptides, A and B, form a collectively autocatalytic set if A catalyzes the formation of B from two of B’s fragments, while B catalyzes A from two of A’s fragments. Then consider two balls in catalytic task space, the first ball, covered by A, constitutes the catalytic task in which two fragments of B are ligated to form B. The second ball, covered by B, constitutes the catalytic task in task space in which two fragments of A are ligated to form A.
The first feature of a collectively autocatalytic set is what I call “catalytic closure.” Every reaction that must find a catalyst, does find a catalyst. The formation of A requires B, and the formation of B requires A. It is important to notice that this closure in catalytic task space is not “local”; there is no single reaction in this collectively autocatalytic set that by itself constitutes the closure in question. In a clear sense, the catalytic closure is a property of the whole system.
A second feature to notice is that A and B as catalysts do not by themselves constitute the closure in question; A and B might catalyze a variety of reactions. In particular, if B is presented with the two “proper” fragments of A, call them A’ and A”, then B will ligate A’ and A’’ to form A. But if B were presented with other substrates, say Q and R, then B might catalyze a reaction transforming Q and R into two other molecular species, S and T. Similarly, A, as a catalyst, will ligate two proper fragments of B, B’ and B’’ to form B. But A, if confronted with two other substrates, say, F and G, might catalyze their ligation to form a single third molecule, H.
While the set A, B, A’, A”, B’, B’’ is collectively autocatalytic, forming more A and B from a substrate pool of A’, A”, B’, and B”, it is not the case that the set A, B, Q, R, F, and G is collectively autocatalytic, for the products of the reactions catalyzed by A and B, namely S, T, and H, are not themselves the catalysts A and B.
In short, the closure of catalytic tasks requires specification of the catalytic tasks themselves plus the specific substrates whose products, here A and B, constitute the very catalysts that carry out the catalytic tasks in question.
The closure of an autocatalytic set and set of catalytic tasks has a kind of dualism. From the point of view of the molecules involved, the specific catalytic tasks constitute the avenues of release of chemical energy by which the molecular system reproduces itself. The tasks coordinate the flow of atoms among the molecules whereby the set reforms itself. From the point of view of the tasks, the molecular species manage to carry out the tasks repeatedly, with no further molecular species being necessary to carry out the tasks. The molecules carry out the tasks, the tasks coordinate, or organize, the processes among the molecules.
The coordination aVorded by the catalytic tasks that are jointly present and fulfilled is highlighted if we recall that, in general, two molecular species, say, A’ and A”, might undergo a variety of diVerent reactions that form, in addition to A, perhaps E, L, M, P, and other molecular species. The specific catalytic task that carries A’ and A’’ to A in the presence of a catalyst, B, speeds that specific reaction in comparison to the alternative reactions forming E, L, M, and P. Thus, the closure of the catalytic tasks, substrates and catalysts, A, B, A’, A”, B’, B”, achieves a coordination, or organization, of the flow of matter and energy into the autocatalytic system.
The organization achieved by the closure of catalytic tasks is similar to the organization achieved by the gears and escapements together with the rest of the idealized Carnot engine. The flow of process is marshaled into an organized whole. In the case of the autocatalytic set, the set reproduces itself. It also seems worth stressing that this closure in catalytic task space is a new concept with real physical meaning. It is a matter of objective fact whether or not a physical reaction system achieves catalytic closure; the hypothetical AB system above, and any free-living cell, achieves catalytic closure.
A final preliminary will bring us to our attempted definition of an autonomous agent. This preliminary is based on the distinction, noted above, between spontaneous and nonspontaneous chemical reactions. At equilibrium, the net rate of formation or destruction of each molecular species is zero, aside from small fluctuations that damp out. Thus, if two molecular species, X and Y, interconvert, the equilibrium is attained at that ratio of X and Y concentrations at which Y converts to X as fast as X converts to Y. If the reaction is displaced in one direction, say there is a higher X concentration than the equilibrium ratio, then the spontaneous, or exergonic, reaction proceeds in the direction toward equilibrium that reduces the excess of X concentration (Figure 3.3).
All spontaneous chemical reactions, if coupled to no other source of energy, are exergonic. On the other hand, if some other free energy source is coupled to the reaction, the reaction can be driven “beyond equilibrium” by using some of the energy source. Reactions that are driven beyond equilibrium by addition of free energy are called endergonic. Thus X might convert to Y, and this reaction might be coupled to another source of free energy, such that the steady state concentration of Y is much higher than the normal equilibrium X:Y ratio (Figure 3.3).
In the Carnot cycle, completion of the work cycle involved the cylinder piston system doing exergonic work on the external world during the power stroke, then the outside world doing work on the cylinder piston system when you pushed on the piston to recompress the working gas. The Carnot cycle links mechanical and thermal energy sources into a work cycle. A chemical reaction network with a work cycle will have to link spontaneous, exergonic and nonspontaneous, endergonic reactions into the chemical analogue of a work cycle. Like the cyclic Carnot engine, the chemical analogue will have to work in a cycle of states, like the 1, 2, 3, 4, 1 cycle of the Carnot cycle. Further, in order for the cycle to operate at a finite rate, hence irreversibly, the autonomous agent must be an open thermodynamic system driven by outside sources of matter or energy = hence “food” = and the continual driving of the system by such “food” holds the system away from equilibrium.
In this light, think again of the Ghadiri autocatalytic system, the 32-amino-acid sequence A that ligates two fragments A’, a 15-amino-acid fragment, and A”, a 17-amino-acid fragment, into A. This reaction is purely exergonic. The reaction proceeds from the substrate fragments A’ and A’’ to form the product molecule A and approaches the equilibrium ratio of substrates to product. Ghadiri’s autocatalytic system is wonderful, but merely exergonic. It does not achieve a work cycle. In general, autocatalytic and collectively autocatalytic systems can be purely exergonic. In any such case, no work cycle is achieved.
Now we can return to my jumped-to definition: An autonomous agent is a reproducing system that carries out at least one thermodynamic work cycle. That bacterium, sculling up the glucose gradient, flagellum flailing in work cycles, is busy as hell doing “it,” reproducing and carrying out one or more work cycles. So too are all free-living cells and organisms. We do, in blunt fact, link spontaneous and nonspontaneous processes in richly webbed pathways of interaction that achieve reproduction and the persistent work cycles by which we act on the world. Beavers do build dams; yet beavers are “just” physical systems.
But Reza Ghadiri’s example of an autocatalytic peptide doesn’t make the grade, nor does Gunter von Kiedrowski’s autocatalytic hexamer DNA or collectively autocatalytic set of two DNA hexamers. All these systems are merely exergonic. No work cycle is performed.
Now that we have stated our proposed definition of an autonomous agent, it is not too hard to imagine a chemical realization. In Figure 3.4 I show a hypothetical molecular autonomous agent. Given visualization of a first case, I expect that we will be constructing molecular autonomous agents within a few years.
Figure 3.4, our first example of a candidate molecular autonomous agent, is “constructed” to link with two further molecular systems, the exergonic auto-catalytic system developed by Gunter von Kiedrowski based on ligation of two DNA trimers by their complementary hexamer. Here, the hexamer is simplified to 3’CCCGGG5’, and the two complementary trimers are 5’GGG3’ + 5’CCC3’. Left to its own devices, this reaction is exergonic and, in the presence of excess trimers compared to the equilibrium ratio of hexamer to trimers, will flow exergonically toward equilibrium by synthesizing the hexamer. Because the hexamer is itself the catalyst for the reaction, the synthesis of hexamer is autocatalytic.
The first additional system consists in pyrophosphate, PP, a high-energy dimer of monophosphate that breaks down to form two monophosphates, P + P. Like any reaction, the reaction converting PP to P + P has an equilibrium, hence an equilibrium ratio of PP and P. In the presence of excess PP compared to equilibrium, the reaction flows toward equilibrium by the spontaneous cleavage of PP to yield P + P.
My purpose in invoking the exergonic conversion of PP to P + P is to utilize the loss of free energy in this exergonic reaction to drive the DNA trimer-hexamer reaction beyond its own equilibrium, leading thereby to an excess synthesis of the 3’CCCGGG5’ hexamer when compared to its equilibrium concentration. Thus, the excess synthesis of the hexamer, which would not occur spontaneously, is driven endergonically by being coupled to the exergonic breakdown of PP to P + P (Figure 3.4). In short, the exergonic breakdown of PP to P + P supplies the free energy to drive the excess buildup of 3’CCCGGG5’ concentration beyond its own equilibrium with respect to its trimer substrates, 5’GGG3’ and 5’CCC3’.
The excess synthesis of 3’CCCGGG5’ constitutes excess reproduction of the hexamer autocatalytic reaction product beyond that which would occur without the coupling to the additional PP free energy source. Thus, the system is reproducing “better” with the coupling to PP than without the coupling.
Another point to note is that the coupling of the breakdown of PP to P + P with the excess synthesis of the DNA hexamer compared to the equilibrium concentration of the DNA hexamer means that energy is stored within the system. This is true because the excess concentration of the hexamer DNA, compared to its equilibrium, could in principle be released by degradation of the hexamer to the two trimer substrates, releasing that stored free energy as this reaction couple flowed toward its own equilibrium ratio of hexamer and trimers. Thus, the coupling to the PP to P + P reaction means that the autonomous agent stores energy internally. Later in evolution, such internally stored energy can be used to drive reactions that correct errors, as in DNA repair in contemporary cells. I am glad to thank Phil Anderson and, indirectly, John Hopfield for this point.
Once the pyrophosphate, PP, is cleaved to form P + P, as this reaction flows toward its own equilibrium ratio of PP to P, that free energy is used up. In order to have a renewed internal supply of the free energy needed to synthesize excess hexamer, it is convenient to resynthesize pyrophosphate from the two monophosphates, P + P. I’ll return below to the meaning of “convenient,” for in a general sense, the convenience reflects the organization of processes that sustains an agent, and that organization is not convenient, it is essential.
Resynthesis of PP from P + P requires the addition of free energy. This is true because we used the exergonic breakdown of PP to P + P to drive the excess synthesis of 3’CCCGGG5’. Now we need to add energy to resynthesize PP from P + P. To do so, I invoke an additional source of free energy in the form of an electron, e, which absorbs a photon, hv; is driven endergonically to an excited state, e*, and falls back exergonically to its low-energy state, e, in a reaction that is coupled to the synthesis of PP from P + P.
The point of this third reaction-couple is clear: PP is resynthesized from P + P so that PP can continue to drive the excess synthesis of the DNA hexamer, 3’CCCGGG5’. Overall, the total system of linked reactions is exergonic = there is an overall loss of free energy that is ultimately supplied by the incoming photon, hv, plus the 2 substrates, 5’GGG3’, and 5’CCC3’. Thus, we are not cheating the second law.
Let’s return to the Carnot cycle, where I had you pushing and pulling on the handle and on the piston itself during the work cycle. We noted that in a real engine you would not be busy pushing and pulling. Your role in organizing the processes would be taken by gears, escapements, rods, connectors, bearings, and other bits of machinery.
I now invoke the analogue of the gears, rods, and connectors in the form of hypothetical molecular couplings that control the reactions I have already invoked. Specifically, I will assume that the hexamer, 3’CCCGGG5’, is the catalyst that couples ligation of the two trimers, 5’GGG3’ + 5’CCC3’, to the exergonic breakdown of PP to P + P. My second assumption is that monophosphate, P, binds to the hexamer and facilitates the reaction. Thus, I am assuming that P is an allosteric enhancer of the reaction. “Allosteric” means that P binds to a site on the enzyme, here the hexamer, other than the hexamer’s own binding site for the substrates. Allosteric enhancers and inhibitors are common in biological systems. 
Here, P might bind to the sugar-phosphate backbone of the DNA hexamer. This coupling implies that as PP breaks down to form P + P, the monophosphate, P, will feed back to further activate the hexamer enzyme, making the catalysis of hexamer formation even more rapid. Just such a positive feedback of a reaction product on the enzyme forming it occurs in the famous glycolytic pathway that is the core of metabolism in your cells. In fact, under appropriate experimental conditions, this positive feedback coupling can cause the glycolytic pathway to undergo sustained temporal oscillations in the concentrations of the glycolytic metabolites.
Finally, I will invoke a few more couplings. I assume that one of the trimers, 5’CCC3’, is the catalyst that couples the exergonic loss of free energy from the activated electron, e* to e, with the resynthesis of PP from P + P. And I invoke an allosteric inhibition of this catalysis by PP itself. Thus, when PP is in high concentration, it tends to inhibit its own resynthesis. But when PP concentration falls, the inhibition on PP resynthesis is removed, and PP is resynthesized. The whole molecular contraption, our first hypothetical autonomous agent, is shown in Figure 3.4.
One of the first things to note about our hypothetical autonomous agent is that it constitutes a previously unstudied class of chemical reaction networks. The behavior of exergonic autocatalytic and cross-catalytic systems is beginning to be studied. The behavior of linked exergonic and endergonic reaction networks is the very stuV of intermediate metabolism and energy’s biochemical transduction, studied for years by biochemists. But, to date, no one has begun to study linked reaction networks in which autocatalysis is coupled to linked exergonic and energonic reactions. So we are entering an entirely new domain.
Thus, our molecular autonomous agent constitutes a system with two essential features of living systems, self-reproduction and metabolism. However, my insistence that an autonomous agent carries out a work cycle refines the generally understood concept of a metabolism to include the requirement that the metabolism carries out a work cycle.
The second feature to note is that our autonomous agent is, necessarily, a nonequilibrium system. Free energy, here in the form of the photon, hv, and the trimer substrates is taken in and used to drive the linked synthesis of PP and excess DNA hexamer. There is no agency at equilibrium. The excess synthesis of DNA hexamer constitutes excess replication of the hexamer by virtue of the coupling of the trimer-hexamer synthesis to the PP  P + P cycle of reactions, which, as noted next, constitute a “chemical engine.”
The third feature to note is the work cycle performed by the agent. The simplest way of seeing the work cycle here is in the behavior of the PP  P + P reaction. In the Carnot cycle, the working gas cycles from compressed and hot to less compressed and cool, back to compressed and hot. In our hypothetical autonomous agent, there is a macroscopic cycle of matter from PP to P + P via the reaction- forming DNA hexamer and back around to PP via the reaction with the high-energy electron. The macroscopic cycling of matter around this cycle is the engine at work. (I am grateful to Peter Wills for this clarification of the concept of a chemical motor.) In addition, depending upon the details of the kinetic constants, our autonomous agent may literally show an oscillatory concentration cycle in which PP concentration begins high and falls as P + P is formed, then the high PP concentration is reformed by use of the photon-energized exergonic e*  e reaction. 
Thus, the PP P + P reaction embedded in the autonomous agent constitutes a chemical engine in which there is a macroscopic net flux of matter around the PP  P + P cycle, which is operating displaced from equilibrium as it is driven by addition of energy from the photon, hv, and addition of the two DNA trimers, and as energy is drained oV to drive excess synthesis of the DNA hexamer.
The fourth thing to note about the autonomous agent is that, like the Carnot engine, the steam engine, the gas engine, and the electric engine, the autonomous agent works in a cycle. At the end of the cycle the system is poised to cycle again. A repeating organization of process is achieved. And next, just as the Carnot engine run backward is a refrigerator and not a pump, if the reactions of the autonomous agent were run backward the PP  P + P engine would run in the reverse direction. This is because all reaction couples would be displaced from equilibrium the opposite way and the analogue of throwing the gears in reverse, namely reversing in sign the positive and negative activator and inhibitor couplings to the two proper enzymes, would convert the excess energy stored in the above equilibrium concentration of hexamer into production of the two trimers and the resynthesis of PP from P + P. Were the release of the photon, hv, a readily reversible step, the excess of PP would drive emission of a photon by the excited electron, thus returning the electron to the initial unexcited state.
In short, if the autonomous agent is run backward, the autonomous agent melts down into its foodstuV. Run backward, the system is not an autonomous agent, for it does not reproduce itself and perform a work cycle. Run backward, the system is a flashlight!
Does the autonomous agent work? The answer is yes. My colleagues Andrew Daley, Andrew Girvin, Peter Wills, and Daniel Yamins and I have simulated the system of diVerential equations that correspond to the dynamics of this autonomous-agent molecular reaction network. The diVerential equations represent the way the concentration of each chemical species in the autonomous agent changes over time as a function of its own and other chemical concentrations. In general in such mathematical models, a number of unchanging constants representing kinetic constants and other parameters enter into the diVerential equations. In the present case, the diVerential equation system has thirteen such parameters. 
The model autonomous agent system is displaced from equilibrium by the persistent addition of the two DNA trimers, 5’GGG3’ and 5’CCC3’, the removal of the DNA hexamer, and the persistent shining of photons, hv, from the outside. The chemical reaction network occurs under “chemostat” conditions. This means that all molecular constituents of the system are treated mathematically as if they were in a real well-stirred container to which the trimers and photons are added at a constant rate. In addition, the hexamer molecular components are removed from the system at an adjustable rate that holds their internal concentrations constant whatever the rate of reproduction of hexamer may be.
The autonomous agent system reproduces more eYciently with the couplings of the DNA trimer-hexamer system to the PP and electron cycles than in the purely exergonic case in which the DNA trimer-hexamer system operates alone. We measured eYciency thermodynamically as the conversion of available free energy coming into the system from the photon source into the excess hexamer with respect to the undriven steady-state rate of reaction concentration of the hexamer.
Figure 3.5 shows the results of our simulations. In these simulations of the chemical reaction network, there are, as noted, thirteen kinetic constants. We carried out computer selection experiments not only comparing the autonomous agent to a nude exergonic DNA trimer-hexamer system, but also computationally mutating the kinetic constants by small amounts and computationally evolving autonomous agents to reproduce with higher thermodynamic eYciency.
Our results demonstrate first and most important that autonomous agents operating displaced from equilibrium and utilizing a work cycle can be more eYcient at using the available free energy coming into the total system in reproducing hexamer DNA than in the absence of the coupling of the trimer-hexamer DNA system to the PP and electron-photon work cycle system. Thus, the autonomous agent as a whole, including its work cycle, reproduces DNA hexamer more rapidly than would the trimer-hexamer exergonic system alone. In short, and also important, being an autonomous agent coupling an autocatalytic system with a work cycle is of selective advantage compared to being a merely exergonic autocatalytic system.
Second, just as in the glycolytic positive-feedback case, our autonomous agent model, for appropriate values of the kinetic constants, can undergo sustained temporal oscillations of PP and other concentrations. The oscillation of PP from high concentration to low concentration then back to high concentration during the work cycle is analogous to the expansion and recompression oscillation of the working gas in the Carnot engine’s work cycle.
Third, a mountainous fitness landscape exists in the mathematical parameter space of the thirteen kinetic constants, in which some values of the kinetic constants lead to higher eYciency of reproduction than others. Darwin’s natural selection could, in principle, operate if there were heritable variation in the kinetic constants.
The main conclusion to draw from our simulation is that autonomous agents coupling one or more autocatalytic and work cycles are a perfectly reasonable, if novel, form of nonequilibrium, open chemical reaction network. There is no hocus pocus here. In the near future we will almost certainly construct such autonomous-agent molecular reaction networks and study their dynamics and evolutionary behavior. A general biology is, in fact, around the corner.
The hypothetical molecular autonomous agent that we have considered has been discussed, for simplicity, as if the problem of retaining the reactants in a confined region of space could be ignored. In fact, this assumption is an idealization. Were our autonomous agent in a dilute solution, the rates of reaction would be very slow. Actual creation of a functioning molecular autonomous agent will require that the reacting molecular species be confined to a small volume or a surface or in some other fashion.
Candidates for isolation to small volumes include micelles and liposomes. Both macromolecular aggregated structures are comprised of “amphipathic molecules,” that is, molecules with hydrophobic and hydrophilic regions such as lipids. Micelles are single-layered structures which, in an aqueous medium have hydrophilic regions directed outward, but are able to enclose an aqueous core in which the other molecular species of an autonomous agent might reside. In an aqueous medium, liposomes form double-layered membranes, homologous to cell membranes, with hydrophilic heads in the aqueous medium and hydrophobic tails abutted. In an aqueous medium, both micelles and liposomes can form and even reproduce by budding. A full-fledged molecular autonomous agent would have to synthesize the lipid or similar molecular constituents of its bounding surface and coordinate budding with dispersion of autocatalytic and work cycle partners to daughter vesicles.
An alternative to isolation of the autocatalytic and work cycle molecular species within a bounding volume is the confinement of such reacting species to a surface. Such confinement has the further advantage of altering diVusive search by reactants from a three- to a two-dimensional search process. The latter can shift the corresponding chemical equilibrium toward synthesis of larger polymers from their smaller substates. Here one can imagine confinement of reactants and products to clay surfaces or confinement of complex organic reactants and products to the surfaces of the abundant dust particles in giant molecular clouds in galaxies.
I will have much more to say in subsequent chapters about the properties of molecular autonomous agents. In particular, in order to understand agents we will have to carry out a critique of the physicist’s concept of “work,” as in a work cycle, for the best understanding of “work” appears to be that work is the constrained release of energy. Yet the very constraints on the release of energy that are essential to the doing of work themselves constitute the analogues of the gears, rods, connectors, and escapements of an ordinary machine. Most important, it typically takes work itself to construct the constraints on the release of energy that then constitutes work. In our first example of an autonomous agent, Figure 3.4, these constraints are present in the invoked couplings of catalysts and allosteric eVectors to the reactions of which the autonomous agent is comprised. I have a hunch = a deep hunch verging on conviction = that the coherent organization of the construction of sets of constraints on the release of energy which constitutes the work by which agents build further constraints on the release of energy that in due course literally build a second copy of the agent itself, is a new concept, the proper formulation of which will be a proper concept of “organization.”
In the meantime, if I am right, what did Schrödinger miss? He was right about his microcode = the microcode will reemerge as a subset of the constraints on the release of energy by which an autonomous agent builds a rough copy of itself. Namely, the microcode is the very structure of DNA, which serves as constraints on the enzymes that then transcribe and translate the code. But Schrödinger missed stating the requirement for an agent to be nonequilibrium. On the other hand, displacement from equilibrium is a necessary condition for a microcode to do anything at all. So perhaps displacement from equilibrium was implicit in his theme. More important, I think, is that he missed the concept that an agent is a union of an autocatalytic system that does one or more work cycles. This union is a new kind of dynamical system.
Now that we have seen an autonomous agent, I find myself wondering whether autonomous agents may constitute a proper definition of life itself. I make no attempt to defend my own strong intuition that the answer is yes. I suspect that the concept of an autonomous agent as an autocatalytic system carrying out one or more work cycles defines life. If so, here is the center, the elusive core of life, that examination of the molecular chunks of cells does not reveal. Most of the remainder of this book is devoted to examining the unexpected unfoldings of this tentative definition of autonomous agents and, perhaps, life. But I certainly will not insist upon my intuition. It suYces at this stage to note that all free-living systems we know = single-cell bacteria, single-cell eukaryotic cells, and multicelled organisms = fulfill my definition of autonomous agent.
If Figure 3.4 shows us a first case of a molecular autonomous agent, how broad a family of systems does the concept of an autonomous agent embrace? I confess I do not know. Clearly, there is nothing in the concept of a reproducing system that carries out at least one thermodynamic work cycle that limits such a system to DNA, RNA, and proteins. As we have seen, Julius Rebek has already created self-reproducing organic molecules well outside the familiar classes of biopolymers. If no such reproducing molecular system yet enfolds a thermodynamic work cycle, that is not to say that we shall long be stalled in creating such systems. It seems plausible that wide classes of chemical reaction networks can fulfill the criteria I have traced above. But must autonomous agents be “molecular” in the familiar sense? Could mutually gravitating systems such as galaxies fulfill the criteria? What of systems made largely of photons, self-reproducing spectra in a lasing cavity fed by a gain medium? What of geomorphology? I do not know. Perhaps it suYces at this stage to have begun an enquiry, an investigation, rather than to have completed it.
Natural Games
I turn, in the final section of this chapter, to yet another puzzle concerning what I call a natural game. A natural game is a way of making a living in an environment. That is, autonomous agents are able to act on their own behalf and regularly do so in order to make a living in an environment. The bacterium swimming upstream in a glucose gradient is making a living in its environment. So, in fact, are all free-living entities in the biosphere.
Well, it seems straightforward enough; we all know more or less what it is to make a living. For example, I am currently writing Investigations as part of my own hopefully not-too-solipsistic eVorts to make my own living as a scientist.
But natural games are not quite so straightforward. I begin by mentioning again the rather surprising no-free-lunch theorem proved by Bill Macready and David Wolpert as postdoctoral fellows at the Santa Fe Institute a few years ago. Recall that Bill and David were wondering whether there might be some search algorithm for adapting on a fitness landscape that was inherently better than all other algorithms. For example, John Holland, another Santa Fe Institute colleague, is justly well-known for inventing his “genetic algorithm” to optimize hard computational problems. The genetic algorithm, which has been rather widely used in academic and industrial settings, is based on analogy with biological adaptation driven by mutation, recombination, and selection.
In eVect, Bill and David were wondering whether biological systems in this biosphere happen to have stumbled on the best possible optimization procedure. Importantly, the answer appears to be no. Macready and Wolpert considered a mathematically well-formulated set of “all possible fitness landscapes” and showed that, averaged over all landscapes, no search algorithm outperforms any other algorithm. No free lunch.
In short, given an arbitrary fitness landscape, only some search algorithms do well on that landscape. The search procedure must be tuned to the fitness landscapes being searched if the search procedure is to be more eVective than average among search procedures.
But this poses the important problem raised in chapter 1. Most organisms are sexual, hence adapt using both mutation and recombination as part of their search procedures in making natural livings. But my own and other research demonstrates that recombination is essentially useless on very rugged fitness landscapes. For example, Mark Feldman and Aviv Bergman at Stanford have shown that if genes that evolve on rugged landscapes increase the frequency of recombination in model populations of organisms, they will not be selected to increase, hence establish, recombination. Yet most organisms are sexual and pay the twofold loss in fitness in requiring two parents rather than one. If so, presumably our biosphere is rife with the kinds of smooth correlated landscapes for which recombination is a good search procedure.
Then how is it that in our biosphere we should find a family of landscapes that happen to be well searched by recombination? Either such smooth landscapes are built into the physical nature of things or evolution has itself somehow brought forth the very kinds of landscapes that are well searched by mutation and recombination. Restated, assuming that mutation and recombination are, in fact, good search procedures for the kinds of fitness landscapes inhabited by we mere mortals as we were hanging around and adaptively hill climbing for the past four billion years, I ask again: Whence these wonderful fitness landscapes that are so well suited to be climbed by mutation and recombination?
Let’s try another tack. Assume for the sake of discussion that I am right about my formulation of molecular autonomous agents. When life = and I argue, autonomous agents = began, their diversity was low. There are now some hundred million species, representing perhaps a thousandth of the total diversity that has wandered our globe. The rest have gone extinct. Natural games, ways of making a living, have obviously coevolved with the autonomous agents, the species, making those livings. So, as I imagined Darwin telling us in chapter 1, “The winning natural games are the games the winning species play.”
Well, of course, the winning natural games are the games the winners play. But what natural games are these? The reasonable answer leaps to mind. The winning games must be those that are readily searched out by the very adaptive search procedures used by the coevolving autonomous agents themselves.
In short, a biosphere is a self-consistent coevolutionary construction of autonomous agents and ways of making a living that are themselves self-consistently well searched by the search procedures the autonomous agents are using. In colloquial terms, from our experience in economic systems, jobs come into existence with jobholders. If no one can learn or exploit a given kind of job, that sort of job will not be widely populated and will not become diVerentiated into a family of similar jobs.
In the biosphere, modes of making a living that are well-searched by mutation and recombination will be populated by many sibling species making their livings by playing slightly diVerent natural games. Those natural games, therefore, proliferate. Ways of making a living that cannot be explored successfully by mutation and recombination will not aVord new niches for many sibling species, so those natural games will not proliferate.
We are literally making our world together, we critters. If we couldn’t make livings at it given our search procedures of mutation, recombination, and selection, we wouldn’t be making our livings doing what we are doing. These comments are only the start of understanding, and I do not profess to hold much of that understanding. But I can begin to point. A biosphere is a self-consistent coevolutionary construction of autonomous agents making livings, the natural games that constitute those livings, and the search mechanisms that allow such modes of living to be persistently mastered by adaptive natural selection.
Most broadly, I believe a general biology awaits founding. And I believe that autonomous agents will prove central to that eVort. The next feature of autonomous agents that I will note in closing this chapter will be central to any general biology. Precisely because an autonomous agent links exergonic and endergonic reactions in work cycles, the breakdown of high-energy sources here can be used to build up structure and organization there. Indeed, the coevolution of autonomous agents naturally leads to a linked web of exergonic and endergonic reactions within and between the autonomous agents. Breakdown of this stuV here is linked to the excess build up of that stuV there. By these linkages, sunlight spattered carelessly on this swirl of stuV ninety-three million miles from our average star cumulates into the wondrous structure of the giant redwoods, tall on the western slopes of the United States and Canada. Precisely because autonomous agents carry out work cycles, they = we = literally build a biosphere.
And the central factors underlying that buildup of organization are the same factors that apply in an economy = that merely human extension of biospheres. The central factors, in fact, center on “advantages of trade.” We can see this keystone concept by supposing that you and I are the only members of a tiny economy. You begin life with an endowment of a thousand pears and a hundred apples. I begin life with an endowment of a hundred pears and a thousand apples. Suppose your happiness, or “utility,” would increase if you had fewer rather than more pears and more rather than fewer apples. Alas, you have more pears than apples. I, in turn, happen to have desires such that I would be happier with rather more pears than apples. Alas, I have more apples than pears.
You and I have advantages of trade. We can both be happier if we swap some of my apples for some of your pears. It is essential to understand that, indeed, both of us can be better oV by trading. Advantages of trade are the fundamental factor driving trade itself in an economy. In an actual simple economic model, advantages of trade are studied in an “Edgeworth box.” Edgeworth invented a two-dimensional box representation of values, or “worths,” plotted along the edge of his box (Figure 3.6). In the Edgeworth box, I am represented at the bottom-left corner, you are represented at the top-right corner. A family of equal happiness, or “isoutility” curves, show your “isohappiness” trade-oVs of apples and pears at any total abundance to you of apples and pears. You are, in general, happier the more total apples and pears you have. Your happiness landscape increases from low to high like a cone-shaped mountain whose peak is located over my head. On that peak, you have all the apples and pears in the system.
My isohappiness trade-oV curves begin low at the lower-left corner and mount to a peak over your head in the upper-right corner, when I would have all the apples and pears.
The curvature of my isohappiness curves and your isohappiness curves are bent such that they are convex from my and your points of view. Therefore, if the initial economy starts with you having most of the pears and I most of the apples, as shown as a point toward the lower-right of the Edgeworth box in Figure 3.6, then that initial point of the economy lies on the intersection of a specific isohappiness curve for you and an isohappiness curve for me.
And now we can see advantages of trade. Any point that lies inside the region bounded by our two isohappiness curves is, therefore, higher on your happiness landscape and also higher on my happiness landscape. Thus, anywhere inside the region bounded by our two isohappiness curves, we are both happier. We have advantages of trade within this area bounded by our two isohappiness curves.
A few more points from Economics 100. Consider the family of your isohappiness curves and the family of my isohappiness curves. Pick one of your isohappiness curves. There will be exactly one of my isohappiness curves that just touches your isohappiness curve at a single position, thus one point of tangency. Therefore, for each of your isohappiness curves, there is a unique point of tangency with one of my isohappiness curves. Therefore we can draw a line connecting those points of tangency. In particular, we can draw a line of those tangencies across the two isohappiness curves, yours and mine, that meet at the initial apple-pear distributions to you and me at the outset, before trading, and define the region where we have advantages of trade.
The line of tangency is called the “contract curve.” Along the contract curve, there is no way to exchange apples and pears that increases both our happiness. If you are happier, I am less happy. The contract curve is said to be “Pareto-eYcient.” There is no way to make you happier without making me less happy, and vice versa. In contrast, if we have not yet attained the contract curve, there are further advantages of trade that we can attain. The economic concept of “price” in this context is just the ratio of exchange between you and me, apples for pears. Evidently, if we attain one of the points on the contract curve, that corresponds to some exchange ratio and is the price of apples for pears.
Now, nothing in a one-shot exchange economy picks any particular point on the contract curve. We tussle along the contract curve, each trying to get all the advantages of the trade. But what if we could take our happiness, now call it utility or wealth, and reinvest it in making orchards that grew apples and pears? In an economy with reinvestment, what happens if we can take our advantages of trade and reinvest any excess so that we can create more apples and pears than we had to start with?
Then let me draw an analogy for bacterial species or other autonomous agents. Let happiness, or the economist’s utility, become “rate of reproduction,” hence, fitness. Let increased happiness become “increased rate of reproduction,” hence, increased fitness. Let the advantages of trade map into mutualistic interactions in which you and I, two species of autonomous agents, help one another reproduce more rapidly. Case in point: Legume root systems with microrhizzae and symbiotic fungi, in which the root and its plant capture sunlight and water and carbon dioxide and supply sugars to the fungi, while the fungi capture nitrogen from the air and fix it into amino acids and supply amino acids to the plant. Plant and fungi feed one another.
Two mutualists, A and B, can have advantages of trade. Molecules created at metabolic cost in A and secreted can help B reproduce faster. Molecules created at metabolic cost in B and secreted can help A reproduce faster. If the help is larger than the metabolic cost in both directions, both win by helping the other. Indeed, you can quickly intuit that, since both A and B will reproduce exponentially, there might be a fixed ratio of the abundance of A and abundance of B species such that each helps the other optimally. If so, then the enhancement in the growth of A and B by their mutual interaction must be the same, otherwise, either A or B would soon be exponentially more abundant than the other, and the mutual help society would fall apart.
Peter Schuster and Peter Stadler of the University of Vienna and I at the Santa Fe Institute several years ago created a simple model of two replicating RNA species, A and B, that did help one another in just these ways, and it confirmed that in the appropriate mutual help regime, the A + B mixed community outgrew A alone or B alone. Further, the growth was such that the ratio of A and B remained fixed. Therefore, the exchange of A’s product molecules and B’s product molecules also remained fixed at a specific point on the contract curve. That point corresponds to price. So in at least some simple models, when autonomous agents form a mutualism, A and B helping one another, they have found a means to create advantages of trade, and they can find and remain on a fixed point on the contract curve that establishes an exchange ratio = the price.
And note with the plant root and the fungi, thermodynamic work has been done by the plant to synthesize the sugars from sunlight, water, and carbon dioxide, and thermodynamic work has been done by the fungi to fix nitrogen from the air into amino acids. In a real biosphere, the linking of exergonic to endergonic reactions by which thermodynamic work is done to build up complex organization is, in fact, inextricably linked with the emergence of new advantages of trade = new, enhanced ways to make livings in new niches. In the present case, the exchange of sugar and amino acids helps both plant and fungi reproduce more rapidly.
So, as noted earlier, the fact that autonomous agents do link exergonic and endergonic reactions is central to the creation of advantages of trade and hence, new niches, new mutualistic opportunities. They lead to the vast web of an ecosystem trapping sunlight; gobbling some water, nitrogen, carbon dioxide, and a few other simple molecular species; and literally building up the vast profusion of Darwin’s tangled bank. Ultimately, we should be able to build a theory that accounts for the distribution of advantages of trade, the distribution of residence times of energy stored in diVerent forms in an ecosystem, as well as the statistical patterns of linking of exergonic and endergonic reactions in a biosphere as it builds itself and persistently explores novel ways of making a living, the novel niches that permit the success of Darwin’s minor variations creating novel species for those niches.
The curious thing about evolution is that everyone thinks he understands it? Not me. Not yet. Yet I hope there may be general principles governing the self-consistent construction of any biosphere. In later chapters I will hazard a hunch or two about such general laws, but we are only at the beginning of a general biology.