CHAPTER FOUR.

Achieving the Software of Human Intelligence: How to Reverse Engineer the Human Brain

There are good reasons to believe that we are at a turning point, and that it will be possible within the next two decades to formulate a meaningful understanding of brain function. This optimistic view is based on several measurable trends, and a simple observation which has been proven repeatedly in the history of science: Scientific advances are enabled by a technology advance that allows us to see what we have not been able to see before. Scientific advances are enabled by a technology advance that allows us to see what we have not been able to see before. At about the turn of the twenty-first century, we pa.s.sed a detectable turning point in both neuroscience knowledge and computing power. For the first time in history, we collectively know enough about our own brains, and have developed such advanced computing technology, that we can now seriously undertake the construction of a verifiable, real-time, high-resolution model of significant parts of our intelligence. At about the turn of the twenty-first century, we pa.s.sed a detectable turning point in both neuroscience knowledge and computing power. For the first time in history, we collectively know enough about our own brains, and have developed such advanced computing technology, that we can now seriously undertake the construction of a verifiable, real-time, high-resolution model of significant parts of our intelligence.-LLOYD WATTS, NEUROSCIENTIST1 Now, for the first time, we are observing the brain at work in a global manner with such clarity that we should be able to discover the overall programs behind its magnificent powers.-J. G. TAYLOR, B. HORWITZ, K. J. FRISTON, NEUROSCIENTISTS2 The brain is good: it is an existence proof that a certain arrangement of matter can produce mind, perform intelligent reasoning, pattern recognition, learning and a lot of other important tasks of engineering interest. Hence we can learn to build new systems by borrowing ideas from the brain....The brain is bad: it is an evolved, messy system where a lot of interactions happen because of evolutionary contingencies. ... On the other hand, it must also be robust (since we can survive with it) and be able to stand fairly major variations and environmental insults, so the truly valuable insight from the brain might be how to create resilient complex systems that self-organize well....The interactions within a neuron are complex, but on the next level neurons seem to be somewhat simple objects that can be put together flexibly into networks. The cortical networks are a real mess locally, but again on the next level the connectivity isn't that complex. It would be likely that evolution has produced a number of modules or repeating themes that are being re-used, and when we understand them and their interactions we can do something similar.-ANDERS SANDBERG, COMPUTATIONAL NEUROSCIENTIST, ROYAL INSt.i.tUTE OF TECHNOLOGY, SWEDEN

Reverse Engineering the Brain: An Overview of the Task

The combination of human-level intelligence with a computer's inherent superiority in speed, accuracy, and memory-sharing ability will be formidable. To date, however, most AI research and development has utilized engineering methods that are not necessarily based on how the human brain functions, for the simple reason that we have not had the precise tools needed to develop detailed models of human cognition.

Our ability to reverse engineer the brain-to see inside, model it, and simulate its regions-is growing exponentially. We will ultimately understand the principles of operation underlying the full range of our own thinking, knowledge that will provide us with powerful procedures for developing the software of intelligent machines. We will modify, refine, and extend these techniques as we apply them to computational technologies that are far more powerful than the electrochemical processing that takes place in biological neurons. A key benefit of this grand project will be the precise insights it offers into ourselves. We will also gain powerful new ways to treat neurological problems such as Alzheimer's, stroke, Parkinson's disease, and sensory disabilities, and ultimately will be able to vastly extend our intelligence.

New Brain-Imaging and Modeling Tools. The first step in reverse engineering the brain is to peer into the brain to determine how it works. So far, our tools for doing this have been crude, but that is now changing, as a significant number of new scanning technologies feature greatly improved spatial and temporal resolution, price-performance, and bandwidth. Simultaneously we are rapidly acc.u.mulating data on the precise characteristics and dynamics of the const.i.tuent parts and systems of the brain, ranging from individual synapses to large regions such as the cerebellum, which comprises more than half of the brain's neurons. Extensive databases are methodically cataloging our exponentially growing knowledge of the brain. The first step in reverse engineering the brain is to peer into the brain to determine how it works. So far, our tools for doing this have been crude, but that is now changing, as a significant number of new scanning technologies feature greatly improved spatial and temporal resolution, price-performance, and bandwidth. Simultaneously we are rapidly acc.u.mulating data on the precise characteristics and dynamics of the const.i.tuent parts and systems of the brain, ranging from individual synapses to large regions such as the cerebellum, which comprises more than half of the brain's neurons. Extensive databases are methodically cataloging our exponentially growing knowledge of the brain.3 Researchers have also shown they can rapidly understand and apply this information by building models and working simulations. These simulations of brain regions are based on the mathematical principles of complexity theory and chaotic computing and are already providing results that closely match experiments performed on actual human and animal brains.

As noted in chapter 2, the power of the scanning and computational tools needed for the task of reverse engineering the brain is accelerating, similar to the acceleration in technology that made the genome project feasible. When we get to the nan.o.bot era (see "Scanning Using Nan.o.bots" on p. 163), we will be able to scan from inside inside the brain with exquisitely high spatial and temporal resolution. the brain with exquisitely high spatial and temporal resolution.4 There are no inherent barriers to our being able to reverse engineer the operating principles of human intelligence and replicate these capabilities in the more powerful computational substrates that will become available in the decades ahead. The human brain is a complex hierarchy of complex systems, but it does not represent a level of complexity beyond what we are already capable of handling. There are no inherent barriers to our being able to reverse engineer the operating principles of human intelligence and replicate these capabilities in the more powerful computational substrates that will become available in the decades ahead. The human brain is a complex hierarchy of complex systems, but it does not represent a level of complexity beyond what we are already capable of handling.

The Software of the Brain. The price-performance of computation and communication is doubling every year. As we saw earlier, the computational capacity needed to emulate human intelligence will be available in less than two decades. The price-performance of computation and communication is doubling every year. As we saw earlier, the computational capacity needed to emulate human intelligence will be available in less than two decades.5 A princ.i.p.al a.s.sumption underlying the expectation of the Singularity is that nonbiological mediums will be able to emulate the richness, subtlety, and depth of human thinking. But achieving the hardware computational capacity of a single human brain-or even of the collective intelligence of villages and nations-will not automatically produce human levels of capability. (By "human levels" I include all the diverse and subtle ways humans are intelligent, including musical and artistic apt.i.tude, creativity, physical motion through the world, and understanding and responding appropriately to emotions.) The hardware computational capacity is necessary but not sufficient. Understanding the organization and content of these resources-the software of intelligence-is even more critical and is the objective of the brain reverse-engineering undertaking. A princ.i.p.al a.s.sumption underlying the expectation of the Singularity is that nonbiological mediums will be able to emulate the richness, subtlety, and depth of human thinking. But achieving the hardware computational capacity of a single human brain-or even of the collective intelligence of villages and nations-will not automatically produce human levels of capability. (By "human levels" I include all the diverse and subtle ways humans are intelligent, including musical and artistic apt.i.tude, creativity, physical motion through the world, and understanding and responding appropriately to emotions.) The hardware computational capacity is necessary but not sufficient. Understanding the organization and content of these resources-the software of intelligence-is even more critical and is the objective of the brain reverse-engineering undertaking.

Once a computer achieves a human level of intelligence, it will necessarily soar past it. A key advantage of nonbiological intelligence is that machines can easily share their knowledge. If you learn French or read War and Peace War and Peace, you can't readily download that learning to me, as I have to acquire that scholarship the same painstaking way that you did. I can't (yet) quickly access or transmit your knowledge, which is embedded in a vast pattern of neurotransmitter concentrations (levels of chemicals in the synapses that allow one neuron to influence another) and interneuronal connections (portions of the neurons called axons and dendrites that connect neurons).

But consider the case of a machine's intelligence. At one of my companies, we spent years teaching one research computer how to recognize continuous human speech, using pattern-recognition software.6 We exposed it to thousands of hours of recorded speech, corrected its errors, and patiently improved its performance by training its "chaotic" self-organizing algorithms (methods that modify their own rules, based on processes that use semirandom initial information, and with results that are not fully predictable). Finally, the computer became quite adept at recognizing speech. Now, if you want your own personal computer to recognize speech, you don't have to put it through the same painstaking learning process (as we do with each human child); you can simply download the already established patterns in seconds. We exposed it to thousands of hours of recorded speech, corrected its errors, and patiently improved its performance by training its "chaotic" self-organizing algorithms (methods that modify their own rules, based on processes that use semirandom initial information, and with results that are not fully predictable). Finally, the computer became quite adept at recognizing speech. Now, if you want your own personal computer to recognize speech, you don't have to put it through the same painstaking learning process (as we do with each human child); you can simply download the already established patterns in seconds.

a.n.a.lytic Versus Neuromorphic Modeling of the Brain. A good example of the divergence between human intelligence and contemporary AI is how each undertakes the solution of a chess problem. Humans do so by recognizing patterns, while machines build huge logical "trees" of possible moves and countermoves. Most technology (of all kinds) to date has used this latter type of "top-down," a.n.a.lytic, engineering approach. Our flying machines, for example, do not attempt to re-create the physiology and mechanics of birds. But as our tools for reverse engineering the ways of nature are growing rapidly in sophistication, technology is moving toward emulating nature while implementing these techniques in far more capable substrates. A good example of the divergence between human intelligence and contemporary AI is how each undertakes the solution of a chess problem. Humans do so by recognizing patterns, while machines build huge logical "trees" of possible moves and countermoves. Most technology (of all kinds) to date has used this latter type of "top-down," a.n.a.lytic, engineering approach. Our flying machines, for example, do not attempt to re-create the physiology and mechanics of birds. But as our tools for reverse engineering the ways of nature are growing rapidly in sophistication, technology is moving toward emulating nature while implementing these techniques in far more capable substrates.

The most compelling scenario for mastering the software of intelligence is to tap directly into the blueprint of the best example we can get our hands on of an intelligent process: the human brain. Although it took its original "designer" (evolution) several billion years to develop the brain, it's readily available to us, protected by a skull but with the right tools not hidden from our view. Its contents are not yet copyrighted or patented. (We can, however, expect that to change; patent applications have already been filed based on brain reverse engineering.)7 We will apply the thousands of trillions of bytes of information derived from brain scans and neural models at many levels to design more intelligent parallel algorithms for our machines, particularly those based on self-organizing paradigms. We will apply the thousands of trillions of bytes of information derived from brain scans and neural models at many levels to design more intelligent parallel algorithms for our machines, particularly those based on self-organizing paradigms.

With this self-organizing approach, we don't have to attempt to replicate every single neural connection. There is a great deal of repet.i.tion and redundancy within any particular brain region. We are discovering that higher-level models of brain regions are often simpler than the detailed models of their neuronal components.

How Complex Is the Brain? Although the information contained in a human brain would require on the order of one billion billion bits (see chapter 3), the initial design of the brain is based on the rather compact human genome. The entire genome consists of eight hundred million bytes, but most of it is redundant, leaving only about thirty to one hundred million bytes (less than 10 Although the information contained in a human brain would require on the order of one billion billion bits (see chapter 3), the initial design of the brain is based on the rather compact human genome. The entire genome consists of eight hundred million bytes, but most of it is redundant, leaving only about thirty to one hundred million bytes (less than 109 bits) of unique information (after compression), which is smaller than the program for Microsoft Word. bits) of unique information (after compression), which is smaller than the program for Microsoft Word.8 To be fair, we should also take into account "epigenetic" data, which is information stored in proteins that control gene expression (that is, that determine which genes are allowed to create proteins in each cell), as well as the entire protein-replication machinery, such as the ribosomes and a host of enzymes. However, such additional information does not significantly change the order of magnitude of this calculation. To be fair, we should also take into account "epigenetic" data, which is information stored in proteins that control gene expression (that is, that determine which genes are allowed to create proteins in each cell), as well as the entire protein-replication machinery, such as the ribosomes and a host of enzymes. However, such additional information does not significantly change the order of magnitude of this calculation.9 Slightly more than half of the genetic and epigenetic information characterizes the initial state of the human brain. Slightly more than half of the genetic and epigenetic information characterizes the initial state of the human brain.

Of course, the complexity of our brains greatly increases as we interact with the world (by a factor of about one billion over the genome).10 But highly repet.i.tive patterns are found in each specific brain region, so it is not necessary to capture each particular detail to successfully reverse engineer the relevant algorithms, which combine digital and a.n.a.log methods (for example, the firing of a neuron can be considered a digital event whereas neurotransmitter levels in the synapse can be considered a.n.a.log values). The basic wiring pattern of the cerebellum, for example, is described in the genome only once but repeated billions of times. With the information from brain scanning and modeling studies, we can design simulated "neuromorphic" equivalent software (that is, algorithms functionally equivalent to the overall performance of a brain region). But highly repet.i.tive patterns are found in each specific brain region, so it is not necessary to capture each particular detail to successfully reverse engineer the relevant algorithms, which combine digital and a.n.a.log methods (for example, the firing of a neuron can be considered a digital event whereas neurotransmitter levels in the synapse can be considered a.n.a.log values). The basic wiring pattern of the cerebellum, for example, is described in the genome only once but repeated billions of times. With the information from brain scanning and modeling studies, we can design simulated "neuromorphic" equivalent software (that is, algorithms functionally equivalent to the overall performance of a brain region).

The pace of building working models and simulations is only slightly behind the availability of brain-scanning and neuron-structure information. There are more than fifty thousand neuroscientists in the world, writing articles for more than three hundred journals.11 The field is broad and diverse, with scientists and engineers creating new scanning and sensing technologies and developing models and theories at many levels. So even people in the field are often not completely aware of the full dimensions of contemporary research. The field is broad and diverse, with scientists and engineers creating new scanning and sensing technologies and developing models and theories at many levels. So even people in the field are often not completely aware of the full dimensions of contemporary research.

Modeling the Brain. In contemporary neuroscience, models and simulations are being developed from diverse sources, including brain scans, interneuronal connection models, neuronal models, and psychophysical testing. As mentioned earlier, auditory-system researcher Lloyd Watts has developed a comprehensive model of a significant portion of the human auditory-processing system from neurobiology studies of specific neuron types and interneuronal-connection information. Watts's model includes five parallel paths and the actual representations of auditory information at each stage of neural processing. Watts has implemented his model in a computer as real-time software that can locate and identify sounds and functions, similar to the way human hearing operates. Although a work in progress, the model ill.u.s.trates the feasibility of converting neurobiological models and brain-connection data into working simulations. In contemporary neuroscience, models and simulations are being developed from diverse sources, including brain scans, interneuronal connection models, neuronal models, and psychophysical testing. As mentioned earlier, auditory-system researcher Lloyd Watts has developed a comprehensive model of a significant portion of the human auditory-processing system from neurobiology studies of specific neuron types and interneuronal-connection information. Watts's model includes five parallel paths and the actual representations of auditory information at each stage of neural processing. Watts has implemented his model in a computer as real-time software that can locate and identify sounds and functions, similar to the way human hearing operates. Although a work in progress, the model ill.u.s.trates the feasibility of converting neurobiological models and brain-connection data into working simulations.

As Hans Moravec and others have speculated, these efficient functional simulations require about one thousand times less computation than would be required if we simulated the nonlinearities in each dendrite, synapse, and other subneural structure in the region being simulated. (As I discussed in chapter 3, we can estimate the computation required for functional simulation of the brain at 1016 calculations per second [cps], versus 10 calculations per second [cps], versus 1019 cps to simulate the subneural nonlinearities.) cps to simulate the subneural nonlinearities.)12 The actual speed ratio between contemporary electronics and the electrochemical signaling in biological interneuronal connections is at least one million to one. We find this same inefficiency in all aspects of our biology, because biological evolution built all of its mechanisms and systems with a severely constrained set of materials: namely, cells, which are themselves made from a limited set of proteins. Although biological proteins are three-dimensional, they are restricted to complex molecules that can be folded from a linear (one-dimensional) sequence of amino acids.

Peeling the Onion. The brain is not a single information-processing organ but rather an intricate and intertwined collection of hundreds of specialized regions. The process of "peeling the onion" to understand the functions of these interleaved regions is well under way. As the requisite neuron descriptions and brain-interconnection data become available, detailed and implementable replicas such as the simulation of the auditory regions described below (see "Another Example: Watts's Model of the Auditory Regions" on p. 183) will be developed for all brain regions. The brain is not a single information-processing organ but rather an intricate and intertwined collection of hundreds of specialized regions. The process of "peeling the onion" to understand the functions of these interleaved regions is well under way. As the requisite neuron descriptions and brain-interconnection data become available, detailed and implementable replicas such as the simulation of the auditory regions described below (see "Another Example: Watts's Model of the Auditory Regions" on p. 183) will be developed for all brain regions.

Most brain-modeling algorithms are not the sequential, logical methods that are commonly used in digital computing today. The brain tends to use self-organizing, chaotic, holographic processes (that is, information not located in one place but distributed throughout a region). It is also ma.s.sively parallel and utilizes hybrid digital-controlled a.n.a.log techniques. However, a wide range of projects has demonstrated our ability to understand these techniques and to extract them from our rapidly escalating knowledge of the brain and its organization.

After the algorithms of a particular region are understood, they can be refined and extended before being implemented in synthetic neural equivalents. They can be run on a computational substrate that is already far faster than neural circuitry. (Current computers perform computations in billionths of a second, compared to thousandths of a second for interneuronal transactions.) And we can also make use of the methods for building intelligent machines that we already understand.

Is the Human Brain Different from a Computer?

The answer to this question depends on what we mean by the word "computer." Most computers today are all digital and perform one (or perhaps a few) computations at a time at extremely high speed. In contrast, the human brain combines digital and a.n.a.log methods but performs most computations in the a.n.a.log (continuous) domain, using neurotransmitters and related mechanisms. Although these neurons execute calculations at extremely slow speeds (typically two hundred transactions per second), the brain as a whole is ma.s.sively parallel: most of its neurons work at the same time, resulting in up to one hundred trillion computations being carried out simultaneously.

The ma.s.sive parallelism of the human brain is the key to its pattern-recognition ability, which is one of the pillars of our species' thinking. Mammalian neurons engage in a chaotic dance (that is, with many apparently random interactions), and if the neural network has learned its lessons well, a stable pattern will emerge, reflecting the network's decision. At the present, parallel designs for computers are somewhat limited. But there is no reason why functionally equivalent nonbiological re-creations of biological neural networks cannot be built using these principles. Indeed, dozens of efforts around the world have already succeeded in doing so. My own technical field is pattern recognition, and the projects that I have been involved in for about forty years use this form of trainable and nondeterministic computing.

Many of the brain's characteristic methods of organization can also be effectively simulated using conventional computing of sufficient power. Duplicating the design paradigms of nature will, I believe, be a key trend in future computing. We should keep in mind, as well, that digital computing can be functionally equivalent to a.n.a.log computing-that is, we can perform all of the functions of a hybrid digital-a.n.a.log network with an all-digital computer. The reverse is not true: we can't simulate all of the functions of a digital computer with an a.n.a.log one.

However, a.n.a.log computing does have an engineering advantage: it is potentially thousands of times more efficient. An a.n.a.log computation can be performed by a few transistors or, in the case of mammalian neurons, specific electrochemical processes. A digital computation, in contrast, requires thousands or tens of thousands of transistors. On the other hand, this advantage can be offset by the ease of programming (and modifying) digital computer-based simulations.

There are a number of other key ways in which the brain differs from a conventional computer:

The brain's circuits are very slow. Synaptic-reset and neuron-stabilization times (the amount of time required for a neuron and its synapses to reset themselves after the neuron fires) are so slow that there are very few neuron-firing cycles available to make pattern-recognition decisions. Functional magnetic-resonance imaging (fMRI) and magnetoencephalography (MEG) scans show that judgments that do not require resolving ambiguities appear to be made in a single neuron-firing cycle (less than twenty milliseconds), involving essentially no iterative (repeated) processes. Recognition of objects occurs in about 150 milliseconds, so that even if we "think something over," the number of cycles of operation is measured in hundreds or thousands at most, not billions, as with a typical computer. Synaptic-reset and neuron-stabilization times (the amount of time required for a neuron and its synapses to reset themselves after the neuron fires) are so slow that there are very few neuron-firing cycles available to make pattern-recognition decisions. Functional magnetic-resonance imaging (fMRI) and magnetoencephalography (MEG) scans show that judgments that do not require resolving ambiguities appear to be made in a single neuron-firing cycle (less than twenty milliseconds), involving essentially no iterative (repeated) processes. Recognition of objects occurs in about 150 milliseconds, so that even if we "think something over," the number of cycles of operation is measured in hundreds or thousands at most, not billions, as with a typical computer.But it's ma.s.sively parallel. The brain has on the order of one hundred trillion interneuronal connections, each potentially processing information simultaneously. These two factors (slow cycle time and ma.s.sive parallelism) result in a certain level of computational capacity for the brain, as we discussed earlier. The brain has on the order of one hundred trillion interneuronal connections, each potentially processing information simultaneously. These two factors (slow cycle time and ma.s.sive parallelism) result in a certain level of computational capacity for the brain, as we discussed earlier.Today our largest supercomputers are approaching this range. The leading supercomputers (including those used by the most popular search engines) measure over 1014 cps, which matches the lower range of the estimates I discussed in chapter 3 for functional simulation. It is not necessary, however, to use the same granularity of parallel processing as the brain itself so long as we match the overall computational speed and memory capacity needed and otherwise simulate the brain's ma.s.sively parallel architecture. cps, which matches the lower range of the estimates I discussed in chapter 3 for functional simulation. It is not necessary, however, to use the same granularity of parallel processing as the brain itself so long as we match the overall computational speed and memory capacity needed and otherwise simulate the brain's ma.s.sively parallel architecture.The brain combines a.n.a.log and digital phenomena. The topology of connections in the brain is essentially digital-a connection exists, or it doesn't. An axon firing is not entirely digital but closely approximates a digital process. Most every function in the brain is a.n.a.log and is filled with nonlinearities (sudden shifts in output, rather than levels changing smoothly) that are substantially more complex than the cla.s.sical model that we have been using for neurons. However, the detailed, nonlinear dynamics of a neuron and all of its const.i.tuents (dendrites, spines, channels, and axons) can be modeled through the mathematics of nonlinear systems. These mathematical models can then be simulated on a digital computer to any desired degree of accuracy. As I mentioned, if we simulate the neural regions using transistors in their native a.n.a.log mode rather than through digital computation, this approach can provide improved capacity by three or four orders of magnitude, as Carver Mead has demonstrated. The topology of connections in the brain is essentially digital-a connection exists, or it doesn't. An axon firing is not entirely digital but closely approximates a digital process. Most every function in the brain is a.n.a.log and is filled with nonlinearities (sudden shifts in output, rather than levels changing smoothly) that are substantially more complex than the cla.s.sical model that we have been using for neurons. However, the detailed, nonlinear dynamics of a neuron and all of its const.i.tuents (dendrites, spines, channels, and axons) can be modeled through the mathematics of nonlinear systems. These mathematical models can then be simulated on a digital computer to any desired degree of accuracy. As I mentioned, if we simulate the neural regions using transistors in their native a.n.a.log mode rather than through digital computation, this approach can provide improved capacity by three or four orders of magnitude, as Carver Mead has demonstrated.13The brain rewires itself. Dendrites are continually exploring new spines and synapses. The topology and conductance of dendrites and synapses are also continually adapting. The nervous system is self-organizing at all levels of its organization. While the mathematical techniques used in computerized pattern-recognition systems such as neural nets and Markov models are much simpler than those used in the brain, we do have substantial engineering experience with self-organizing models. Dendrites are continually exploring new spines and synapses. The topology and conductance of dendrites and synapses are also continually adapting. The nervous system is self-organizing at all levels of its organization. While the mathematical techniques used in computerized pattern-recognition systems such as neural nets and Markov models are much simpler than those used in the brain, we do have substantial engineering experience with self-organizing models.14 Contemporary computers don't literally rewire themselves (although emerging "self-healing systems" are starting to do this), but we can effectively simulate this process in software. Contemporary computers don't literally rewire themselves (although emerging "self-healing systems" are starting to do this), but we can effectively simulate this process in software.15 In the future, we can implement this in hardware, as well, although there may be advantages to implementing most self-organization in software, which provides more flexibility for programmers. In the future, we can implement this in hardware, as well, although there may be advantages to implementing most self-organization in software, which provides more flexibility for programmers.Most of the details in the brain are random. While there is a great deal of stochastic (random within carefully controlled constraints) process in every aspect of the brain, it is not necessary to model every "dimple" on the surface of every dendrite, any more than it is necessary to model every tiny variation in the surface of every transistor in understanding the principles of operation of a computer. But certain details are critical in decoding the principles of operation of the brain, which compels us to distinguish between them and those that comprise stochastic "noise" or chaos. The chaotic (random and unpredictable) aspects of neural function can be modeled using the mathematical techniques of complexity theory and chaos theory. While there is a great deal of stochastic (random within carefully controlled constraints) process in every aspect of the brain, it is not necessary to model every "dimple" on the surface of every dendrite, any more than it is necessary to model every tiny variation in the surface of every transistor in understanding the principles of operation of a computer. But certain details are critical in decoding the principles of operation of the brain, which compels us to distinguish between them and those that comprise stochastic "noise" or chaos. The chaotic (random and unpredictable) aspects of neural function can be modeled using the mathematical techniques of complexity theory and chaos theory.16The brain uses emergent properties. Intelligent behavior is an emergent property of the brain's chaotic and complex activity. Consider the a.n.a.logy to the apparently intelligent design of termite and ant colonies, with their delicately constructed interconnecting tunnels and ventilation systems. Despite their clever and intricate design, ant and termite hills have no master architects; the architecture emerges from the unpredictable interactions of all the colony members, each following relatively simple rules. Intelligent behavior is an emergent property of the brain's chaotic and complex activity. Consider the a.n.a.logy to the apparently intelligent design of termite and ant colonies, with their delicately constructed interconnecting tunnels and ventilation systems. Despite their clever and intricate design, ant and termite hills have no master architects; the architecture emerges from the unpredictable interactions of all the colony members, each following relatively simple rules.The brain is imperfect. It is the nature of complex adaptive systems that the emergent intelligence of its decisions is suboptimal. (That is, it reflects a lower level of intelligence than would be represented by an optimal arrangement of its elements.} It needs only to be good enough, which in the case of our species meant a level of intelligence sufficient to enable us to outwit the compet.i.tors in our ecological niche (for example, primates who also combine a cognitive function with an opposable appendage but whose brains are not as developed as humans and whose hands do not work as well). It is the nature of complex adaptive systems that the emergent intelligence of its decisions is suboptimal. (That is, it reflects a lower level of intelligence than would be represented by an optimal arrangement of its elements.} It needs only to be good enough, which in the case of our species meant a level of intelligence sufficient to enable us to outwit the compet.i.tors in our ecological niche (for example, primates who also combine a cognitive function with an opposable appendage but whose brains are not as developed as humans and whose hands do not work as well).We contradict ourselves. A variety of ideas and approaches, including conflicting ones, leads to superior outcomes. Our brains are quite capable of holding contradictory views. In fact, we thrive on this internal diversity. Consider the a.n.a.logy to a human society, particularly a democratic one, with its constructive ways of resolving multiple viewpoints. A variety of ideas and approaches, including conflicting ones, leads to superior outcomes. Our brains are quite capable of holding contradictory views. In fact, we thrive on this internal diversity. Consider the a.n.a.logy to a human society, particularly a democratic one, with its constructive ways of resolving multiple viewpoints.The brain uses evolution. The basic learning paradigm used by the brain is an evolutionary one: the patterns of connections that are most successful in making sense of the world and contributing to recognitions and decisions survive. A newborn's brain contains mostly randomly linked interneuronal connections, and only a portion of those survive in the two-year-old brain. The basic learning paradigm used by the brain is an evolutionary one: the patterns of connections that are most successful in making sense of the world and contributing to recognitions and decisions survive. A newborn's brain contains mostly randomly linked interneuronal connections, and only a portion of those survive in the two-year-old brain.17The patterns are important. Certain details of these chaotic self-organizing methods, expressed as model constraints (rules defining the initial conditions and the means for self-organization), are crucial, whereas many details within the constraints are initially set randomly. The system then self-organizes and gradually represents the invariant features of the information that has been presented to the system. The resulting information is not found in specific nodes or connections but rather is a distributed pattern. Certain details of these chaotic self-organizing methods, expressed as model constraints (rules defining the initial conditions and the means for self-organization), are crucial, whereas many details within the constraints are initially set randomly. The system then self-organizes and gradually represents the invariant features of the information that has been presented to the system. The resulting information is not found in specific nodes or connections but rather is a distributed pattern.The brain is holographic. There is an a.n.a.logy between distributed information in a hologram and the method of information representation in brain networks. We find this also in the self-organizing methods used in computerized pattern recognition, such as neural nets, Markov models, and genetic algorithms. There is an a.n.a.logy between distributed information in a hologram and the method of information representation in brain networks. We find this also in the self-organizing methods used in computerized pattern recognition, such as neural nets, Markov models, and genetic algorithms.18The brain is deeply connected. The brain gets its resilience from being a deeply connected network in which information has many ways of navigating from one point to another. Consider the a.n.a.logy to the Internet, which has become increasingly stable as the number of its const.i.tuent nodes has increased. Nodes, even entire hubs of the Internet, can become inoperative without ever bringing down the entire network. Similarly, we continually lose neurons without affecting the integrity of the entire brain. The brain gets its resilience from being a deeply connected network in which information has many ways of navigating from one point to another. Consider the a.n.a.logy to the Internet, which has become increasingly stable as the number of its const.i.tuent nodes has increased. Nodes, even entire hubs of the Internet, can become inoperative without ever bringing down the entire network. Similarly, we continually lose neurons without affecting the integrity of the entire brain.The brain does have an architecture of regions. Although the details of connections within a region are initially random within constraints and self-organizing, there is an architecture of several hundred regions that perform specific functions, with specific patterns of connections between regions. Although the details of connections within a region are initially random within constraints and self-organizing, there is an architecture of several hundred regions that perform specific functions, with specific patterns of connections between regions.The design of a brain region is simpler than the design of a neuron. Models often get simpler at a higher level, not more complex. Consider an a.n.a.logy with a computer. We do need to understand the detailed physics of semiconductors to model a transistor, and the equations underlying a single real transistor are complex. However, a digital circuit that multiplies two numbers, although involving hundreds of transistors, can be modeled far more simply, with only a few formulas. An entire computer with billions of transistors can be modeled through its instruction set and register description, which can be described on a handful of written pages of text and mathematical transformations. Models often get simpler at a higher level, not more complex. Consider an a.n.a.logy with a computer. We do need to understand the detailed physics of semiconductors to model a transistor, and the equations underlying a single real transistor are complex. However, a digital circuit that multiplies two numbers, although involving hundreds of transistors, can be modeled far more simply, with only a few formulas. An entire computer with billions of transistors can be modeled through its instruction set and register description, which can be described on a handful of written pages of text and mathematical transformations.

The software programs for an operating system, language compilers, and a.s.semblers are reasonably complex, but modeling a particular program-for example, a speech-recognition program based on Markov modeling-may be described in only a few pages of equations. Nowhere in such a description would be found the details of semiconductor physics. A similar observation also holds true for the brain. A particular neural arrangement that detects a particular invariant visual feature (such as a face) or that performs a band-pa.s.s filtering (restricting input to a specific frequency range) operation on auditory information or that evaluates the temporal proximity of two events can be described with far greater simplicity than the actual physics and chemical relations controlling the neurotransmitters and other synaptic and dendritic variables involved in the respective processes. Although all of this neural complexity will have to be carefully considered before advancing to the next higher level (modeling the brain), much of it can be simplified once the operating principles of the brain are understood.

Trying to Understand Our Own Thinking The Accelerating Pace of Research

We are now approaching the knee of the curve (the period of rapid exponential growth) in the accelerating pace of understanding the human brain, but our attempts in this area have a long history. Our ability to reflect on and build models of our thinking is a unique attribute of our species. Early mental models were of necessity based on simply observing our external behavior (for example, Aristotle's a.n.a.lysis of the human ability to a.s.sociate ideas, written 2,350 years ago).19 At the beginning of the twentieth century we developed the tools to examine the physical processes inside inside the brain. An early breakthrough was the measurement of the electrical output of nerve cells, developed in 1928 by neuroscience pioneer E. D. Adrian, which demonstrated that there were electrical processes taking place inside the brain. the brain. An early breakthrough was the measurement of the electrical output of nerve cells, developed in 1928 by neuroscience pioneer E. D. Adrian, which demonstrated that there were electrical processes taking place inside the brain.20 As Adrian write, "I had arranged some electrodes on the optic nerve of a toad in connection with some experiment on the retina. The room was nearly dark and I was puzzled to hear repeated noises in the loudspeaker attached to the amplifier, noises indicating that a great deal of impulse activity was going on. It was not until I compared the noises with my own movements around the room that I realized I was in the field of vision of the toad's eye and that it was signaling what I was doing." As Adrian write, "I had arranged some electrodes on the optic nerve of a toad in connection with some experiment on the retina. The room was nearly dark and I was puzzled to hear repeated noises in the loudspeaker attached to the amplifier, noises indicating that a great deal of impulse activity was going on. It was not until I compared the noises with my own movements around the room that I realized I was in the field of vision of the toad's eye and that it was signaling what I was doing."

Adrian's key insight from this experiment remains a cornerstone of neuroscience today: the frequency of the impulses from the sensory nerve is proportional to the intensity of the sensory phenomena being measured. Fr example, the higher the intensity of the light, the higher the frequency (pulses per second) of the neural impulses from the retina to the brain. It was a student of Adrian, Horace Barlow, who contributed another lasting insight, "trigger features" in neurons, with the discovery that the retinas of frogs and rabbits has single neurons that would trigger on "seeing" specific shapes, directions, or velocities. In other words, perception involves a series of stages, with each layer of neurons recognizing more sophisticated features of the image.

In 1939 we began to develop an idea of how neurons perform: by acc.u.mulating (adding) their inputs and then producing a spike of membrane conductance (a sudden increase in the ability of the neuron's membrane to conduct a signal) an voltage along the neuron's axon (which connects to other neuron's via a synapse). A. L. Hodgkin and A. F. Huxley described their theory of the axon's "action potential" (voltage).21 They also made an actual measurement of an action potential on an animal neural axon in 1952. They chose squid neurons because of their size and accessible anatomy. They also made an actual measurement of an action potential on an animal neural axon in 1952. They chose squid neurons because of their size and accessible anatomy.

Building on Hodgkin and Huxley's insight W. S. McCulloch and W. Pitts developed in 1943 a simplified model of neural nets that motivated a half century of work on artificial (simulated) neural nets (using a computer program to simulate the way neurons work in the brain as a network). This model was further refined by Hodgkin and Huxley in 1952. Although we now realize that actual neurons are far more complex that these early models, the original concept has held up well. This basic neural-net model has a neural "weight" (representing the "strength" of the connection) for each synapse and a nonlinearity (firing threshold) in the neuron soma (cell body).

As the sum of the weighted inputs to the neuron soma increases, there is relatively little response from the neuron until a critical threshold is reached, as which point the neuron rapidly increased the output of its axon and fires. Different neurons have different thresholds. Although recent research shows that the actual response is more complex than this, the McCulloch-Pitts and Hodgkin-Huxley models remain essentially valid.

These insights led to an enormous amount of early work in creating artificial neural nets, in a field that became known as connectionism. This was perhaps the first self-organizing paradigm introduced to the field of computation.

A key requirement for a self-organizing system is a nonlinearity: some means of creating outputs that are not simple weights sums of the inputs. The early neural-net models provided this nonlinearity in their replica of the neuron nucleus.23 (The basic neural-net method is straightforward.) (The basic neural-net method is straightforward.)24 Work initiated by Alan Turing on theoretical models of computation around the same time also showed that computation requires a nonlinearity. A system that simple creates weighted sums of its inputs cannot perform the essential requirements of computation. Work initiated by Alan Turing on theoretical models of computation around the same time also showed that computation requires a nonlinearity. A system that simple creates weighted sums of its inputs cannot perform the essential requirements of computation.

We now know that actual biological neurons have many other nonlinearities resulting from the electrochemical action of the synapses and the morphology (shape) of the dendrites. Different arrangements of biological neurons can perform computations, including subtracting, multiplying, averaging, filtering, normalizing, and thresholding signals, among other types of transformations.

The ability of neurons to perform multiplication is important because it allowed the behavior of one network of neurons in the brain to be modulated (influenced) by the results of computations of another network. Experiments using electrophysiological measurements on monkeys provide evidence that the rate of signaling by neurons in the visual cortex when processing an image is increased or decreased by whether or not the monkey is paying attention to a particular area of that image.25 Human fMRI studies have also shown that paying attention to a particular area of an image increases the responsiveness of the neurons processing that image in a cortical region called V5, which is responsible for motion detection. Human fMRI studies have also shown that paying attention to a particular area of an image increases the responsiveness of the neurons processing that image in a cortical region called V5, which is responsible for motion detection.26 The connectionism movement experienced a setback in 1969 with the publication of the book Perceptrons Perceptrons by MIT's Marvin Minsky and Seymour Papert. by MIT's Marvin Minsky and Seymour Papert.27 It included a key theorem demonstrating that the most common (and simplest) type of neural net used at the time (called a Perceptron, pioneered by Cornell's Frank Rosenblatt), was unable to solve the simple problem of determining whether or not a line drawing was fully connected. It included a key theorem demonstrating that the most common (and simplest) type of neural net used at the time (called a Perceptron, pioneered by Cornell's Frank Rosenblatt), was unable to solve the simple problem of determining whether or not a line drawing was fully connected.28 The neural-net movement had a resurgence in the 1980s using a method called "backpropagation," in which the strength of each simulated synapse was determined using a learning algorithm that adjusted the weight (the strength of the output of each of artificial neuron after each training trial so the network could "learn" to more correctly match the right answer. The neural-net movement had a resurgence in the 1980s using a method called "backpropagation," in which the strength of each simulated synapse was determined using a learning algorithm that adjusted the weight (the strength of the output of each of artificial neuron after each training trial so the network could "learn" to more correctly match the right answer.

However, backpropagation is not a feasible model of training synaptic weight in an actual biological neural network, because backward connections to actually adjust the strength of the synaptic connections do not appear to exist in mammalian brains. In computers, however, this type of self-organizing system can solve a wide range of pattern-recognition problems, and the power of this simple model of self-organizing interconnected neurons has been demonstrated.

Less well know is Hebb's second form of learning: a hypothesized loop in which he excitation of the neuron would feed back on itself (possibly through other layers), causing a reverberation (a continued reexcitation could be the source of short-term learning). He also suggested that this short-term reverberation could lead to long-term memories: "Let us a.s.sume then that the persistence or repet.i.tion of a reverberatory activity (or 'trace') tends to induce lasting cellular changes that add to its stability. The a.s.sumption can be precisely stated as follows: When an axon of cell A is near enough to excite a cell B and repeatedly or persistently take part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cell's firing B, is increased."

Although Hebbian reverberatory memory is not as well established as Hebb's synaptic learning, instances have been recently discovered. For example, sets of excitatory neurons (ones that stimulate a synapse) and inhibitory neurons (ones that block a stimulus) begin an oscillation when certain visual patterns are presented.29 And researchers at MIT and Lucent Technologies' Bell Labs have created an electronic integrated circuit, composed of transistors, that simulates the action of sixteen excitatory neurons and one inhibitory neuron to mimic the biological circuitry of the cerebral cortex. And researchers at MIT and Lucent Technologies' Bell Labs have created an electronic integrated circuit, composed of transistors, that simulates the action of sixteen excitatory neurons and one inhibitory neuron to mimic the biological circuitry of the cerebral cortex.30 These early models of neurons and neural information processing, although overly simplified and inaccurate in some respects, were remarkable, given the lack of data and tools when these theories were developed.

Peering into the Brain

We've been able to reduce drift and noise in our instruments to such an extent that we can see the tiniest motions of these molecules, through distances that are less than their own diameters....[T]hese kinds of experiments were just pipedreams 15 years ago.-STEVEN BLOCK, PROFESSOR OF BIOLOGICAL SCIENCES AND OF APPLIED PHYSICS, STANFORD UNIVERSITY

Imagine that we were trying to reverse engineer a computer without knowing anything about it (the "black box" approach). We might start by placing arrays of magnetic sensors around the device. We would notice that during operations that updated a database, significant activity was taking place in a particular circuit board. We would be likely to take note that there was also action in the hard disk during these operations. (Indeed, listening to the hard disk has always been one crude window into what a computer is doing.) We might then theorize that the disk had something to do with the long-term memory that stores the databases and that the circuit board that is active during these operations was involved in transforming the data to be stored. This tells us approximately where and when the operations are taking place but relatively little about how these tasks are accomplished.

If the computer's registers (temporary memory locations) were connected to front-panel lights (as was the case with early computers), we would see certain patterns of light flickering that indicated rapid changes in the states of these registers during periods when the computer was a.n.a.lyzing data but relatively slow changes when the computer was transmitting data. We might then theorize that these lights reflected changes in logic state during some kind of a.n.a.lytic behavior. Such insights would be accurate but crude and would fail to provide us with a theory of operation or any insights as to how information is actually coded or transformed.

The hypothetical situation described above mirrors the sort of efforts that have been undertaken to scan and model the human brain with the crude tools that have historically been available. Most models based on contemporary brain-scanning research (utilizing such methods as fMRI, MEG, and others discussed below) are only suggestive of the underlying mechanisms. Although these studies are valuable, their crude spatial and temporal resolution is not adequate for reverse engineering the salient features of the brain.

New Tools for Scanning the Brain. Now imagine, in our computer example above, that we are able to actually place precise sensors at specific points in the circuitry and that these sensors are capable of tracking specific signals at very high speeds. We would now have the tools needed to follow the actual information being transformed in real time, and we would be able to create a detailed description of how the circuits actually work. This is, in fact, exactly how electrical engineers go about understanding and debugging circuits such as computer boards (to reverse engineer a compet.i.tor's product, for example), using logic a.n.a.lyzers that visualize computer signals. Now imagine, in our computer example above, that we are able to actually place precise sensors at specific points in the circuitry and that these sensors are capable of tracking specific signals at very high speeds. We would now have the tools needed to follow the actual information being transformed in real time, and we would be able to create a detailed description of how the circuits actually work. This is, in fact, exactly how electrical engineers go about understanding and debugging circuits such as computer boards (to reverse engineer a compet.i.tor's product, for example), using logic a.n.a.lyzers that visualize computer signals.

Neuroscience has not yet had access to sensor technology that would achieve this type of a.n.a.lysis, but that situation is about to change. Our tools for peering into our brains are improving at an exponential pace. The resolution of noninvasive brain-scanning devices is doubling about every twelve months (per unit volume).31 We see comparable improvements in the speed of brain scanning image reconstruction: The most commonly used brain-scanning tool is fMRI, which provides relatively high spatial resolution of one to three millimeters (not high enough to image individual neurons) but low temporal (time) resolution of a few seconds. Recent generations of fMRI technology provide time resolution of about one second, or a tenth of a second for a thin brain slice.

Another commonly used technique is MEG, which measures weak magnetic fields outside the skull, produced princ.i.p.ally by the pyramidal neurons of the cortex. MEG is capable of rapid (one millisecond) temporal resolution but only very crude spatial resolution, about one centimeter.

Fritz Sommer, a princ.i.p.al investigator at Redwood Neuroscience Inst.i.tute, is developing methods of combining fMRI and MEG to improve the spatiotemporal precision of the measurements. Other recent advances have demonstrated fMRI techniques capable of mapping regions called columnar and laminar structures, which are only a fraction of a millimeter wide, and of detecting tasks that take place in tens of milliseconds.32 fMRI and a related scanning technique using positrons called positron-emission tomography (PET) both gauge neuronal activity through indirect means. PET measures regional cerebral blood flow (rCBF), while tMRI measures blood-oxygen levels.33 Although the relationship of these blood-flow amounts to neural activity is the subject of some controversy, the consensus is that they reflect local synaptic activity, not the spiking of neurons. The relationship of neural activity to blood flow was first articulated in the late nineteenth century. Although the relationship of these blood-flow amounts to neural activity is the subject of some controversy, the consensus is that they reflect local synaptic activity, not the spiking of neurons. The relationship of neural activity to blood flow was first articulated in the late nineteenth century.34 A limitation of tMRI, however, is that the relationship of blood flow to synaptic activity is not direct: a variety of metabolic mechanisms affect the relationship between the two phenomena. A limitation of tMRI, however, is that the relationship of blood flow to synaptic activity is not direct: a variety of metabolic mechanisms affect the relationship between the two phenomena.

However, both PET and tMRI are believed to be most reliable for measuring relative changes in brain state. The primary method they use is the "subtraction paradigm," which can show regions that are most active during particular tasks.35 This procedure involves subtracting data produced by a scan when the subject is not performing an activity from data produced while the subject is performing a specified mental activity. The difference represents the change in brain state. This procedure involves subtracting data produced by a scan when the subject is not performing an activity from data produced while the subject is performing a specified mental activity. The difference represents the change in brain state.

An invasive technique that provides high spatial and temporal resolution is "optical imaging," which involves removing part of the skull, staining the living brain tissue with a dye that fluoresces upon neural activity, and then imaging the emitted light with a digital camera. Since optical imaging requires surgery, it has been used mainly in animal, particularly mouse, experiments.

Another approach to identifying brain functionality in different regions is transcranial magnetic stimulation (TMS), which involves applying a strong-pulsed magnetic field from outside the skull, using a magnetic coil precisely positioned over the head. By either stimulating or inducing a "virtual lesion" of (by temporarily disabling) small regions of the brain, skills can be diminished or enhanced.36 TMS can also be used to study the relationship of different areas of the brain on specific tasks and can even induce sensations of mystical experiences. TMS can also be used to study the relationship of different areas of the brain on specific tasks and can even induce sensations of mystical experiences.37 Brain scientist Allan Snyder has reported that about 40 percent of his test subjects hooked up to TMS display significant new skills, many of which are remarkable, such as drawing abilities. Brain scientist Allan Snyder has reported that about 40 percent of his test subjects hooked up to TMS display significant new skills, many of which are remarkable, such as drawing abilities.38 If we have the option of destroying the brain that we are scanning, dramatically higher spatial resolution becomes possible. Scanning a frozen brain is feasible today, though not yet at sufficient speed or bandwidth to fully map all interconnections. But again, in accordance with the law of accelerating returns, this potential is growing exponentially, as are all other facets of brain scanning.

Carnegie Mellon University's Andreas Nowatzyk is scanning the nervous system of the brain and body of a mouse with a resolution of less than two hundred nanometers, which is approaching the resolution needed for full reverse engineering. Another destructive scanner called the "Brain Tissue Scanner" developed at the Brain Networks Laboratory at Texas A&M University is able to scan an entire mouse brain at a resolution of 250 nanometers in one month, using slices.39

Improving Resolution. Many new brain-scanning technologies now in development are dramatically improving both temporal and spatial resolution. This new generation of sensing and scanning systems is providing the tools needed to develop models with unprecedented fine levels of detail. Following is a small sample of these emerging imaging and sensing systems. Many new brain-scanning technologies now in development are dramatically improving both temporal and spatial resolution. This new generation of sensing and scanning systems is providing the tools needed to develop models with unprecedented fine levels of detail. Following is a small sample of these emerging imaging a