While dinosaurs ruled the world some 200 million years ago, a group of nocturnal, shrewlike proto-mammals unwittingly sniffed out a strategy for survival that eventually led to the evolution of larger brains. Fossil skulls of two ancient, mammal-like reptiles suggest that natural selection for a keener sense of smell was the initial spur behind bigger brains  in early mammals, according to a report online today in Science. “Mammals didn’t get our larger brains for thinking,” says co-author Zhe-Xi Luo, a paleontologist at the Carnegie Museum of Natural History in Pittsburgh, Pennsylvania. “We got it for a more urgent and more basic need—our sense of smell was far more important.”
Birds and mammals have brains that are up to 10 times larger, relative to body size, than those of reptiles and other animals. Why? Some researchers have proposed that the early, nocturnal mammals evolved larger brains to boost their hearing, because sight was less important at night. Others have suggested that mammals’ brains are proportionately larger because as many early mammals evolved smaller bodies, their brains failed to shrink to scale.
By reconstructing the two oldest known skulls of proto-mammals—fossils of Morganucodon and Hadrocodium discovered in the famed Lufeng Formation in Yunnan, China, in 1986—Luo and colleagues found clues to how the mammalian brain began to enlarge. The researchers scanned the skulls with computed tomography (CT) scans, creating three-dimensional, virtual endocasts of the brain, based on the impressions brain tissue and spaces left on the inside of the skull. That gave them a detailed view of the surface of the brain and the nasal cavities.
Then they compared the two endocasts with those for seven fossils of early cynodonts—carnivorous reptiles that are close relatives of the first mammals—as well as with endocasts for 27 other primitive mammals that lived between 65 million and 190 million years ago and with the brains of 270 living mammals.
They found that the size of the mammalian brain evolved in three major stages. First, by the time Morganucodon was alive 190 million years ago, the brain was almost 50% larger than in cynodonts, and areas that process smell, such as the olfactory bulb, were distinctly larger. Then, a short time later in Hadrocodium, the closest known relative of living mammals, the brain had expanded another 50%, with parts involved in smell accounting for most of the increase. And third, by 65 million years ago when modern types of “crown” mammals arose, regions of the brain that control neuromuscular coordination by integrating different senses had enlarged.
The paper “provides the first evidence of the relative size of the brains and which parts were initially enlarging during critical stages in the evolution of modern mammalian brains,” says biologist R. Glenn Northcutt of the University of California (UC), San Diego. “Until now, we could only speculate what changes were occurring and at what rate. Now we have data and can infer what selective pressures were driving brain evolution in the radiation that led to mammals.” Biological anthropologist Terrence Deacon of UC Berkeley also praises the descriptive work that shows the olfactory bulb was “unambiguously enlarged” compared with the structure in reptiles. But he warns that just because it was larger when the brain began to expand doesn’t mean that the sense of smell drove the size increase—it could just correlate with enlargement caused by another adaptation.
Regardless of the trigger, though, the fossils show that a “mammalian pattern of brain organization is apparent at this very early stage of proto-mammalian evolution,” Deacon says.