The Burgess Shale (about 510 Mya) is not the oldest Cambrian deposit known. There are deposits from China (Chengjiang) closer to the beginning of the era. But it is particularly rich and well studied. It also featured in debates about some Big Questions: How important are evolutionary laws versus historical accidents? Has the living world become more or less diverse over time?
Stephen Jay Gould (1941-2002) was one of the most widely known evolutionary biologists of his time. In 1989 he wrote a book about the Burgess Shale, called “Wonderful Life.” The title alluded to Frank Capra’s movie, “It’s a Wonderful Life,”* starring Jimmy Stewart. In the movie, the Stewart character, who thinks he’s wasted his life stuck in his home town, has a chance to see how things would have turned out if he had died young in an accident. He learns that his existence made a huge difference to his town.
Similarly, Gould argued that accidents of which Cambrian species survived and which went extinct made a huge difference to the later evolution of life. To make his case, he developed a subsidiary argument: that the Cambrian fauna displays a radical diversity of body plans in comparison with later eras. Gould made an analogy with the early development of automobiles, which featured diesel engines, steam engines, and electrical engines, before settling down on mostly just gasoline engines. (He was writing before Priuses and Teslas, of course.)
This subsidiary argument has not fared well. Most of the supposedly radically different forms from the Burgess Shale turn out to be not that radically different from one another, or from modern forms. Most notorious was the case of a specimen called Hallucigenia. Paleontologists thought that this creature was like nothing that ever lived before. But later more complete finds of related forms made it clear they were looking at it upside down; its “legs” are actually defensive spikes, the “tentacles” on top are actually legs. It’s probably related to the ancestor of velvet worms, a group related to vertebrates and still living in Australia. A different perspective comes from Simon Conway Morris one of the experts on the Burgess Shale, in in his book “Crucible of Creation.”
And here, from a nice review of more recent work on contingency and determinism in evolution:
Gould would be pleased that his thought experiment of replaying life’s tape has been transformed into an empirical research program that explores the roles of historical contingency and natural selection at multiple levels. However, his view of historical influences as the central feature of evolution remains debatable. Laboratory replay experiments show that repeatable outcomes are common, at least when defined broadly (e.g., at the level of genes, not mutations). Moreover, convergence in nature is more common than many biologists would have wagered not long ago. On the other hand, as evolving lineages accumulate more differences, both experimental and comparative approaches suggest that the power of selection to drive convergence is reduced, and the contingent effects of history are amplified.
*Not to be confused with Jerome Bixby’s short story “It’s a Good Life,” about a very different small town.