Developmental programs as the search space for evolution - and its relation to the success of AI.

I had to pause and think through this quote by Neil Gershenfeld on Lex Fridman's podcast. It greatly improved my intuitive understanding of how evolution could get us to where we are. Fascinating.

[...] nowhere in your genome is the number five. It doesn't store the fact that you have five fingers. What it stores is what's called the developmental program. It's a series of steps and the steps have the character of like, grow up a gradient or break symmetry. And at the end of that developmental program, you have five fingers. So you are stored not as a body plan, but as a growth plan. And there's two reasons for that. One reason is just compression. Billions of genes can place trillions of cells. But the much deeper one is evolution doesn't randomly perturb. Almost anything you did randomly in the genome would be fatal or inconsequential, but not interesting. But when you modify things in these developmental programs, you go from like webs for swimming to fingers, or you go from walking to wings for flying. It's a space in which search is interesting. So this is the heart of the success of AI. In part, it was the scaling we talked about a while ago. And in part, it was the representations for which search is effective. AI has found good representations. It hasn't found new ways to search.

Lex Fridman Podcast #380, transcript.

I remember the first time I trained an autoencoder to map the MNIST dataset to 2D and back. The sense of awe and satisfaction when interpolating between various samples in the learnt compressed space, mapping it back to the data domain to observe how the new samples made sense intuitively as an intermediate point between the original ones.

It is fascinating to think of DNA sequences as the analogous compressed space for functioning biological creatures. This idea makes the evolutional process seem more plausible - that perturbations in this space lead to meaningful tweaks and features of the resulting organisms. It somehow also intuitively makes sense that in this space of descriptions of developmental programs, extrapolation works well enough to discover new favorable modifications under the pressure of natural selection.


Profile picture

Thanks for dropping by. Don't hesitate to reach out, I'd be happy to connect!