Book Review: Creativity in Science
This book proposes viewing scientific creativity as a combinatorial process. Each scientific domain contains a population of ideas. Finding new permutations of these to create original ideas is what we think of as creativity. The more permutations a scientist can create, the more likely they are to produce creative work.
This idea has some interesting implications. It might explain why scientific creativity may diminish over time. The more ideas in your head, the more permutations you can find. When you're young, you can devote your entire life to acquiring new knowledge and giving yourself more ideas to permutate. As you get older and have other obligations (family, etc) your rate of acquiring new knowledge goes down and so you have a reduced supply of new ideas to permutate. Also you tend to become more focused on acquiring new knowledge within a narrower domain (i.e. you start specializing) which again reduces your potential permutations. Finally you become less receptive to new ideas as you age (Planck's Principle).
There were some interesting observations from studies about the process of scientific research itself. Most studies measure creative output as the number of highly citied publications. This lets us measure scientific output at higher resolution than just relying on works like Principa Mathematica. The data shows that the most impactful scientists are also the most prolific and it's a very skewed distribution. An elite group of scientists are responsible for producing both the highest quality and quantity of research. Interestingly, the more scientists there are within a particular domain — the more elite it becomes. The combinatorial view of creativity would explain this as the larger population of scientists can create a larger population of ideas and therefore a large set of possible permutations for all scientists to find. The most creative scientists can take better advantage of this expanded population of ideas to increase their creative output and grow their market share of all creative ideas. This view fits with my experiences of extremely smart people in the startup world. The gap between them and everyone else seems to only grow over time as they continue spending weekends and free time learning new things and starting new projects.
There also seems to be a strong correlation between the most creative scientists and the diversity of projects they're working on at one time. Darwin worked on multiple projects while developing the Origin of Species and when he felt stuck on one, he'd moved onto another. It seems the optimal balance is to have a set of concurrent projects with a core set of themes but with a diversity of difficulty, type and how much of your attention they currently command.
There is also a look at some of the psychological attributes of highly creative scientists. It seems that IQ is important but it's more of a minimum threshold than a differentiator amongst the top scientists. You need an IQ higher than 120 but after that, other factors determine your scientific creativity.
One of these factors might be "associative richness" — the ability to connect seemingly unrelated ideas to form new, logically sound ideas. Ernest Mach, the physicist, described this as the distinction between a mechanical memory that's good for recalling facts (which is necessary to avoid mistakes and work on hard problems) and the ability to connect these facts in interesting ways to invent new things. This distinction fits my personal experiences. I've worked with smart people who I wouldn't describe as creative. They're suited to solving well defined problems that require algorithmic problem solving but you wouldn't expect them to propose new or interesting ideas. They'll usually argue that "creativity" is a meaningless term and can be reduced to rigorous, logical thought. The author opposes this view and note that even computer programs best mimicking human creativity, have to introduce some random element or mechanism.
You can fit the associative model of ideas into the combinatorial model of scientific creativity. If you think of all the ideas in your head and graph the number of associations between them against the strength of each association. A steep curve would indicate you had a small number of associations but very strong links between them. A flat curve would mean a large number of associations with weaker links between them. To be more creative you want the latter. The more associations you have between ideas, the more permutations you can find and the greater your odds of finding a revelatory new one.
This model also explains why the most creative people also have a lot of bad ideas. If the associations between your ideas are high in quantity but lower in average quality, you'll draw a lot of connections that lead to dead ends i.e. you'll link together ideas that just create new and bad ideas. Still, the alternative is to generate fewer, more accurate ideas but be less likely to generate truly creative ones.
Overall I thought this book gave me a more detailed way to express my existing views on creativity and science. I do think it overemphasizes the importance of chance relative to how special the individuals who product scientific breakthroughs are. Having the energy to be the most prolific scientist within a domain and maintain a diversity of projects and interests, requires a special type of person.