Slow Science: The Epilogue - Surprise!

Bet you weren't expecting this series to rear its ugly head again?

I was perusing my Google reader and to my delight came across an article in PNAS by Lindenmayer et al. (2010) called, "Improved probability of detection of ecological surprises." In it the authors describe four long-term research programs that have led to both ecological surprises as well as ecological discoveries. From these important surprises and discoveries, the authors derive four "lessons." They further argue that we should provide more financial support to long-term experiments, encourage scientists to develop natural history skills, increase collaborations among different disciplines, and provide an environment where time is plentiful.

The slipperiness of empirical truth

I just finished reading an engaging article in The New Yorker, called "The Truth Wears Off." The author Jonah Lehrer talks about a problem that many scientific disciplines face - it's called the decline effect. The decline effect is when well-established and multiply confirmed empirical studies begin to show a reduced effect size or are no longer provable.

One example of this is called the phenomenon of "verbal overshadowing" demonstrated by a psychologist named Jonathan Schooler in 1990. He showed that subjects shown a face and asked to describe it are LESS likely to recognize the face when shown the same face later than those who simply view the face. But Schooler states in this New Yorker article that he has since found it difficult to replicate this earlier finding. He says,

"It was as if nature gave me this great result and then tried to take it back."

Thanks!

Wow! Thanks to Sandwalk for the plug. And many thanks to all of you who contributed suggestions about what to read. I have a lot of freakin' work ahead. Good thing I'm still young.

I found it interesting that what many of you considered "classics" were not works by scientists who had died many, many, many years ago, but came from those still living or recently deceased. This made me wonder what defines a "classic" scientific text? And is what we define as "classic" constrained by the surrounding cultural, psychological and social context?

Also this question partly arose because, I had just read about Francis Bacon in Gould's Lying Stones of Marrakech, where he discussed Francis Bacon's Great Instauration. Bacon believed that how we process knowledge about the empirical world was by "passing sensory data through the biased processing machinery of the brain." He identified three different idols that held us back from knowing the truth about the natural world. These "idols" - deceived and created fallacies in our minds that blinded us to the Truth. Idols of the tribe, idols of the theatre, and idols of the marketplace.

Idols of the tribe include our strong tendency to measure all things against ourselves and to order nature by creating opposites (black and white, etc.) I definitely don't disagree with this.

Idols of the theatre are what limit our ability to interpret and think about empirical data because we are stuck in the muck of others. This means that our ideas, questions, patterns that we see are limited because we view them through the scientific context in which we were raised. Gould cites the example of the vulva stone/hysteroliths (a brachiopod fossil that many in the 17th century thought had an inorganic origin and looked coincidentally like female genitalia). It made me think do we have anything that in 200 years future scientists will think, "what idiots." And I wondered if model organisms have become the idols of the theatre for the 21st century.

Slow Science: The Death of the Sniper Scientist

[These ideas on Slow Science are a work-in-progress a first draft of sorts. With some help from those of you who read this post, via a challenge or support, I hope that my thoughts on the subject will eventually become strengthened and clarified. Ultimately, I’d like to see some of this published somewhere in the form of short essays.]

“Our current reality features an unslayable Goliath of commercialism, and modern scientific Davids must make an honorable peace, for a slingshot cannot win this battle…Opportunities for increasing fusion with the world of commerce surround us with almost overwhelming temptation, for the immediate and palable “rewards” are so great. So scientists go to work for competing pharmaceutical or computer companies, make monumental salaries, but cannot choose their topics of research or publish their work.”

This statement came at the end of the first chapter of The Lying Stones of Marrakech by Stephen Jay Gould. This ninth collection of his essays, originally written for the Natural History magazine, is sadly the first Gould book I’ve read. I’m ashamed to say that as an evolutionary and population geneticist, I had not read any Gould (with the exception of The Spandrels paper). Dawkins, yes. Some Haldane and a little bit of Williams, Wright, Fisher and Muller. But my reading list would largely include articles directly related to my topic and are mostly from 1990 and onwards. More importantly, I was under the misguided impression that because my manuscripts are unlikely to include anything prior to that time period, I should probably focus my efforts on learning the current literature.

[Plus, if I can't access it on JSTOR, then I would actually have to WALK to the library. Library, what's that? I thought Google was the library.]

I’m sure that I’m not alone in this sentiment.

Since meeting TheDude, I have felt that my own scientific thinking is narrow and thin and so I’ve started reading more broadly and deeply than my own narrow discipline of genetics. Hence I've started the task of going through the reading list he provided. Gould is on it.

Let’s get back to the quote. Gould used it in the context of explaining why Dr Johann Bartholomew Adam Beringer, an 18th century professor, paleontologist, and physician, fell victim to an absurd fossil hoax. Beringer believed that several fossils he had “found,” which included spiders’ webs, lizards’ eyes and the Hebrew name of God, were authentic. From a modern understanding of fossils, we would laugh at him for being such a dufus because we could see that it was so obviously fake. But Gould argues that at the time, scientists believed that simple organisms arose by spontaneous generation and debated the origin of fossils. Both Berginger’s context and his lack of scientific questioning led him to becoming a patsy.

After describing the hoax, he then makes the statement above. I think he does it for two reasons. First, he believes that the world of intellect and the world of commerce should remain separate because by their intrinsic nature they have different priorities and values. For him, paleontology has devolved into selling trinkets, like coffee mugs, T-shirts and the purchase of Sue the Tyrannosuarus Rex by McDs. These commercial items are akin to the lying stones of Marrakech, ie, they carry no trace of the science from which they were born. Secondly, Gould aruges that social context can affect scientific judgement. If the social context is commerce, then the science will become fakery, “engulfed and destroyed if we make a devil’s bargain of fusion for short-term gain.”

Gould is thinking about the more obvious fusion of commerce and science. I believe however, there is a subtler form. I believe that academic science, this could easily extent to the university itself, has integrated the principles and underlying philosophy of commerce. It is there in how we trained, who we become as scientists and how we carry out the science itself. It is my belief that the subtle fusion of commerce and science likely contributed to the demise of the Sniper Scientist.

Slow Science III - The Death of the Sniper Scientist - part a.

In my previous post, I asked you who was the man in the image. The reason I wanted to know if you knew - is because I didn’t have a clue when my officemate, Phylogenisto asked me.

Phylogenisto, is an awesome officemate because he just as passionate about his science as I am about mine. And although he talks my ear off at length about the taxonomy wars, I get a chance to shoot the scientific shit with him. Something I didn’t have at SmallUniversity. Plus, he is an example of the sniper scientist species, that I believe is becoming extinct in our discipline.

Phylogenisto is the kind of scientist who knows inside and out the families he works with at all levels. If you were to take a walk through the proverbial "woods," he could identify different species, their natural history, morphological characteristics, odd features about their anatomy, as well as the resolved and unresolved evolutionary relationships in the group. Phylogenisto spends an incredible number of hours painstakingly hand drawing images of the anatomy of the different species he studies, because in his words, “The only way to understand or intuit the function of a trait, is to draw it.” And he spends his spare time fashioning specimen collections like many of the old scientific masters, such as Alexander von Humboldt, the man in the picture.

So when I didn’t know who was Alexander von Humboldt, it clearly disturbed Phylogenisto. In fact, it upset him so much that he began to pace back and forth in front of his desk, speaking in his native German tongue.

"Umm, Who is Alexander von Humboldt? I asked.

Prelude to Slow Science III

As a prelude to my last essay on the topic of slow science, I want to know how many of you know who is the scientist in the picture below. Without looking him up, do you know who he is or can you guess? And secondly, what has been his contribution to science?

What is the purpose of a university?

Okay, thanks to those of you for your words of encouragement.

It's in the making. In the meantime, I suggest that you follow Nat Blair's lead and refresh your memories by reading these posts.

Or you could just chew on this quote from Alfred North Whitehead,

"The tragedy of the world is that those who are imaginative have but slight experience, and those who are experienced have feeble imaginations. Fools act on imagination without knowledge; pendants act on knowledge without imagination. The task of the university is to weld together imagination and experience."

Ugh.

I'm feeling totally chagrined because I promised to write that last post in my Slow Science series.

I need your help, peeps. Prod me.

Slow Science Gets the Shaft - Part Deux

Earlier this week, I had an encounter with a collaborator of mine who is an established researcher at a well-known university. After I recounted my tale of experimental woe he said to me, "You should pursue this it sounds like a really interesting problem and could turn out to be very cool." My answer, "I would but I don't really have the time. My supervisor Dr.Add'EmUp has to apply for a grant soon and he would really like to have this experiment finished and the paper submitted." He nodded, acknowledging the situation and said, "Yes of course. Of primary importance is getting that paper done."

This anecdote is my way of saying, that as academic scientists we make choices on a daily basis to pursue what is expedient at the cost of what may turn out to be interesting, all because of the lack of time. This rushed time frame creates an environment that does not support slow science. And it made me think back to my first post on Slow Science Gets The Shaft: Part 1. And the supposed follow-up that I said I would write and never did. Well folks, here it is. Part 2. Albeit, terribly, terribly slow (bad pun intended) to arrive. It’s a novel – so get yourself a cup of java and a healthy gluten free muffin and sit ‘er down.

In February the Lenski lab celebrated the 50,000 generation mark of their long term E.coli experiment. This experiment was started in Feb 1988 with a single genotype or clone (not a single microbial cell). From this single clone, 12 replicate populations were grown in 12 separate liquid environments (12 flasks with Davis Media broth supplemented with glucose and citrate). The lines are identical, except for a neutral marker that distinguishes six of the lines from the other six. Once in the flask, the populations are grown at 37°C for 24 hrs. After 24h, a subset of the population from each of the 12 flasks are transferred to a new flask with fresh media and the whole growth process is started anew. Furthermore every 75-500 generations (depends on which paper you read), samples are frozen down. These then provide a fossil record with which to ask what were the changes and how many occurred over time, etc. In 24h bacteria, divide approximately 6.67x, which means that Lenski and his students/postdocs have been doing this for every day for 7496.25 days or 20 years.

Lenski is a fantastic evolutionary biologist and a visionary. His experimental designs are awesome. He’s TheMan. If you look at the list of former students and postdocs that have come out of his lab, it reads like a Hollywood’s who’s who in evolutionary biology. I realize the term visionary might seem a little extreme to many, but it’s not. The reason is because Lenski had the foresight to recognize that what’s interesting and unpredictable is found, not in the short term, but often emerges from a long-term pattern. Although many organisms can undergo adaptive change in a relatively few generations and strong selection creates observable differences among populations within a species, it is only through the long-term changes that we can really understand what processes were relevant. As Conway Morris has said, “The possible evolutionary routes are many, but the destinations limited.” As a young academic, Lenski had to invest time and money to follow his curiousity about science, in a way that today’s young tenure track academics, limited by the drive to get tenure and funding, can’t do.

Did it pay off? Yes, of course. Work from these populations demonstrate parallel phenotypic evolution, changes in morphology relative to the ancestor, the evolution of increased DNA supercoiling with parallel changes in gene expression profiles, and the evolution of mutator phenotypes. But there are two major findings that came out only after the experiment was run for 20 years. First last fall, this paper came out in Nature. It showed that the rate of adaptation, as measured by the number of beneficial mutations accrued over time, exhibits a clock-like regularity. This clock-like behaviour is often expected from neutral evolution but not necessarily from adaptive change. Lenskiites were not the first to show this surprising result, Wichman and colleagues (2005), demonstrated a similar result with a bacteriophage growing in a chemostat for 13,000 generations. Whether macroevolution is nothing more than an aggregate of many small events, as Sean Carroll (2007) suggests is only explainable by experiments that quantify those events over the long term.

The second very cool result was that a key innovation happened in one of the replicate populations. Typically, under oxygen rich conditions, E.coli eats and metabolizes glucose (its carbon source), with no ability to use citrate as an energy source. Well guess what? In one of the replicate populations, a citrate-using genotype finally evolved at generation 31,500. That a key innovation evolved so late in the experiment, is telling about the importance of doing long-term studies.



In 2002, Peter Grant and Rosemary Grant published a 30-year study that showed how the direction and magnitude of selection fluctuates wildly over the long term. Environmental change and infrequent hybridization led to a phenotypic trajectory in the Galapagos finches that was not predictable in the short term. Both Lenskis work and the Grants study, however pale in comparison to the Park Grass experiment started by John B. Lawes and Joseph H. Gilbert in 1856 at Rothamsted, Hertfordshire, England. This experiment is the longest running ecological experiment in the world and over 170 publications have come out of it. Although it was started to test how different fertilizers would improve yield, it has since inspired new ecological theory (resource ratio hypotheses), demonstrated long term population dynamics related to life history not detectable over a shorter time period, and provided examples of local adaptation, reproductive isolation and drift. More importantly what this experiment and the two other long-term studies show is that these types of experiments grow in value with time. Although conceived to investigate one scientific question, they can be used to answer a multitude of interesting and often unexplored areas.

The benefits of long term studies like this one seem obvious and yet it is no surprise that they are rare. In fact, in the book The Clock of the Long Now, Stewart Brand laments that science today “is more often driven at a commercial or even fashion velocity than at the deliberate pace of governance or the even slower pace of nature. “ He offers seven reasons for why more scientists are not performing this kind of research.

1. Long term studies aren’t about proving or disproving hypotheses.
2. They don’t generate quick papers, the coin of science
3. They bear no relation to scientific fashion, where the excitement is
4. Not subject to money making patent or copyright.
5. They die when the primary researcher dies.
6. Extremely difficult to maintain funding
7. Archives are expensive and a hassle to service and keep accessible.

There are many short term studies that don’t use hypothesis driven research. We only need to look to the new discipline of bioinformatics to see examples of research that look for patterns in DNA sequence and expression data. Furthermore, I disagree with his last point. In such a computer advanced and internet driven society, the database and archival capacity of computers is enormous and the internet makes accessibility less of a problem. I think, that what explains why we don't see many long term experiments, is largely due to the structure and incentive model of granting agencies and academic institutions, specifically in North America.

Long-term experiments or studies require a scientist who is patient, thorough, and slow. The superstars in my field are anything but slow. Instead, as Brand states they tend to “track noisy signals too closely and confuse themselves by making changes before the effects of previous actions are clear.” In other words, publish one paper and then six months later publish another renouncing the results of the first. Why? Because that is how the game is played. The current game rewards prolific at the expense of being profound.

(Although I've heard the common refrain from faculty that some members of a search committee do look for quality, I wonder how many of them have actually read any of the papers from the job candidates. And if that assessment of quality is based on the journal's impact factor or the faculty member's own assessment of the candidate's science? It seems to me that there is a clear unwritten understanding that not every paper that gets into those high impact journals is actually profound and quality science. Here again the time factor creates an atmosphere of rush.)

Prolific is what gets you the chance at a t-t job, grants and ultimately the sweetest of all carrots - tenure. The system in the US (maybe less so in Canada) doesn’t support patient, thorough, slow, and profound science. Here in the US, despite being hired by colleagues, a good scientist can find themselves fired by these same people in 3-5 years when they go up for tenure. Fired or given terminal contracts simply because they didn't have the requisite number of publications or a lack of external funding. But really, how many are actually lucky enough to get funding when success rates at NIH and NSF are 7-12%. In Canada, this doesn’t happen. Once hired as an assistant professor, it’s rare that you don’t get tenure. The screening process for tenure is in the hiring, as my PhD supervisor once told me.

An environment that uses a carrot (tenure) and stick (terminal contract) incentive model narrows people’s focus and destroys creativity. If you don’t believe me, listen to the facts put forward by Daniel Pink, in his TED talk (it's worth 17 minutes of your time) on the science of motivation. Why would you spend time doing science that you think is worthwhile when it doesn’t get you the carrot. Instead, the choice is obvious, you do science that you know will work, ie get you the publications, the grants, all in the drive for tenure. This carrot and stick model leaves no room for innovation and creativity. In fact, TheDude, a tenured professor at a prestigious university told me, “Academia is broken. It's out of control. Getting tenure is the part that makes it broken.” He advises his students to do whatever it takes to get tenure and then “You can start doing the science you really think is worthwhile.” But I wonder what does that say about the science you do up until that point? And really by the time you do get tenure, if all your training is focused on routine, obvious, mechanical science, will you be practiced in innovative thinking such that you will even know which questions to ask?

A second effect that this “if you do this then you will get this or else” atmosphere does is it creates and attracts a particular type of scientist to academia and selects against another. One academic I know, has said that he doesn’t participate in a project unless he sees a publication in it for himself. Cutthroat, yes. But, at least honest. I can think of several colleagues of mine who are so much smarter than the known superstars in my field, both in terms of the quality of science and the level of innovation in their science. But they won’t make it. Why? Because they don’t want to publish just anything for the sake of publishing. And, they would argue, isn’t there enough shit to wade through already? Instead they want the work to matter. They would much rather have solid, well thought out, and fully explored ideas in 3 papers than 10 papers that either test the obvious, review a topic that has already been reviewed, or just do acceptable science. Again words from TheDude, “The problem is the number of papers that anyone individual produces is out of control. If I were the king I would eliminate half the journals especially N and S and limit people to publishing only two papers a year.”

My feeling is that somewhere between the two extremes is probably the right place. Half baked ideas are okay to publish as long as the author acknowledges the limitations and the caveats associated with their incompleteness. And we definitely need innovation and profound, thoughtful scientists. After all, diversity is the stuff of evolution. And progress is only achieved when there is diversity. So the real question is can academia in its current state support both types of scientists (fast and slow) and both types of studies (short and long term)? My belief is that it cannot in its current form. What will be the effect in the long term on the quality of academic science?

“We see nothing of these slow changes in progress, until the hand of time has marked the long lapse of ages.” Darwin (1859).

I guess we shall just have to wait to see the outcome of this long-term experiment.




Interested in reading some of the papers I cited? See below:

Barrick, J. E., D. S. Yu, S. H. Yoon, H. Jeong, T. K. Oh, D. Schneider, R. E. Lenski, and J. F. Kim. 2009. Genome evolution and adaptation in a long-term experiment with Escherichia coli. Nature 461:1243-1247.

Blount et al. (2008) Historical contingency and the evolution of a key innovation in an experimental population of Escheria coli. PNAS 105:7899-7906.

Brand, S. (1999) The Clock of the Long Now: time and responsibilities.

Carroll et al. (2007) Evolution on ecological time-scales. Functional Ecology 21: 387-393.

Grant, P.R. and Grant, R. (2002) Unpredictable evolution in a 30-year study of Darwin’s finches. Science 296: 707-711.

Conway Morris, S (2003) Life's Solution. Cambridge Uni Press, Cambridge, UK.

Silvertown et al. (2006) The Park Grass Experiment 1856-2006: its contribution to ecology. Journal of Ecology 94: 801-814.

Wichman, H.A., J. Millstein, and J.J. Bull. (2005) Adaptive molecular evolution for 13,000 phage generations: a possible arms race. Genetics 170:19-31.

Slow Science gets the Shaft - Part I

This blog post will be the first of a three-part series on my ideas of slow science.

“False facts are highly injurious to the progress of science, for they often endure long; but false views, if supported by some evidence, do little harm, for every one takes a salutary pleasure in proving their falseness.” Charles Darwin

Today, we had a seminar presentation by an "old school" scientist who told us some amazing stories about a group of organisms that he had worked on since the 1950s. It wasn't a slick powerpoint talk with fancy slides, a simple one with pictures of the different representative species. With each picture he told us about key innovations in the group, what these things eat, their ecology, morphological differences, predatory behaviour - in other words basic biology.

OldSchool is a naturalist. He doesn't make fancy models, nor do sophisticated statistics on his data. But he knows everything about the group that he works on because he has been accumulating data slowly and over the long-term. Fifty years is a heck of a long time! Instead of chasing the "sexy" and "cutting edge" questions that happen to be the hot items that year, he's a "sit and wait" scientist that lets the interesting questions arise from what he observes and experiences with these organisms in their own habitats.

OldSchool does slow science and I think that this breed of scientist is going extinct to be replace by Fast'nFurious scientists; all of whom clamber over each other to get papers in Science and Nature. Crabs in a bucket. Frankly, given the rewards, ie a scientific career that is seen as set for life, why wouldn't they?

Often to get those high impact papers, I think many choose to work on model organisms because of the cost-benefit ratio. Given the nature of the academic treadmill, we don't have the luxury of spending time with a system to acquire basic natural history questions because that takes too long. And if you choose that road in my field, it can often mean fewer publications and papers with "less impact." Ultimately leading to fewer job opportunities and less funding. And universities value scientists directly in proportion to how much money they bring in, i.e. $$$=good little scientist.

And really I am as much a party to this game as anyone. Although, I worked on an organism during my PhD, whose first name was Large and whose last name was Slow, when given the choice to work on a similar sort of organism or switch and work with a small fast one, I opted to work on a small and fast one for my postdoc. (The humour in how the size of my study species mirrors the school I'm at, is not lost on me. LargeandSlow at LargeUniversityInCanada and SmallandFast at SmallUniversity in SmallTown America.)

Well, honestly I thought - a fast and small organism will result in more publications.

I think if OldSchool were to apply for a job now (with the same qualifications he had when he started), I don't think his application would even see the light of day. And, in my opinion, that would be a huge loss to science.

The seminar today made me wonder if our focus on fast science will impact our understanding of the natural world. By fast science, I mean a few different things: what we study, how we set up experiments and for how long we run them. The focus of this blogpost will be on what we study.

Much of the work in my field has been conducted on species that are small and fast: Drosophila, annual plants, viruses, etc. These species are easily amenable to field, greenhouse and/or laboratory research. And because they have short generation times, experiments can be conducted in a timely manner (i.e., completed within the timeframe of a Masters or Ph.D). This is not to say that people don't attempt to work on LargeandSlow species, but there is a lag in the payback.

Okay so before I go all postal on the fast organism I need to demonstrate if there really a bias in what we know. Do we have equal information on organisms with vastly different generation times?

I did a quick little survey, nothing I would ever stake my scientific career on, but it yielded some interesting things. In the Wiley InterScience Life Science Search page, I did three types of word searches. The first was simply finding the total number of articles published in Wiley journals for a given organism (eg "bacteria", "Drosophila").

The graph below shows what I think we all know is obvious. Of the total number of articles written, most of our knowledge is on the following organisms: mouse, fish, and bacteria. Not surprising really that we have a strong bias toward biomedical and applied research.

But in my second search instead of just typing in the name of the organism, I used the following keywords: “bacteria and ecology”, “bacteria and evolution. ” The results were much the same. The rank order was different (fish, mouse, bacteriophage, bacteria), but the shape of the curve suggests that even in ecology and evolution, research is focused on model organisms with a short generation times. There are 136X more articles on virus ecology and evolution, than there are about a deciduous tree.

As part of the MTV generation, I understand the desire for immediate gratification. This need for immediate results and productivity is heightened under our current climate of publish or perish without any money in an unmarked grave.

But I think something is lost when what we know of the natural world is observed using a lens that is made up of organisms with “easy” life cycles.

As Charles Darwin said, “...it is always advisable to perceive clearly our ignorance.”