I love writing grants. I love planning the experiments, sorting out all the details, talking to tons of people and getting their ideas and improvements, looking up products and catalog numbers, dreaming big. Building hopes.
Then the grant is submitted, and if it’s funded, that’s exciting (and scary), and if it’s not funded, I love reading the reviewer comments and diving back into it to try again. I’m definitely a bit odd that way–most people hate getting rejected and despise the reviewers. My initial reaction to the comments is the same as most people’s: “Did he even READ my grant?” But my next reaction is to take on the challenge to make my grant read so clearly that even an idiot can’t misunderstand. And, while I’m at it, to incorporate the valid points.
I’ve heard that reviewers stay up late to read all their assigned grants the night before they are discussed, and then are exhausted and bleary-eyed for the discussion. Or they read them on the plane or in the airport filled with endless distractions. From the applicant’s perspective the scores may be random for all I can tell. I’ve heard that my application is read by only the 2 assigned reviewers, and everyone else on the panel just goes along with what they say. If just one of those has a bee in his bonnet, or a grudge, or a bad day, or doesn’t really read it at all, your grant doesn’t have a chance, no matter how good it was. The time of day that your grant gets discussed can determine if it gets a thorough review or if everyone is tired and doesn’t care anymore. That’s what I’ve heard.
Today I’m on an NIH study section reviewing grants and I am having a wonderful time. I loved reading my 5 assigned grants. I got really excited about one, and angry at another. I loved writing my reviewer comments.
There’s no reading the grants the night before the study section meets, because our initial scores and comments are due two weeks earlier. A minimum of 3, and often 4, people read the grant.
I was so excited when the other reviewer comments were available. Would they agree with me? It was fun to see what they thought, and it was interesting to see how our opinions changed after we saw what the others said. On the application that made me angry, someone else loved it! But I noticed later, her scores had risen*.
The comments were thorough! The other reviewers took their job as seriously as I took mine.
I reviewed my scores in light of the other reviewers’ comments and scores on my assigned grants, and decided that my initial scores stood. Then we all got to DC and discussed the grants. During the discussion, some of my scores changed slightly. Rarely, someone’s score would change dramatically. Interestingly, the assigned reviewers’ scores are often close together, but sometimes there is a wide spread. Usually when there is a wide spread, the scores tend to get closer after discussion, occasionally to a unanimous number. Other times, the scores gravitate up*. I don’t think it was ever the case that someone thought an application was bad and changed their mind to think it was good. But often the other way around.
We change our minds after the initial reading, because the other reviewers bring their own expertise, they point out things that we didn’t realize. I’m impressed with how thoroughly everyone reads these. In fact many people read all the grants, not just the ones assigned to them. (If I’m asked to review again, I’ll be doing that!)
For some grants, there is vigorous debate. Time of day does influence that a little bit, but not much. I suspect the science review officer, who asked that he NOT appear in this blog, might select applications which require less discussion for immediately after lunch. Sometimes an assigned reviewer really wanted to like the application (the underlying idea appeals, or they reviewed the initial submission), and initially scores them well, but sadly concedes the application really isn’t up to it.
Once the assigned reviewers have settled on their final scores, the rest of the panel can vote within the range of the assigned reviewers’ scores. For example, the assigned reviewers vote 2, 3, and 4. The rest of the panel can vote 2, 3, or 4. If someone wants to vote outside the range, they raise their hand and explain why (we humorously call that the Penalty).
After the meeting, the program officers consider our scores, and other factors, and make Funding Recommendations to the center’s director, who makes the final funding decision. Since I’m not privy to that process, there may yet be an element of randomness to the process.
I really enjoyed the entire process, the traveling was a necessary evil, and my opinion of the NIH review process has improved considerably.
*In NIH scoring, a low number is good, and high number is bad. Her initial score was a low number, mine was a high number, and her score increased.