The dark side

Previously I referred to my experience on NIH study sections as “From the inside” and “From the other side“. Since I designated our science review officer “He Who Must Not Be Named*”, it’s only appropriate that my latest study section experience be “The dark side”.

He who must not be named

At this week’s study section, the SRO showed up with a completely shaved head. Coincidence?

After the last study section I reviewed for, I gave a talk to my colleagues back home about the AREA grants. These are special grants NIH reserves for smaller institutions that are less competitive in general for NIH funding. The goal of most NIH grants is to fund good research. AREA grants have three goals: 1) to fund good research, 2) to improve research at these underfunded institutions, and 3) to encourage students to pursue a career in biomedical or behavioral sciences. Many of the AREA applications we reviewed completely missed #2 and especially #3. In my talk, I emphasized the importance of knowing the goals of the funding agency or funding mechanism.

I was particularly interested in the AREA grants that we reviewed this week. I was assigned to a couple of these, but I took a look at the others, checking specifically for student involvement. During the study section, I brought that up repeatedly whether I was an assigned reviewer on that grant or not. We’d had a new document this time to help us review these grants, but I’m not sure all the reviewers had noticed and read it.

The last time I reviewed, I made an effort to read all the grants, not just the ones I was assigned to review. I mostly succeeded. This time, life got ahead of me. I managed to get four of my five assigned grants reviewed before I left for a week on Biking Across Kansas, but I was hard pressed to get my fifth grant reviewed before the deadline, the day after I returned from Kansas. I had no chance to read the other grants, except for looking through the AREA grants for student involvement.

Let’s talk again about scores. I gave an overview of how we assign scores in an earlier post. After discussion, scores may change. We had a couple applications with a wide range of scores, which meant that some of the reviewers thought highly of the application and others thought less of it. The discussions were intense and almost heated, and ended with the reviewers maintaining their original opinion. That was, in my experience, a little unusual. Also it was fun.

The scoring system is 1-9, with 1 being the best and 9 the worst. Mostly the full range is used, with 9 being the rarest. For the best applications, the ones that will be funded, the scores are appropriate. I did feel that there was a tendency to gravitate toward the middle of the range, so that a wide range of quality of applications could get a 5 whereas some might deserve a 7. However, neither 5 nor 7 is a typically a fundable score so I don’t think that would make a difference in whether the grant was funded.

On one grant, the initial scores were 2, 2, 3, and 4. After discussion the scores were revisited: 4, 4, 4, and– the fourth reviewer said “1. No, just kidding, 4.” This is what passes for reviewer humor.

Although some grants are designated “ND” (not discussed), so as to not waste our time on grants that are not competitive, any reviewer who wants to discuss a grant can request that it be discussed. That happened for one grant this time.

I realized recently, talking to someone who has served on study sections for another NIH institute and for other types of grants (I have not reviewed R01s), that some of the details I’ve described are specific to the study section I serve on. However, the general operation applies across the board.

On the training grants, I feel a little frustrated by what I see as unrealistic expectations of K01 (a training grant designed to provide a “bridge to independence”) candidates. If a candidate was too new a postdoc, reviewers suggested the candidate should be applying for a different mechanism. If a candidate was too long in the tooth, reviewers suggested the candidate should be applying for an R21 or an R01. I wondered what the expected experience level of a postdoc is supposed to be for one of these. It seemed to me an applicant must be 3 and no more than 3 years from their PhD. However, no candidate was “perfect” so these factors affected their scores fairly equally.

Once again, I enjoyed reviewing the grants, but the distraction of Biking Across Kansas and a spot of poor health immediately following that trip and during the study section attenuated my enjoyment of the process.

I had one minor complaint, to which I now devote an entire paragraph, which is eerily like our reviews of the applications. (“This application is fantastic! Well done! Just one teeny tiny thing…”)

I was a little resentful that NIH will no longer provide refreshments during the meeting. I found the availability of fresh fruit all morning essential in helping me concentrate. The hotel generously provided free breakfast and I stocked up on fresh fruit, so I got by. But it seems like a short sighted, bureaucratic, and ineffective attempt to save money. A few months ago when I heard about this change, I filed my objections as far up the ladder as I could reach. My opinion was noted, but the policy is too general–it applies far beyond NIH. So it’s political. It’s too bad because I think the review process suffers. But that’s bureaucracy for you, and NIH is nothing if not bureaucratic!

*My SRO appreciates his new title.

Posted in Uncategorized | Comments Off on The dark side

It’s not really insanity

At least in the life sciences, research involves an awful lot of trying the same thing over and over again hoping for different results. Albert Einstein said the same thing about insanity. Insanity = science? It could be.

Here is a brief background of the experiment I’m attempting. In our grant we proposed the hypothesis that bending the leg back and forth would cause changes in gene expression in the spinal cord. (The rationale for this…never mind, let’s not go there.) Before I joined the project, a genome microarray of 50,000 genes showed changes in gene expression in 180 genes. (Again, let’s not go into the probability that three rats would have changes in gene expression if you looked at 50,000 genes even if you did NOTHING to them. Like I said, all this was done before I joined the team.)

Microarray results have to be confirmed with quantitative RT-PCR. RT-PCR stands for Reverse Transcriptase-Polymerase Chain Reaction. Basically that means you take a strand of RNA, create a DNA copy, then copy that. When you copy DNA, each cycle doubles the  DNA. The “quantitative” part means that a bit of fluorescence labels each DNA molecule and a machine reads how bright the fluorescence is at the end of each cycle. All this is converted back into relative numbers so we can compare the amount of RNA that was in the original samples.

Ideally, in the end we are able to say, “This sample had more of this RNA than that sample did.”

My explanation is probably too complicated for most of my readers to follow and too simple (ie inaccurate) for the biosci readers.

All that is still a long way away for me. As each experiment failed to show any RNA at all, I peeled it back step by step. Frustrated at inconsistent RNA yields, I finally gave up using the procedures that had been used by the team, and ordered stuff I’ve used before successfully. As in, I’ve published on it.

I got good RNA. (Qiagen’s RNeasy. Great stuff.)

I moved on to RT-PCR, just the regular RT-PCR, no “q”. Nothing. I tried different primers. At this point I’m not even trying primers for the genes I’m trying to confirm. I’m just trying to get control primers to work.

I ordered control RNA, control primers, and GAPDH primers. I don’t care about GAPDH, and I don’t want to use GAPDH as an internal standard. I want to use a ribosomal RNA, such as Rpl27, as an internal standard. The reason I ordered GAPDH primers is because that is one thing I know every cell in every rat has. If I can find GAPDH in my rats, then I can focus on what’s wrong with the primers for the genes I care about.

(Sorry for the non-researchers who don’t know what primers are. Don’t worry about it. I think you’ll get the point of this article anyway.)

After repeating an RT-PCR over and over again, it finally happened. I got different results.

That’s right, ladies and gentleman. Three lovely bands. Lane 1 is the new RNA with GAPDH primers. Lane 2 is the old RNA with GAPDH primers. Lane 3 is control template and control primers.

Today I did another RT-PCR, testing 5 Rpl27 primers, GAPDH, and a negative control that had GAPDH primers but no RNA.

I got two, and only two, lovely bands. One in the GAPDH lane, and one in the negative control lane.

This probably isn’t clear to non-scientists. “Negative control” means there is NOT supposed to be a band. Clearly I have offended the Flying Spaghetti Monster.

To be quite honest, I don’t exactly do the same thing over and over again expecting different results. In one of the experiments I suspected one of the pipettors was sometimes malfunctioning. I had been doing this upstairs in the lab that has the thermocyclers. I switched to doing it in my own lab with pipettors that I trust. I checked every time I pipetted to make sure there was an appropriate volume of liquid in the pipet tip. And that it expelled.

Another time I thought the gel was questionable. One of the labs upstairs uses TBE. The other one uses TAE. I had made the gel with TBE, and I suspected that the electrophoresis chamber had TAE in it. I made up new TAE, emptied and rinsed the electrophoresis chamber, made a new gel with the new TAE and filled the chamber with the new TAE.

This latest result, where both my positive and negative control had bands? Although I thought I had been extremely careful, I don’t remember with 100% certainty that I put water and NOT TEMPLATE in the negative control tube. So I’ll do it again…and hope for different results.

On the other hand I’m pretty darn confident now that NONE of the five primer sets for Rpl27 work. I have a couple options. I can try to monkey around with the Mg concentration and the primer concentration. I’ve used all of PrimerQuest’s 5 suggested primer sets, so I could try another primer generator software and see what it comes up with. I can assume the published Rpl27 sequence is wrong, perhaps my rats have a different allele than the published sequence (I don’t know how highly conserved it is, although I’d think fairly conserved since it is a ribosomal RNA), and pick a different gene, like Rpl13. I’ll probably order primer sets for Rpl13, and while waiting for them to arrive I’ll play with the Mg and Rpl27 concentrations.

After I eat my spaghetti and repeat the positive and negative controls.


Posted in Uncategorized | Comments Off on It’s not really insanity

Sangji’s legacy

January 2009, a lab accident resulted in the death of a lab tech. She and the postdocs who came to her aid were poorly trained to cope with accidents, and her university, UCLA, and the primary investigator, Dr. Harran, are facing criminal charges for willful negligence. This is a landmark case because it could change the way science is done.

Whether the outcome is in favor of UCLA and Dr. Harran or not, there is potential here for major change throughout academia. The case will probably come to plea bargain, and Dr. Harran and UCLA probably won’t experience in jail time.

Dr. Harran’s defense is that he was acting like any other professor would act. That is true. My opinion is that does not exonerate him. It exposes an enormous employee safety gap. Academia sorely needs to be held accountable in its labor transgressions. Safety is perhaps its most egregious violation. Paying its administrators ridiculously large salaries while paying adjuncts a pittance is another labor violation. Sheri Sangji’s death will be a wake up call to universities and professors for safety procedures, but not for the exploitation of adjuncts, postdocs, and grad students.

UCLA, and other universities, haven’t yet reacted to the wake up call by implementing more rigorous safety oversights. Individual professors would be well advised to do anything in their power to change how they operate. Not only will that save the lives of their lab personnel and protect the professor herself in the event of a tragedy like Sangji’s, but it will also prevent others like Dr. Harran from using the excuse, “This is what all the professors are (not) doing.”

Posted in Uncategorized | Comments Off on Sangji’s legacy

From the other side again

Once again I’m on an NIH review panel. I just read my post from the first time I did this, and I’m a little amazed at how similar the process was. Ok, so we had a lot of the same reviewers, and the science review officer (he who must not be named) was also the same, and the funding mechanisms that we reviewed were all the same. Still, it’s good to know there really isn’t much in the way of variation from one study section to the next.

Once again I am impressed with how NOT random the process seems, at least from this side. But it’s easy to see how random it appears to the grantees. Especially if their grant was Not Discussed. Maybe 1/3 of the grants or so are Not Discussed. They still receive summary critiques from the 3 or 4 assigned reviewers. And if any reviewer wants the grant to be discussed, it will be. So the idea that one reviewer having an off day can kill your grant isn’t really true. If two reviewers thought your grant was great, and one thought it wasn’t, and that caused your initial score to be low enough that it was designated Not Discussed, the two reviewers who thought it was great will speak up for it.

More likely though, is that two reviewers might think your grant is mediocre, and one might think it is abysmal, and no one speaks up for it.

Reviewers who are enamored with a grant will champion it. Others might voice valid objections, and the champion will argue for why this grant deserves a good score anyway.

What was most striking this time around was how many flaws even the strongest grants have. I can’t help but think that next time I submit a grant to NIH (still a far off event, as we are a long way from having enough preliminary data for that), my grant will be solid because I’ll know exactly what they’ll look for.

Then I laugh at myself for being naive. I’ll find new ways to make mistakes!

A common mistake is how people respond to reviewers. If a grant is rejected, it can be resubmitted once. The resubmission is a good opportunity. Most of the reviewer comments are fairly sound and can only improve the project. Statistics are frequently omitted from first time submissions, for example. Some reviewer comments show the grantee that something was misunderstood. That’s easy to respond to: explain (respectfully) what you really meant. And rephrase it so that it can’t be misunderstood again. Some reviewer comments you just disagree with. That’s ok too! Just explain (respectfully) why you disagree.

What I was seeing today was more than one resubmission that ignored reviewers’ comments, disagreed but not respectfully, or disagreed but didn’t explain why.

For example, suppose the reviewer said “This project is too much to be completed in three years.”
Wrong response: “No it isn’t.”
Correct response: “Aim 2 is now Aim 2 and 3. I removed the original Aim 3.”
Correct response: “I have included a timeline that illustrates that it is reasonable to complete this in three years.”

One thing that has really bothered me about grants I have submitted is when the reviewer completely misunderstands something. Did he even read it? Well, these grants are read surprisingly thoroughly. If one reviewer misunderstood something, other reviewers will catch that and point it out. We even look things up for ourselves to better understand the qualities of a proposal.

Last time, I said I was going to try to read all the grants, not just the ones assigned. I didn’t quite succeed. Of the ones I did read, I didn’t read them nearly as thoroughly as the ones I critiqued, but I was looking for specific things that I could find or fail to find easily. I still didn’t make it through all of them. I decided not to read the conference grants, or the educational program grants. They have a different critique form so I don’t really know what I’m looking for on those. (Maybe next time I’ll go ahead and critique one even though I’m not assigned to, just so I know what to look for.) There were still a few I hadn’t read by the time we started this morning. (I read them while we were reviewing the conference grants and the educational program grants.)

As we were reviewing, I came up with a strategy for next time. I’ll read my assigned grants & critique them. Then I’ll wait until the initial scores are in and only read the ones that are likely to be discussed. Plus any others that catch my eye.

On the other hand, reading the ones that are Not Discussed is a good way to get a feel for the range of scores.

Posted in Uncategorized | Comments Off on From the other side again


Research isn’t done in a vacuum. We rely on the knowledge already out there to decide what questions to ask and how to find answers. While it is possible for a research idea to spring up out of nowhere like Athena bursting forth from Zeus’ head, more commonly even if the question is novel, we use established methods to find answers. During today’s morning run, I was thinking about the many times I’ve based a large project on published evidence and methods, and nothing worked ever.

A responsible mentor will encourage grad students to use established methods and not try to do anything ground breaking and risky, because they risk never graduating. Not just use  established methods, but methods that have been used successfully in his or her lab, so the grad student can learn it from people instead of from a paper. A researcher further along in his or her career can afford to take that risk.

When I was a grad student my mentor suggested a project that required a method we hadn’t done in our lab, but a collaborator was doing successfully. She had published a couple papers on it. We spent a day in her lab and went home and I tried to get it going in our lab. We had a phone meeting or two with her, and some emails, so she was continuing to help us. Then in the middle of it all she quit her job. We never did get the experiment quite the way we wanted it, although we did manage to get some data from that project. Fortunately that wasn’t my only project, so I had other data to fall back on for my dissertation.

As a postdoc I came up with another research question and dug into the literature to find out how best to answer it. There were a few somewhat questionable models of this disorder, nothing well established and they didn’t model the disorder very well. I found a couple reports, from a German lab, that seemed promising. I tried to contact them but they never responded. I designed a project around their method anyway, got a grant, and started trying to test it. It didn’t work at all.

Next I tried another method to test a different disorder. A researcher at a university I had been at had developed this one. I knew of her, if I didn’t know her personally. I tried it out exactly as she described in her paper (which had been published in a high-impact journal). It didn’t work as she described. I called her and asked if she had any suggestions. Her response: “We’ve been having difficulties replicating our results.” Her own lab couldn’t repeat the study they had published in a highly respected journal!

When that sort of thing happens, should the authors retract their original paper? I don’t think so, because the data are the data. Statistics aren’t perfect, they are 95% perfect, or 99% (depending on where you set your level of confidence). They didn’t do anything wrong in the original study. But they should publish the follow up study too. Unfortunately, no journal is going to want to publish that. Journals will publish “Hey we’ve got this new thing!” or “We have evidence that the other guys were wrong”. But they won’t publish “We have evidence that shows the last paper we published is wrong.” It suggests that you didn’t know what you were doing last time, and if you didn’t know what you were doing then, why should they trust that you know what you are doing now? Even though it happens to everyone.

You’d think after 3 papers like that I’d have learned. I certainly felt burned, and felt like I’d learned something. I’m very cautious about other papers. But nonetheless I got in a situation where I was taking over a project that was designed long ago, based on published data and with the cooperation and assistance and advice of the lab who published that. Should be safe enough, but the original method they had published on hadn’t worked in our team’s hands. The team came up with another method, one that hadn’t really been used for this. (In fact I’m not sure where the idea came from.) It seemed, in theory, a much better method for many reasons. But it hadn’t been published on in this context. The first experiment, with just 8 animals per treatment, worked great. The treated rats performed better than the controls. Then the next set of rats came in and the effect was the opposite. They repeated the experiment on 5 sets of rats. It was at that point I joined the team and started crunching the data. There was absolutely no effect of treatment.

We never did get what we hoped for out of that study. We couldn’t find a valid positive control, among other things.

My initial reaction is to wonder how I can avoid this in the future. The German group that refused to speak to me is a big red flag. And always consulting with the authors long before I even start the project is another step, so long as they are honest with me about little things like “We haven’t been able to replicate our PNAS results”. But some of my experiences were with collaborators. They didn’t do anything wrong, but we still ended up investing a lot into projects that went no where.

My second reaction is to wonder if this isn’t part of a bigger picture. Something to do with too little rigor on publications. Not enough statistical training for authors or reviewers. A result of the pressure to publish or perish. When the result of a rejected paper is a dead career, authors fight hard against that rejection. I know, I’ve been there.

Posted in Uncategorized | Comments Off on Unreliable

Interview of a Researcher

I love getting interviewed. Mostly I get interviewed about bicycling. Sometimes I’m interviewed about science culture. Less often I’m interviewed about my research. Interviews have been for tv, newspaper, articles, and student projects. Today I was interviewed by a student in a Master’s of Public Health program for her research class. The assignment was “Interview a researcher”. I had a grand time talking for an hour and a half. She sent me the questions the day before so I had time to think about it.

Q: How do you start a research project?
A: Ideally it starts with a question. Then you spend a long time whittling down the question into one you can answer. For example, “How do I cure cancer?” first becomes “Can I cure cancer with crystals?” Since cancer is many diseases, you further refine that to “Can I cure estrogen responsive ductal carcinoma in situ with crystals?” Of course you can’t just go around giving crystals to women with breast cancer– suppose the question you started with had been “Can I cure cancer with toxic sludge?” You can’t give crystals to women to cure their breast cancer until you’ve shown that it works in models (animal models, or cell culture) AND you’ve shown HOW IT WORKS. So those crystals are never going to be in a clinical trial until you’ve gotten past “Magic” and moved on to “crystals selectively induce apoptosis in tumor cells by upregulating caspase 3”.

However, all of that is merely the ideal start. The latest project I’ve started began with 4 tools we had available, then we crafted a question that could be answered with those 4 tools. I have expertise in reproductive physiology and rat models of disease. My collaborator has expertise in cartilage and post-injury osteoarthritis. Our institution has a special interest in and promotes research in manual therapy and mind-body which includes exercise. Put these together and you are testing the efficacy of manual therapy and exercise to prevent post-joint injury osteoarthritis in a female rat model.

Sometimes it starts with the question. Sometimes it starts with the tools. One of my thesis advisers liked to tell the joke about the drunk looking for his keys under the streetlight. Another fellow starts helping him look. They look and look, and don’t find the keys. Finally the other fellow says, “Are you sure you dropped them here?” The drunk answers, “No, I dropped them over there, but this is where the light is.” That describes research very well. This technique won’t answer the question we have, but this is what we have. This is where the light is.

In grant-ese terms, it’s a balance between the significance of the research question, and the feasibility of the proposed research.

It’s great to collaborate and get ideas from other people. Sadly, some people start their research project by stealing someone else’s idea.

Q: What specific tools do you use?
A: I prefer Medline for my literature searches. It’s really important to find out what has been done in this area and how it was done, before embarking on any project. That will keep you from falling into the same traps as before, or re-inventing the wheel. Or re-inventing the wheel and then rolling it headlong into the pitfall, to completely mix the metaphors.

More importantly, talk to the people who have published these papers. I have made numerous contacts, usually by phone, sometimes I start with an email, to people who have done something similar or worked with similar approaches to what I want to do. I can’t stress this enough, how important it is to talk to other people.

The statistician is one person you should consult…BEFORE the project. Our statistician has in her email sig this quote:

To call in the statistician after the experiment is done may be no more than asking him to perform a postmortem examination:  he may be able to say what the experiment died of. –Sir Ronald Fisher

This came up yesterday. She very tactfully and gently explained that we have to throw out an experiment because of a design flaw. I said, “This is where we should have talked to you before doing the experiment, like in your sig!”

Q: How do you gain your expertise with the various tools you use?
A: It’s important to keep up on new technology. I find the older I get, the harder this is to do. It’s so easy to get set in your ways. You find something that works and why take the time to learn something new? But this is a trap I see a lot of older researchers fall into. Nowadays it’s all microarrays and high throughput, and they’re still doing old school research one gene at a time. You have to learn new things, and you have to keep up your skill at learning new things. Your expertise isn’t enough to carry you through more than a few months.

Q: What are some important experiences you suggest for a novice researcher?
A: Fail, and laugh at yourself. Fail again and again and again. Probably the most important thing my PhD has given me is the ability to make dumb mistakes with confidence and without losing any self worth over them. A year ago I presented data at a meeting and someone said, “Did you pH the saline? Because anything acidic will cause a pain response.” Oh. I don’t know if I can explain to a non-scientist how that is such a basic, rookie, stupid mistake. It would be easy to feel ashamed of having missed such an obvious thing, to try to hide that I had done that. I didn’t do that. I freely owned up to it and repeated the experiment with properly pH’d saline. I called it what it was–a dumb mistake.

Aside from dumb mistakes, most of research is failure. What’s the legend about Thomas Edison and the lightbulb, that he tried hundreds of filaments before finding the one that worked? (A quick google search reveals that he tried 100, 1000, or 10000 materials. Or 6000. Whatever, the point is he had a LOT of experience at failing.)

Again putting aside the experiments that “failed” because the data was negative, there are the failures in techniques. These are a little different than the dumb mistakes. These are when you try a technique for the first time. I have a theory that if an assay works the first time I do it, it will never work again. So far I have not had an opportunity to test my theory. So, plan to do a dry run a few times (depending on the expense and time commitment) before testing something.

In research, most of success is failure. Only we prefer to call it “negative data”.

Laugh at yourself. Don’t take your mistakes and failures personally. Everyone makes mistakes. That was a mantra for me, for several years as a grad student, because I would beat myself up over every mistake and it felt horrible. Everyone makes mistakes, everyone does dumb things. (That used to be a GEICO commercial. “That’s why there’s GEICO.”) Eventually I internalized the message because now I wear my mistakes with pride, as an example to everyone around me that they don’t need to be ashamed of their mistakes either.

Q: If I wanted to be learn how to become a competent researcher, what specific tools would you suggest I work with?
A: Other people, a statistician, failure, and humor.

Posted in Uncategorized | Tagged , , | Comments Off on Interview of a Researcher

My ideal student, or Cultural conflicts

My lab is a small lab. I don’t have any graduate students, postdocs, or research technicians. I have me and a handful of students. Two undergraduates and one medical student. They undergrads are due to graduate this year. The medical student is a 2nd year and will be moving on soon for her 3rd and 4th year clinical rotations. That means I have to replace all three of them. I’ve been giving some thought to how I will do this.

I have some experience now in the process. My experience supports the hypothesis that there is no fool proof method. I can interview a lot of students, I can put out an announcement on a board, I can only go with word-of-mouth. Most of the time I get pretty good students. Once in a while I get a great one. Once in a while I get a bad apple. Knowing that no matter how thoroughly I investigate the applicants, I might get a bad one anyway frees me up to just do as good a job as I can and make the best decision I can.

Usually there’s a few that are obviously not a good match. You might think it’s because the person is such a low achiever that I just throw out the application. But more often it’s because the person is such a high achiever. With several scholarships, awards, activities, and offices held, I don’t see how such a student has time to be a research assistant, or how that student would be motivated to do it. The overachieving student doesn’t need the research experience because her resume is packed. So I tend to shy away from them. That would have made me very angry when I was a student, if someone took that approach and I found out about it. I was one of the overachievers.

In preparing for this upcoming round of interviews and selections, I wrote up a description of my ideal student. I’ve learned not to publicize all the characteristics of my ideal students, because they take them so literally. I only meant it is my ideal, not that these are requirements! Yes, my ideal student is in town over Christmas and spring break, but how likely am I to find someone like that? Not very, so go ahead and apply. If you are a better fit in other capacities I might take you over someone who never leaves Kirksville. To tell the truth you’d have to be a little bit weird to stay in town during all the vacations so I’d want that one anyway. (It is slightly possible that I don’t know what I want.)

I went ahead and pulled out all the plugs as I wrote up this description. It was for my eyes only, not a job posting, so I didn’t worry about saying things like “I prefer this kind of personality” or “I want someone who likes to bicycle”. I do prefer a somewhat outgoing personality, because it’s a bit of a struggle for me to connect with the reserved, shy ones. They make me feel loud and awkward. I love to talk about bicycling, so I can connect really easily with a student who likes to bicycle.

Of my three current students, one of them is quiet and bicycles, and I’ve connected well with him. The others aren’t as interested in bicycling, but they don’t mind hearing about it, and they aren’t shy, so I’ve connected pretty well with them too.

From this I concluded that it is important for me to feel like I have connected with the students. It puts me at ease around them, which lets me communicate with them much better. Regardless of the specifics–personality, or personal interests–the important thing is that I can connect with the student.

There was another trait that came out on my “for my eyes only” that troubled me quite a bit. I’ve worked with or around at least a dozen scientists from India/ Pakistan during my career. With very few exceptions, they’ve caused or had problems. I’ve heard horror stories from their labs, I’ve experienced the horror stories in their labs, I’ve had troubling interactions with the ones who were my peers. As I was writing this, I realized that my ideal student is probably not from India.

That bothered me. A lot.

It prompted me to ask why that is. Why do the Indian scientists have troubles in the US? Is it cultural? (Of course it is.) What is it about the culture? Is it something I could learn to work around? I started feeling curious about it. I thought, if I did take an Indian student, I would do it knowing that she would be a lot of work, that maybe I would learn more about the culture and where is the friction between that culture and ours. Maybe we would successfully figure out how it could work. Or maybe it wouldn’t work out, but I would have learned a lot.

In thinking about the many interactions I’ve had with Indians, some of them moderately uncomfortable and some of them explosively uncomfortable, I suddenly noticed a common theme. Indians (in general, it’s a big place and not uniform, anymore than US culture is uniform) value the need to save face. Saving face is very important to them, and they understand another person’s need to save face. In the US, honesty is valued more than saving face. Even if they knew the person was lying to save face, Indians won’t call them on it. Furthermore, they’d judge anyone harshly for calling someone on their face-saving dishonesty.

This is pretty hard to swallow for an American. We’re proud of taking each other to task. We like finding out and publicizing someone’s dishonesty. We like bringing someone down. (Again, generalizations. Obviously neither Indians nor Americans are a single culture or personality.)

Being able to see this cultural trait without judging it is really important if I want to understand how their culture interacts with mine. I can’t judge either culture. I can’t say that Indian culture is bad for prizing saving face above honesty, nor can I say that American culture is good for being honest. I can’t stop there, I have to look at how part of American honesty is this delight in crushing other people. I have to see how the Indian respect for saving face is a kindness to someone who has failed. But again, without judgment. Neither the American honesty nor the Indian saving face is bad or good. It is just how it is.

Now I’m nearly keen to take on an Indian student, to learn more about the cultural friction, to gain a deeper understanding of my own culture. That is one of the things I love about science. For all that the internationals are flooding the PhD job market, I love it because it’s a little like getting a study abroad experience without ever leaving Missouri.

Posted in Uncategorized | Tagged , , , , | Comments Off on My ideal student, or Cultural conflicts

Negative data and positive controls

When I first came to my institution, I inherited a funded project. The project had foundered because no one had the time to work on it. In theory, I had all the resources I needed, the project just needed someone (me) with time to pursue it. The most daunting hurdle was that the project was in a field I knew very little about. The project team had a lot of the knowledge and expertise, so it turned out that I didn’t really need to become an expert in the field. At least, the most pressing reason to do so was for my own ego, because I was uncomfortable discussing a subject that I am ignorant of.

Working on this project has given me a lot more confidence in talking about something that I don’t know much about. I don’t really like it when people do that, because they tend to have opinions that don’t hold up, but I’ve gained a new trust in myself and my ability to form–and change–opinions with the available data.

My expertise in doing research with animals had been a missing component. One of the team had some experience with animals, but not a lot, and not with a behavioral endpoint.

The next challenge on this project was to coordinate. My role was chiefly as project coordinator. Everyone else had the skills to do all the pieces. I learned some skills, and I did a lot of scheduling and organizing.

Some work had been done before I arrived. I worked on analyzing that data, with the help of the statistician. The dismaying results were that our treatment (manual therapy) did NOT improve the endpoint (running behavior), after injury. The immediate conclusion one might make is that manual therapy did not work. But there was a fatal flaw in the study design. There was no POSITIVE CONTROL.

Being a trained scientist, I spotted that immediately. (That is slightly tongue-in-cheek. There are many scientists who would miss that. In fact, all of the team members involved in designing the project missed it.) Without a positive control, our results are meaningless. We can’t say that manual therapy didn’t work because we don’t know if anything would work in our model. And so I set out to find a positive control.

The obvious choice was morphine. But that didn’t work either. The half-life of morphine is 6 hours, and rats do most of their running at night. So we changed the study design such that the morphine was given right before their running. It still didn’t work. We tried two doses. We tried a steroid. We tried a prescription NSAID known as Torodol. None of it worked. At that point, with lots of data but not much to say about it, we tried to publish. One of the reviewers also missed the importance of a positive control and said that manual therapy didn’t work. Our manuscript was rejected.

We tried many, many variations on the manual therapy protocol. It didn’t matter what the positive control was, if we could make something work we had a positive control. Even if it turned out to be manual therapy, the thing we were trying to test. None of the variations worked, either.

Trying to publish negative data is difficult. I suggested we submit to the Journal of Negative Results, but my team members didn’t like that idea. (I was being serious. I think JNR is an admirable project.) Honesty, I’m not sure we could even get into JNR. They emphasize study design, and I think they would probably reject us because we don’t have a positive control. It’s not really a negative result unless it has a positive control.

Recently, a journal editor saw our poster at a conference and expressed interest, despite all the negative data. Now that we have all these variations on the manual therapy protocol, we have even more data. So maybe we will get published.

Which brings me to another ethical issue. Have you heard of grade inflation? Well, there’s also publication inflation. There are so many journals, and so many articles published every year, that the amount of data and words is immense. Even with smart search engines, digging through it to find the one gem of information you need when you need it is challenging. There is huge pressure to publish. Scientists use tricks like breaking up an experiment into ridiculously small units sometimes called the “least publishable unit” to maximize the number of publications they can get on their cv.

You should share your data, because that fosters science and collaboration and progress and gives meaning to your work. But you should not over-share, diluting the pool of publications with meaningless data or splitting up your data into puzzle pieces that others have to put back together.

Just a note, I have only seen one researcher in the habit of the “least publishable unit”, and I’m not entirely certain she did engage in it. So I’m not sure it’s as widespread as it has been made out to be. Also, combining more data into a single publication typically makes it a stronger paper, and most researchers I know want to get into the more prestigious journals.

I conclude that publishing our negative data is not unduly diluting the publication pool. On the other hand, I also believe that if we chose to withhold our data, that would be ethical, because there is not much interest in what we have to share.

Posted in Uncategorized | Tagged , , , , , | Comments Off on Negative data and positive controls

A scientist’s family

Science magazine surveyed postdocs and postdoc advisers about what makes a successful postdoctoral experience (“Postdocs: Striving for Success in a Tough Economy“). One glaring discrepancy between the two groups is the importance of spousal accommodation. 37% of supervisors and 86% of postdocs rated spousal accommodation as important to a positive postdoctoral experience. But more disturbing than this was a quote from someone who had chosen her postdoctoral position based on where her husband had a job: “If you want to have a striking career and be famous then you should choose on that basis… But if your personal life is important to you, then you need to take that into consideration.”

This type of statement makes me angry. It perpetuates the myth that to be a successful scientist you must sacrifice your family. A “real” scientist stays in the lab all hours of the day and night, takes the best job for her career, and it doesn’t matter that her spouse & children live in another city or on another continent because she wouldn’t see them anyway, spending all her waking moments in the lab. Since no human lives that way, any failure can be attributed to the scientist’s lack of commitment and dedication, not to inadequate mentoring, insufficient funding, or the enormous collapsing pyramid scheme that is academic research.

If you want to have a striking career and be famous, then you should not become a scientist in academia. At least not in the USA.

Posted in Uncategorized | Tagged , , , | Comments Off on A scientist’s family

Observer Effect

The observer effect is when the act of observation changes what which is being observed. That probably sounds much snazzier in Latin. Yesterday we conducted our first ever bicycle and pedestrian traffic counts in Kirksville, MO. I was assigned the intersection of La Harpe and Boundary. The National Bike/Ped Documentation Project recommends conducting counts between 5 and 7 pm. The main thing we learned is that this time frame isn’t ideal for that location in July. We know that a lot of people run or bike on Boundary. But not between 5 and 7 pm when it is 90F!

The observer effect came into play during the second hour. To keep our first counts simple, we had decided to count only one hour, from 5 to 6 pm. I thought perhaps the bike/ped traffic might pick up later in the evening as it got cooler, so I elected to stay until 7 pm. The answer is, bike/ped traffic might pick up later as it gets cooler, but it wasn’t getting any cooler before 7 pm.

The other person doing the counts with me at that intersection couldn’t stay the extra hour so she left. On her bicycle. I dutifully counted her. I drank the last of my water. I realized my friend who lives down the street the other way would be getting home from doing her counts at another intersection in town. I called her and asked her to bring me some water. She biked to my intersection, and I dutifully counted her. She biked out, and I counted her again.

The total count was 5 bicyclists in 2 hours. One of those was an observer, and two of them were because I called for more water. Should I count it was only 2 cyclists? No, I reasoned, because if we hadn’t been conducting counts, we might have been out riding our bikes through that same intersection. By being observers, we are taking cyclists (ourselves) off the road.

In fact we plan to go for a ride tonight. After 7 pm, when we hope it might be a little cooler.

Posted in Uncategorized | Tagged , , , | Comments Off on Observer Effect