Sunday, December 17, 2006

Academic Curiosity in a Real World: A Criticism of the Methods Used by Funding Agencies to Evaluate Deserving Research Proposals- Karl Stasko

Why does common sense often take a secondary role in public health research? Gary G. Bennett, a PhD from the Harvard school of Public Health, recently published a surprising study in the American Journal of Public Health. His research utilized a pedometer to show the effects of television viewing on the physical activity of multiethnic low income families. The results revealed that each hour of watching television corresponded to fewer steps taken, and ultimately decreased the chances that individuals will meet the proposed health goal of taking 10,000 steps a day (2006). Dr. Bennett should be commended for his good intentions; however, the surprising part of this study is that it was funded by grants and published in a respected journal. Shouldn’t it be clear that watching television correlates to less activity, especially to public health practitioners the main audience of the American Journal of Public Health? Common sense reveals that most people aren’t running 5 miles on the treadmill while they watch television, regardless of income and ethnic background. Yet, studies such as these continue to be published and awarded grants by funding agencies. There is something wrong with the system.

Public Health is dependent upon academic research and actual real world interventions. Resources are often scarce for public health departments who must compete with other services, such as fire and police departments for funding. It can be argued that public health researchers might often use grant monies for academic curiosity rather then for direct intervention. Funding agencies must change their grant evaluation policies in order to encourage researchers to solve problems, rather then to merely point them out in redundant variations. Arguments can be made that: (1)Public Health researchers often feel pressure to obtain grants
and publish from their particular institutions and peers, which doesn’t always lead to truly novel research proposals; (2)Current criteria used by funding agencies may exclude more deserving research proposals due to grant requirements; and (3)Grants are allocated to researchers for studies which are too similar to existing research; therefore, they provide no new insight to the current body of knowledge in the their respective fields.

Research is important for developing directed and successful intervention. Therefore, the quality of studies and subsequent publications are paramount to the future direction of public health.

Pressure to Publish
The pressure to publish is very real in the world of academia. The term “publish or perish” is an all too familiar cliché proving this very point. Professors must often publish works in order to obtain tenure at their academic institutions. Even graduate students in certain fields may be urged to publish in order to be competitive in the job market. This is a problem because this pressure may lead to hastily done, mediocre research, which isn’t relevant to real world interventions (Fox, 1983). In order to address this all too prevalent situation it is necessary to find out what might be driving this behavior.

Alfred Adler believed that a sense of inferiority, or more precisely, avoiding a sense of inferiority, drives many people in their life’s goals and aspirations. This drives individuals to act unrealistically and manifests itself as a desire for power and dominance. In this case, researchers might retreat from real world problems and develop self centered attitudes (Boyd, 2005). The current environment at academic institutions may very well promote this type of innate behavior. The pressure to perform and obtain tenure is surely a source of competition among professors, with publications becoming a way to keep score, and obtain a sense of prestige. The price of this game is a separation from real world issues, and a decline in useful and thoughtful research. The truth is that funding agencies are inundated with study proposals which are fueled with this underlying motivation. It is unfair to say that principal investigators have deviant intentions when it comes to being funded and published. However, it may be more realistic to say that there is a sense of malaise when it comes to pursing breakthrough proposals. Why is this encouraged by funding agencies, by awarding funding for such proposals?

Exclusion
The current criteria and characteristics of successful grant candidates reveals a
system which tends to fund a traditional group of researchers. A study in the Annals of Epidemiology states that the NIH budget has not kept up with inflation in the past three years, even though it has doubled its budget from 1998 to 2003 to 27 billion dollars (Bonetta, 2006). Researchers are still finding it difficult to obtain grants for their work. The NIH criteria includes meeting standards of significance, approach, innovation, investigators, and environment (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-05-002.html). These standards seem to encourage novel ideas and responsible research; however, a closer look at the allocation of grants reveals a striking pattern.

A 2005 study in the American Journal of Medicine found that R01 grants were
disproportionately allocated to universities ranked in the top ten by US News and World Report, most often with principal investigators who were full professors. Competing renewal applications for previously funded studies were also more often funded then new proposals. According to Druss of the American Journal of Medicine, the problem which arises from this concentration of funding is a limited scope of education and expertise which decreases the social benefits of the research undertaken (2005). There are underlying roots and consequences to this selection process.

In many situations confidence is key. Albert Bandura’s theory of self efficacy corresponds to a person’s sense of ability to effectively deal with the environment. This tends to effect the motivation and behavior of individuals (Boyd, 2005). Therefore, it can be easy to see why there might be a stigma in the grant allocation process. Researchers who aren’t in top tier institutions or full professors, may develop an attitude which lessens their confidence, causing them to believe their proposals aren’t important or fundable. In contrast to this phenomenon, professors from more prestigious universities might find more value in their own work which might be mediocre, due to their higher sense of self efficacy. How does this affect those people who make funding decisions? The answer is that self efficacy can create a bias. Decision makers might feel more confident in choosing historically prestigious candidates because it increases their own sense of self efficacy. The end result is a similar population of researchers with similar ideas. This does not question that there are great minds at these top institutions, only that there is an increase in the chance of redundancy where great minds think alike.

Redundancy
Repetition is too acceptable in research. Grants are allocated to researchers for studies that are often too alike, and provide no real useful insight. Skepticism, or a desire to extend the findings of previous research often fuels this redundancy (Mullen, 2006). The grant review process is often an arduous task, and committee members may not have the time to pick out unoriginal works, or the exposure to the thousands of other works which might bring this type of situation to light. In some cases reviewers might be experts in the field who are accustomed to certain methods of research, and are therefore less receptive to novel theories.
It is also noted in the Journal of Medical Ethics that journals are not doing enough
to criticize and exempt unoriginal repetitive works (Barnes, 2003). The consequence of this issue is a terrible waste of funds. Grants average in the hundreds of thousands of dollars. This is a travesty when unoriginal studies have no real impact, or return on investment. There is such a lack of funding that too often important research interventions are lost over time because there isn’t an adequate budget to implement the project. Clearly there would be a benefit in being more selective in allocating funds for research proposals. Certainly the funds left over from choosing only high impact novel proposals, could be used to actually push those proposals into real life interventions. There is also something to be said about grants that are specified for underprivileged or underrepresented populations. The example in the beginning of this criticism about television and physical activity conveys the idea that we should use common sense in this area. Underprivileged people are not a different species. In some cases, cultural differences may require different methods of intervention, but in others we can use existing knowledge to deal with the issues at hand. Conditioned behavior may be a reason for this repetition.

B.F. Skinner’s model of operant behavior states that behavior which is reinforced will often reoccur (Boyd, 2005). This is truly the core of redundancy. The current system rewards unoriginal work through grants, publications, prestige, and tenure. There must be a new paradigm where quality is truly celebrated over quantity. Reinforcing this ideal will hopefully change the behavior of researchers, and this reinforcement can start with a more selective grant allocation process.

Conclusion
Public health goals are focused on improving the overall health of society. In a sense society must believe in public health practitioners in order to reap the benefits of health programs and recommendations. Trust is important. However, many people might picture public health professionals as a group of academics sitting around a table, out of touch with the true public’s needs and concerns. The current grant system of funding unoriginal research only solidifies the perception of the out of touch academic elitist. Research forms the support structure of intervention. Novel high impact research creates the best foundation for action. Funding agencies should support quality, and utilize grants for real world problem solving, thus supporting the fact that academic curiosity can exist in the real world.

References
American Public Health Association. “Watching Television May Drain Time That Could
Be Spent Exercising.” Retrieved November 5, 2006 from http://www.apha.org/news/press/ajph_current.htm.

Barnes, D., and V. Yank. 2003. “Concensus and Contention Regarding Redundant
Publications in Clinical Research: Cross-Sectional Survey of Editors and
Authors.” Journal of Medical Ethics 29: 109-114.

Bonetta, L. 2006. “Growing Pains for NIH Grant Review.” Cell 125 (5) 823-825.

Boyd, M. A. 2005. Psychiatric Nursing Contemporary Practice Philadelphia PA,
Lippincott Williams & Wilkins.

Druss, B.G. “Tracking Publication Outcomes of National Institutes of Health Grants.”
The American Journal of Medicine 118 (6): 658-663.

Fox, R.D. 1983. “The Pressure to Publish: A Graduate Student’s Personal Plea.”
Teaching of Psychology 10: 177-178 Retrieved November 2, 2006
www.dennisfox.net/papers/pressure-plea.html.

Mullen, P.D. and G. Ramirez. 2006. “The Promise and Pitfalls of Systematic Reviews.” Annual Review of Public Health 27: 81-102.

“NIH Announces Updated Criteria for Evaluating Research Grant Applications.”
Retrieved November 5, 2006 from http://grants.nih.gov/grants/guide/notice-files/NOT-OD-05-002.html.

1 Comments:

Anonymous Casandra A said...

I thought you did a nice job with your paper! This issue is definitely one I've talked about many times with friends. In my naivity, I tend to hold research in a high regard because it is a means to satisfy curiousity and advocate for change and interventions; however, as you highlight in your paper, the grant system is heavily rooted in business--with intense competition and funding at its core. I do agree with you that changes and improved regulations within the grant system are definitely needed! Good job and interesting topic!

1:05 PM  

Post a Comment

<< Home