|
Are You Ready for the New Forms?Here's a quick review of action items to help you get ready for the new grant applications. As we told you in earlier articles, the new applications will have a restructured Research Plan -- including a new section called Research Strategy -- and smaller page limits. Changes affect both paper and electronic applications.
These changes affect most applications with due dates on or after January 25, 2010. For some AIDS applicants there is an exception: you may use the original forms and instructions through February 7, 2010, if you're eligible for continuous submission and preparing an R01, R21, or R34. NIH clarified these dates in an October 8, 2009, Guide notice. Stay tuned to NIAID Funding News for more on this topic.
Rating Applications: Scores and PercentilesAlong with an overall impact/priority score, R01 applications reviewed at the Center for Scientific Review get a percentile too. Is that percentile a true indicator of quality? It's a complex issue.
While percentiling generally broadens the distribution of scores, two factors can skew results and even create mathematical imprecision:
Using the new scoring system, we simulated hypothetical scores and percentiles over three peer review cycles to illustrate how percentiles relate to scores and how clustering may work for the new scoring system. NIH normally uses three peer review cycles to calculate percentiles. (Read about how NIH is calculating percentiles during the first year of the transition to enhanced peer review at Percentiles Indicate Relative Rank on the NIH Grant Cycle: Application to Renewal). Please note that the factors cited above predate and are not related to the peer review changes. We are revisiting this topic to follow up on our June 3, 2009, article, "Paylines and Percentiles -- a Tale of Two 'Ps'." Case for PercentilingIn mathematics, a score's percentile rank reflects the percentage of scores that are lower than it is. At NIH that is reversed: a percentile roughly translates to the percentage of applications receiving a better overall impact/priority score than others reviewed by its study section during one year (i.e., in the same and previous two review cycles). NIH includes applications that were not discussed in the percentile calculation. NIH uses percentiling to counter a well-documented trend: study section members have given applications increasingly better overall impact/priority scores over time, resulting in scores clustered at the better (lower) end of the spectrum. Taken to the most extreme, skewed score distributions can result in scores with little meaning, giving institutes no basis for making rational funding decisions. Another reason for percentiling is that each study section can apply the review criteria differently, scoring either more harshly or more favorably. Thus we have percentiling, which despite its drawbacks, enhances fairness to applicants. Read more at Percentiles Indicate Relative Rank and How Percentiles Are Determined on the NIH Grant Cycle: Application to Renewal. Shifting SumsOur data illustrate the two main factors that distort percentiles noted above -- a clustered score distribution and a shifting application cohort. These hypothetical data depict three scenarios representing the three study section meetings used to calculate percentiles. Remember that these tables are for illustrative purposes only: do not use them to decide whether to resubmit an application. Table 1 shows the effect of score distribution on percentiles. In cycle 1, only 2 grants got an overall impact/priority score of 15 or lower, which became a percentile of 3. Contrast that result with the numbers in cycles 2 and 3. The study section clustered scoring towards the lower end: 7 and 15 applications scored 15 or lower resulting in percentiles of 8 and 13, respectively, for an overall impact/priority score of 15. When scores are converted to percentiles, percentiling can spread out scores across a broader range. This happened in cycle 3 where scores of 15 and 20 became percentiles of 13 and 24. Table 1. Overall Impact/Priority Score Distribution for Study Section A
Table 2 further shows the effect of score distribution and illustrates the impact of the moving three-review-cycle window on percentiles. In cycle 1, 10 applications scored 15 or below, and a score of 15 ranked at the 17 percentile. Compare that figure to cycle 3, in which 15 applications -- one-third more -- had a score of 15 or better. You would expect the larger number of applications to result in a significantly higher percentile as we saw in Table 1. Yet in cycle 3, a score of 15 ranked at the 18 percentile -- only one point higher than in cycle 1 -- because it was calculated using a different cohort of applications. Tables 1 and 2 also highlight the different scoring behaviors of study sections. Compared to the study section in Table 1, the study section in Table 2 consistently judged more applications in the top range, resulting in very different percentiles. Table 2. Overall Impact/Priority Score Distribution for Study Section B
Percentiling spreads out scores across all possible rankings, so you expect a redistribution when hundreds of overall impact/priority scores translate into percentiles. But the more scores are bunched together, the more percentiling exaggerates their differences. Keep in mind that we are showing scores in five-step intervals. In reality, there could be scores at each integer. The second factor that skews results -- the entry of a new cohort of applications and dropping off of an old one for each review cycle -- makes the system fluid. One-third of the base used to calculate percentiles turns over at each study section meeting, while percentiles for the earlier two meetings are fixed, contributing to a lack of mathematical precision.
Picked for Peer Review Survey? Please Participate
Now that enhanced peer review is getting into full swing, NIH wants to know how the changes -- new scoring system, revised review criteria, new summary statement format -- are going over with applicants and reviewers. To gauge reaction, NIH is using two online surveys to poll a random sample of participants on their recent experience with peer review. If you're among the thousands chosen, please take a moment to offer your two cents. These surveys are the first but certainly not the last. NIH plans to conduct more over the next several years as part of its Continuous Review of Peer Review, a component of Enhancing Peer Review at NIH.
Reminder: NIH Requires Data on Graduate Students, Postdocs, and OthersOctober 1 wasn't just the beginning of the new fiscal year. It was also the start date for reporting data on postdocs and graduate students in progress reports and some applications. We briefly touched on this in our September 9, 2009, article "The Times, They Are A-Changin'," but realize it may have flown under your radar. We bring this topic to your attention again for two important reasons: 1) reporting requirements are mandated by the NIH Health Reform Act of 2006 and 2) data will provide Congress and the public a more complete picture of NIH research training and the research workforce it supports. Get the Who, What, and Where of ReportingNow that you know why your compliance is key, here's the nitty gritty on the new reporting rules. Details on doctoral degrees
Find further details in the August 28, 2009, Guide notice. Naming personnel
For more information, read the August 28, 2009, Guide notice. Postdocs Have Something in "Commons"Speaking of postdocs who work at least one person month on a project, be aware that they must get a Commons ID. NIH announced this policy in another August 28, 2009, Guide notice.
Office of Research Training Gets a Helping Hand
Good news. A new special assistant to the director of our Division of Extramural Activities (DEA) is now on board to help out its Office of Special Populations and Research Training (OSPRT). Katrin Eichelberg, who comes from DEA's Scientific Review Program, will be with OSPRT for a year, handling various aspects of training, fellowship, and career development grants. This includes providing advice and information to investigators and Institute staff. By pitching in this way, she'll help fill the void left by former OSPRT Director Milton Hernandez, who went to NIH's Loan Repayment Program. We wrote about his departure in last month's article "Get a Head Start on Your Loan Repayment Program Application." Also in that article, we told you "Where to Send Questions and Issues for LRP and Training, Fellowship, and Career Development Awards." For the time being, you should continue emailing AITrainingHelpDesk@niaid.nih.gov. Katrin or another staff member will get back to you.
Thanks and Kudos to NIAID Committee Members
As we do each year, we are recognizing people who generously volunteered to serve on an NIAID peer review committee, Council, or other advisory group in FY 2009. To see the list, go to Thank You Members of NIAID Peer Review Groups and Advisory Committees. Check your entry to make sure it's accurate, and if you see a problem, email deaweb@niaid.nih.gov. Since we pulled the information from the NIH database, errors are possible. If you haven't served yet, be aware that being on a review committee is an invaluable experience for qualified investigators. Learn how you can participate at How to Volunteer. IOM Recognizes Volunteers on Our CommitteesCongratulations to the current members of NIAID's peer review and advisory committees who were recently elected to the prestigious Institute of Medicine.
IOM members are elected for their professional achievement and commitment to service. For more on our Council members, go to Biographical Sketches of NIAID Council Members.
Latest Concepts Get Green Light -- See What We're Planning
When it comes to unearthing potential future initiatives, you don't have to dig too deep. Just go to Concepts: Potential Opportunities where you can now find the latest concepts -- possible future funding opportunities -- our advisory Council approved at its September meeting. You also don't have to go far to learn more about concepts and how you can use them to your advantage, even if you plan to submit an investigator-initiated application. It's all spelled out in our September 9, 2009, article "How We Plan for Science Opportunities and Needs" and in Concepts May Turn Into Initiatives in the NIH Grant Cycle: Application to Renewal.
Few ARRA Opportunities Are Left
While most ARRA awards are underway, four opportunities remain unfunded, including one institutions can apply for. Beginning November 11, institutions can apply for Building Sustainable Community-Linked Infrastructure to Enable Health Science Research (RC4). This opportunity targets work on infrastructure needed for collaborations between academic health centers and community-based organizations. Applications are due December 11, 2009. And for these three requests for applications, peer reviews of applications are underway:
To view a list of all ARRA funded NIH projects to date, go to Recovery Act, part of Research Portfolio Online Reporting Tool (RePORT).
Get Free NIAID Bioinformatics Software
Find free data mining and analysis software on NIAID's Exon site. Here's a summary of what Exon offers you. PaVE: Papillomavirus EpistemeAs the latest item featured on Exon, PaVE is a resource for Papillomavirus researchers. PaVE includes over 70 complete genomic sequences of Papillomaviridae and tools for the comparative analysis of viral isolates. PaVE includes unique visualization tools for sequence information. SPICE: Simplified Presentation of Incredibly Complex EvaluationsSPICE software was originally designed as data mining and visualization software for multicolor flow cytometry. However, you may find many other uses since it can analyze any multivariate data set that has a series of nominal measurements and a single continuous measurement. dCAS: Desktop cDNA Annotation SystemdCAS automates large-scale cDNA sequence analysis. Import your raw cDNA sequences and dCAS will output an annotated Microsoft Excel file with assemblies. About NIAID's Office of Cyber Infrastructure and Computational BiologyNIAID's Office of Cyber Infrastructure and Computational Biology (OCICB) Bioinformatics and Computational Biosciences Branch developed Exon to share with the public. You can learn more about OCICB by emailing ScienceApps@niaid.nih.gov.
Leadership Position Opens at NIAID
NIAID is seeking a deputy director for its Division of Extramural Activities (DEA), which oversees operations and policy for the Institute's peer review, grants, contracts, and other extramural functions. The deputy director will work on issues involving a full range of challenging award types, including research training, small business, and international awards. To apply, you will need to send your CV, bibliography, and a three-page summary of your vision for extramural research, your interest in the position, and the leadership skills and experience you would bring. Applications are due by December 31, 2009. For more information about this position, go to Deputy Director of the Division of Extramural Activities, DEA (PDF) at Working at NIAID: Open Scientific Positions. For more information about NIAID's career opportunities, go to Working at NIAID.
Reader QuestionsFeel free to send us a question at deaweb@niaid.nih.gov. After responding to you, we may include your question in the newsletter, incorporate it into the NIAID Research Funding site, or both. "How soon after I am awarded a K23 grant can I apply for an R21?" -- Kyle Popovich, Rush University Medical CenterYou may get salary support from an R21 during the final two years of your K23 -- but since it can take from five to 18 months after the receipt date to get an award (three months less for AIDS applications), consider applying well before then. Read more about timing your application at Get a Grip on Timing and R01 Application to Award Timeline for Initial Submission and Resubmission. An R21 has the same timeline as an R01, just a different receipt date. For NIH's policy on concurrent salary support for mentored career development awards, see the April 10, 2008, Guide notice. We provided a brief summary in our May 7, 2008, article "Mentored Ks and Concurrent Support: Revised Policy Kicks In.""It is difficult to determine if research is considered human subjects. Since my IRB is pending approval, do I need to complete the human subjects section?" -- Rahmi Oklu, Massachusetts General Hospital, Harvard Medical SchoolYes, you must complete the human subjects section. If not, the award will get a bar, which can take a long time to remove. Read Are You Conducting Human Subjects Research? in the NIAID Human Subjects Application and Grant Handbook for additional advice on whether your research is considered to be human subjects. "As a college senior interested in studying EEG neurofeedback to treat seizure disorders, which grant type can I apply for?" -- Christina Fike, James Madison UniversityUnfortunately, we don't have grant opportunities for someone at your level. However, if you'd like to talk to an NIH staff person about your career, you could try contacting the Office of Training and Career Development at the National Institute of Neurological Disorders and Stroke. "How do we apply for NIAID support for the ISOPOL XVII meeting?" -- Paula Teixeira, Organizing Committee of ISOPOL XVIINIAID has a process for requesting conference support, which we describe in our Conference Awards SOP. Before proceeding further, you may want to talk to Dr. Barbara Mulach, the NIAID program officer, to find out if this is worth pursuing.
See these and older announcements at NIH Funding Opportunities Relevant to NIAID.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Subscribe to Alerts
