University Affairs - June/July 2016

“Success;rates;are;signifi-
cantly;lower;for;early;career
applicants;from;small;univer-
sities;relative;to;their;peers
from;large;institutions.”
À mon avis
In my opinion
Shortchanging;smaller;universities
Anomalies in NSERC’s
evaluation process

by Douglas Morris, Dennis Murray, Hugh MacIsaac, Claude Lavoie, Peter Leavitt, Michael Masson and Marc-André Villard

Lead author Douglas Morris is a professor of evolutionary and conservation ecology in the department of biology, Lakehead University.

recently completed analyses of the Natural Sciences and Engineering Research Council’s Discovery Grant decisions. These analyses consistently reveal that the probability of success and the mean monetary value of awards in Discovery Grant competitions are higher for large universities than for small- and medium-sized universities. The funding level awarded to an application depends on which of the 16 quality bins it is allocated to (typically only the top 10 bins receive grants). Lower success rates and funding levels by applicants from small and medium-sized universities can thus occur only if their applications are typically allocated to “lower” bins than those from large universities. The data confirm our expectation: a greater proportion of applications from small- and medium-sized universities fall into the lower bins than is the case for large universities.

Critics will note that Discovery Grant funding rates and levels have always been lower at small universities compared to large ones. They will attribute the discrepancy to differences in priorities, research culture and infrastructure, graduate programming and faculty recruitment. Teaching loads are typically higher at medium-sized and small universities, graduate training opportunities are often fewer, and the critical mass necessary to form integrated research teams may be missing. Each effect is likely responsible to some degree. Even so, there are at least two basic reasons to suspect that differences in funding success and grant size represent true anomalies.

First, all applications are evaluated on three equally scored metrics: excellence of the researcher, merit of the proposal and contributions to training of highly qualified personnel, or HQP. The combined score determines the quality bin allocated to each application.

Virtually all first-time applicants enter the competition with similar HQP records because most lack previous opportunities to supervise and train graduate students. Consequently, the assessment of HQP for first-time applicants is typically focussed “on the plan for future training,” according to NSERC’s 2015-16 Discovery Grants manual. We can thereby anticipate that the assessment of HQP for first-time applicants should level the field. If the field is indeed level, then differences in grant funding success between small and large universities should tend to disappear for these early career applicants.

Comparisons between medium and large schools confirm our intuition. There is no statistically significant difference in their likelihood of successful funding. That is not the case at small universities. Success rates are significantly lower for early career applicants from small universities relative to their peers from large institutions. The pattern is perplexing and disturbing because it suggests that scores allocated for HQP to early career researchers are higher at large than small universities. Should one expect consistently superior plans for future training from applicants at large institutions?

Second, the distortion in funding success and
funding levels is self-reinforcing. Canada’s Federal
Research Support Fund is linked directly to the
funding received from tri-council funding agen-
cies. Reduced grant success yields less support for
research that necessarily feeds back into an insti-
tution’s research opportunities and future success.

NSERC began to address the problem in the 2015 competition by instituting a pilot program of Discovery Development Grants for deserving but unfunded Discovery Grant applicants. The program is designed to allow its recipients to maintain high-quality research and research training in small universities. But the small size of the grants ($10,000 annually), even if matched by institutional funding, and their short duration (two years), are bound to be ineffective in a system where so many better-funded and higher-ranking applicants subsequently fail in attempts to renew their Discovery Grants.

It is not exclusively the patterns and projections that concern us, but also their consequences. Smaller universities cast a wide net that identifies and encourages promising students to pursue research. Smaller universities serve our disparate regions, our rural communities and the heartland of our nation’s resource-driven economy. Those regions and communities are not well-served by concentrating fewer grant-holding professors in fewer universities. Doing so compromises Canada’s future research capacity and it reduces our ability to solve regional problems. It limits our options to understand and incorporate regional variance into ideas and policy. It also denies bright students in rural Canada opportunities to contribute to our collective futures.