“Success;rates;are;signifi-
cantly;lower;for;early;career
applicants;from;small;univer-
sities;relative;to;their;peers
from;large;institutions.”
À mon avis
In my opinion
Shortchanging;smaller;universities
Anomalies in NSERC’s
evaluation process
by Douglas Morris, Dennis Murray, Hugh MacIsaac, Claude Lavoie,
Peter Leavitt, Michael Masson and Marc-André Villard
Lead author Douglas Morris is
a professor of evolutionary
and conservation ecology in
the department of biology,
Lakehead University.
recently completed analyses of
the Natural Sciences and Engineering Research Council’s Discovery Grant decisions. These
analyses consistently reveal that
the probability of success and the mean monetary value of awards in Discovery Grant competitions are higher for large universities than for
small- and medium-sized universities. The funding level awarded to an application depends on
which of the 16 quality bins it is allocated to
(typically only the top 10 bins receive grants).
Lower success rates and funding levels by applicants from small and medium-sized universities
can thus occur only if their applications are typically allocated to “lower” bins than those from
large universities. The data confirm our expectation: a greater proportion of applications from
small- and medium-sized universities fall into the
lower bins than is the case for large universities.
Critics will note that Discovery Grant funding rates and levels have always been lower at
small universities compared to large ones. They
will attribute the discrepancy to differences in
priorities, research culture and infrastructure,
graduate programming and faculty recruitment.
Teaching loads are typically higher at medium-sized and small universities, graduate training
opportunities are often fewer, and the critical mass
necessary to form integrated research teams may
be missing. Each effect is likely responsible to
some degree. Even so, there are at least two basic
reasons to suspect that differences in funding
success and grant size represent true anomalies.
First, all applications are evaluated on three
equally scored metrics: excellence of the researcher,
merit of the proposal and contributions to training of highly qualified personnel, or HQP. The
combined score determines the quality bin allocated to each application.
Virtually all first-time applicants enter the
competition with similar HQP records because
most lack previous opportunities to supervise
and train graduate students. Consequently, the
assessment of HQP for first-time applicants is
typically focussed “on the plan for future training,” according to NSERC’s 2015-16 Discovery
Grants manual. We can thereby anticipate that
the assessment of HQP for first-time applicants
should level the field. If the field is indeed level,
then differences in grant funding success
between small and large universities should tend
to disappear for these early career applicants.
Comparisons between medium and large
schools confirm our intuition. There is no statistically significant difference in their likelihood
of successful funding. That is not the case at
small universities. Success rates are significantly
lower for early career applicants from small universities relative to their peers from large institutions. The pattern is perplexing and disturbing
because it suggests that scores allocated for HQP
to early career researchers are higher at large
than small universities. Should one expect consistently superior plans for future training from
applicants at large institutions?
Second, the distortion in funding success and
funding levels is self-reinforcing. Canada’s Federal
Research Support Fund is linked directly to the
funding received from tri-council funding agen-
cies. Reduced grant success yields less support for
research that necessarily feeds back into an insti-
tution’s research opportunities and future success.
NSERC began to address the problem in the
2015 competition by instituting a pilot program
of Discovery Development Grants for deserving
but unfunded Discovery Grant applicants. The
program is designed to allow its recipients to
maintain high-quality research and research
training in small universities. But the small size
of the grants ($10,000 annually), even if matched
by institutional funding, and their short duration
(two years), are bound to be ineffective in a system where so many better-funded and higher-ranking applicants subsequently fail in attempts
to renew their Discovery Grants.
It is not exclusively the patterns and projections that concern us, but also their consequences.
Smaller universities cast a wide net that identifies
and encourages promising students to pursue
research. Smaller universities serve our disparate
regions, our rural communities and the heartland
of our nation’s resource-driven economy. Those
regions and communities are not well-served by
concentrating fewer grant-holding professors in
fewer universities. Doing so compromises Canada’s future research capacity and it reduces our
ability to solve regional problems. It limits our
options to understand and incorporate regional
variance into ideas and policy. It also denies
bright students in rural Canada opportunities to
contribute to our collective futures.
WE
Cet article est également
disponible en français
sur notre site web,
www.affairesuniversitaires.ca