Research Assessment Reform Beyond DORA: What Happens After Universities Stop Counting Journal Impact Factors

In December 2012, a group of editors and researchers met at the American Society for Cell Biology’s annual meeting in San Francisco and wrote a declaration. It has since attracted thousands of signatures from universities, funders, publishers, and professional societies. The San Francisco Declaration on Research Assessment, known as DORA, makes one central demand: do not judge a researcher by the journal in which their work appears.

I first read DORA the way I read gym membership contracts — with enthusiasm, then I forgot about it until renewal. That changed about five years later, when I sat on a hiring committee that spent far too long sorting candidates by the titles on their CVs. I’m not proud of it. The only thing we reliably measured was how quickly we could get to coffee.

The gap between signing a pledge and changing a promotions file is where research assessment reform actually succeeds or stalls. And that gap has become the main story in 2025.

What DORA actually asks

DORA’s most quoted instruction is a negative one: funders, universities, research institutes, and the professional societies that run many journals should not use the Journal Impact Factor, citation counts, or any other journal-level metric as a substitute for reading a candidate’s work. The Journal Impact Factor, usually JIF, is a simple average: the number of citations in one year to articles published in the previous two years, divided by the number of citable articles. Clarivate calculates it. It was designed to help librarians choose journal subscriptions, not to rank scientists.

The declaration also targets publishers and data providers. It asks publishers to stop marketing the impact factor as the central badge of quality, to make individual article metrics available, to support author name identifiers, and to be explicit about what any metric can and cannot show. It asks funders to write evaluation criteria that say explicitly what counts as good science, not merely where it was published.

But a signature does not rewrite a tenure checklist. DORA has always been a set of recommendations, not an enforcement body. That is precisely the problem the next wave of reform has tried to solve.

Promotion files are where reform meets reality

The Netherlands offers the clearest example of institutions trying to move past pledges. In 2019, Dutch universities, university medical centres, and research funders published “Room for everyone’s talent,” which recast academic quality as more than publication output. That programme, now called Recognition and Rewards, pushes departments to assess research, teaching, societal impact, leadership, and patient care together, and to build separate career paths for people who excel in different combinations of those tasks.

Utrecht University has gone further in practice. Its hiring and promotion guidance tells committees not to use the impact factor as a shortcut and asks candidates to describe their contributions across research, education, team science, and public engagement. Ghent University in Belgium uses a similar broad evaluation format. The common feature is that the conversation starts with the work, not the journal cover.

The full declaration, including its recommendations for funders and publishers, is available at sfdora.org. Reading it takes twenty minutes; rewriting a promotion rubric takes years.

Photo by Daria Nepriakhina 🇺🇦 on Unsplash

CoARA turned a volunteer movement into a timetable

The most important structural development arrived in July 2022, when the European University Association, Science Europe, and the European Commission launched the Agreement on Reforming Research Assessment. The Coalition for Advancing Research Assessment, or CoARA, now coordinates the commitments. Signatories agree to base assessment primarily on qualitative judgement, with quantitative indicators used only where meaningful, and to abandon the inappropriate use of journal-based metrics in hiring, promotion, fellowship applications, and grant reviews.

What makes CoARA different from DORA is that it comes with deadlines. Members are expected to publish action plans, review institutional criteria, report on progress, and update the plans when evidence shows they are not working. Some universities now post those plans publicly, which means an external reader can check whether the institution does anything after signing.

China has made a parallel move from the state, though through its own machinery. In 2020, the Ministry of Education and the Ministry of Science and Technology told universities to stop treating SCI paper counts and journal impact factors as the core criterion for titles, bonuses, research project approvals, and recruitment. AcademicJobs covered that policy shift in more detail in China’s research evaluation reform. The lesson from both Europe and China is that reform accelerates when funders stop rewarding old metrics, because universities follow the money faster than they follow declarations.

The UK’s REF 2029 gives culture a number

The Research Excellence Framework, or REF, shapes how UK universities allocate research funding and how departments are judged. The next cycle, REF 2029, changes the formula: research outputs count for 50 per cent, impact for 25 per cent, and the new People, Culture and Environment block for 25 per cent. That third block makes research culture, staff development, collegiality, and fair reward systems part of the formal score, not a vague aspiration.

The shift matters because it changes what department heads ask for in annual reviews. A unit that protects time for mentorship, records how it supports early-career researchers, treats open research as normal, and reviews workload allocations can now argue that this is evidence, not admin. The official guidance is on the REF 2029 pages. At the same time, the open access deposit rules have their own requirements, as AcademicJobs explained in an earlier guide to the REF 2029 open access deposit policy.

One caution from past cycles is that new categories often get filled with old evidence: a department may simply rebadge its publication list as “culture” without changing behaviour. The test will be whether panels reward genuine practices rather than polished narratives.

Metrics do not disappear; they relocate

Removing the impact factor usually leaves a vacuum, and evaluation committees fill vacuums quickly. A candidate may now be asked for citation counts at the article level, field-normalised citation percentiles, download figures, media mentions, or social media shares. None of those is inherently better at identifying careful research. The Leiden Manifesto, published in Nature in 2015, warned that quantitative evaluation should support, not replace, expert judgement. The ten principles remain the most practical guide to using research metrics responsibly; the full text is available at Nature.

Narrative CVs create their own biases. A long, well-edited narrative can favour candidates with time, native-language fluency, and strong mentoring. Shorter structured formats, such as the Résumé for Researchers used by UK Research and Innovation, try to reduce that gap by asking for fixed sections: contributions to research, team science, teaching, leadership, and wider society. The format works best when assessors are trained to read it and given enough time. Without that, a narrative CV becomes another document to skim for keywords.

For researchers, the practical implication is that evidence now matters more than adjectives. “I am a highly collaborative scientist” says nothing. “I coordinated a six-institution cohort study and wrote the data-sharing agreement used by all sites” says quite a lot.

Photo by Clayton Robbins on Unsplash

Nobody tells you this about applying under the new rules

Most researchers will meet the reformed assessment system through one document: the promotion or fellowship application. You can prepare for it without waiting for your university to publish a new policy. Start by collecting evidence in the categories committees now tend to ask about: research outputs you can explain in plain language, contributions to teaching and supervision, service roles that changed a process, open research practices, and collaborations where you did the unglamorous coordination work.

A persistent identifier helps. ORCID links your works, grants, peer review activity, and employment history so that assessors can find what you actually did, rather than what a search engine retrieves. AcademicJobs has covered the expansion of ORCID requirements in peer review and editorial workflows. Registering and keeping an ORCID profile current is cheap insurance against being judged on an incomplete publication list.

When you write an application, show the relation between your role and the output. For each major piece of work, specify whether you designed the study, managed the data, secured ethical approval, wrote the manuscript, or led the team. If a committee refuses to read beyond the journal name, that is a signal about the institution, not a reflection on your application.

I once added my name to a responsible research assessment pledge and then, two weeks later, was asked to submit my h-index for an internal review. This is not an argument against the pledge. It is a description of the gap that matters: declarations change language, but only hiring panels, funding boards, and the annual review forms that department heads actually complete change behaviour.

The next test for research assessment reform will be whether job advertisements stop describing “top-tier journals” as the first requirement. Some have stopped. Others still carry the old line while displaying a DORA badge elsewhere on the same site, a combination I have started reading as a gym membership pinned to a fridge.