A person in a brown knitted cardigan, sitting at a table, types on a laptop. A note pad, pen and cup filled with a beverage sits next to their laptop

REF2021 consultation on the draft guidance and criteria

Universities & higher education Research Councils & HE sector bodies Excellence Frameworks (REF, TEF and KEF)2018

Please provide any comments on Part 2

69-95 – We urge caution in expectations for mapping Units of Assessment (UoAs) to HESA cost centres. In geography, like a number of other subjects, the HESA cost centre data are error-riddled and incomplete. HESA costs centre codes in many departments split human and physical geography depending on teaching assignments, at least historically, and do not map neatly onto a unit assessment. Staff returned to the Geography (C17) UoA in the last REF would be allocated elsewhere. The resulting misalignment would be counter to the articulated purposes of REF. Using HESA cost centre codes to allocate staff can also mitigate against other efforts to encourage and foster multi and interdisciplinarily e.g. those in large research centres and/or other institutional strategic research initiatives.

The proposed approach for taking account of circumstances will achieve the aim of promoting equality and diversity in REF 2021 (comments):

Concerns have been expressed by many in the community that decoupling outputs from individual staff might inadvertently have unintended consequences in terms of equality and diversity, in terms of whose outputs are selected and implications for staff subsequently based on this selection/submission. The geography community request guidance to encourage all sub-panels to look carefully across a unit’s submission in this context, to ensure claims made about equality and diversity (for example, in the environment section) are consistent with actions (e.g. as reflected by selected outputs). Clarity is also requested about the extent to which data on submissions (gender, career stage etc) will be shared with the panel during the process (or evaluation after).

Please provide any further comments on these proposals, including any suggestions for clarifying or refining the guidance

The geography community expressed concerns over the workload and administrative complexity of the proposed approach to output reductions due to special circumstances which overlaps with the established approach of 1-5 outputs per member of submitted staff. A range of concerns are relevant here relating to using 0.5 output reductions as a basis for decision making and then applying rounding; upfront provision of the evidence base at the time of submission for all output reductions; etc.

Do you agree with the proposed intention to permit the submission of co-authored outputs only once within the same submission?

Some in the geography community question the proposed limit on submission of co-authored outputs within the same submission. This, they argue, unfairly disadvantages collaboration within departments/UoAs. They do, though, welcome recognition that some co-authored projects can be more demanding and ambitious than single authored work and thus warrant double-weighting.

Do the UoA descriptors provide a clear and appropriate description of the disciplines covered by the UoAs? Please include any suggestions for refining the descriptors and state which UoA(s) you are commenting on.

UoA 14: Geography and Environmental Studies

The geography community welcomes the broad and inclusive description of the sub-panel UoA 14 and its scope. Given the breadth of UoA 14 we expect (and hope) a significant number of additional sub-panel members will be appointed in 2020 to represent fully the diversity of geography, geographers and institutions that will be participating in REF2021. One minor addition to the description, for completeness, in terms of methods is to add archival (………: and work that uses a range of available methods, from science-based to humanistic and participatory, including numerical, theoretical, experimental, modelling, ARCHIVAL and field-based).

Please comment on the criteria in ‘Part 3, Section 1: Submissions’, in particular on:

where further clarification is required
where refinements could be made
whether there are areas where more consistency across panels could be achieved
whether there are differences between the disciplines that justify further differentiation between the main panel criteria.

We welcome the recognition that interdisciplinary research us research that not only crosses main panel boundaries, but also disciplines within the sub-panels within a main panel. We urge REF2021 to go further though to recognise that interdisciplinary research can also be conducted within a UoA such as Geography and Environmental Studies.

We welcome the statement made, para 117, that there is no advantage or disadvantage of identifying outputs as interdisciplinary.

We note the recognition that institutional structures do not always map neatly onto UoAs. This is frequently the case for a discipline such as Geography. We welcome robust and equitable processes for cross-referrals which we might anticipate between UoA 7 and UoA 14, Geography and Environmental Studies.

Please comment on the criteria in ‘Part 3, Section 2: Outputs’, in particular on:

the proposed criteria for double-weighting outputs in Main Panels C and D, and on whether requests to double-weight books should automatically be accepted
whether Annex C ‘Main Panel D – outputs types and submission guidance’ is helpful and clear
where further clarification is required
where refinements could be made
whether there are areas where consistency across panels could be achieved
whether there are differences between the disciplines that justify further differentiation between the main panel criteria.

The community welcome the positive statement that all outputs will be treated equitably.

Double-weighting. The geography community welcomes the opportunity to request that some outputs are double-weighted. Some in the community expressed their support the statement that there is no presumption that books should be double-weighted. In this context they question, Main Panel C’s statement that ‘most books…warrant double-weighting’ (para 239).

The community also welcome the opportunity to nominate a reserve output in case the sub-panel rejects the request for double-weighting.

The geographical community also welcome the decision by Panel C that no additional information be provided on co-authors and urge the panel if they receive publications with author contributions documented (as required now by some journals in our field), this information be ignored.

The community also welcome the decision the Geography and Environmental Studies sub-panel will not use journal impact factors (para 269), consistent with DORA and Leiden statements.

Please comment on the criteria in ‘Part 3, Section 3: Impact’, in particular on:

where further clarification is required
where refinements could be made
whether there are areas where more consistency across panels could be achieved
whether there are differences between the disciplines that justify further differentiation between the main panel criteria

The geography community requests greater clarity about co-production and co-submission of the same impact case study for different departments/for different panels, specifically in terms of the extent to which they can be the same or are expected to differ.

More guidance is also requested on impacts through teaching.

Concerns have also been raised about the burden of having to submit corroborating evidence at the time of submission.

Please comment on the criteria in ‘Part 3, Section 4: Environment’, in particular on:

whether the difference in section weightings across main panels is sufficiently justified by disciplinary difference (paragraphs 322 and 323)
whether the list of quantitative indicators provided at www.ref.ac.uk is clear and helpful
where further clarification is required
where refinements could be made
whether there are areas where more consistency across panels could be achieved
whether there are differences between the disciplines that justify further differentiation between the main panels, please state which one(s).

Unit of Assessment environment template: The geography community urge REF2021 to reconsider the environment template and to include a separate section on impact. In its current form it is not as well organised as it could be. For example, the final proposed section combines contributions to the discipline etc along with impact beyond the academy in an unhelpful way. Greater clarity of structure is needed. If restructured it could also be easier to assess.

The geography community request guidance to encourage all sub-panels to look carefully across a unit’s submission in the context of equality and diversity, to ensure that claims made about equality and diversity in the environment statement are matched in the submission of outputs.

Please comment on the criteria in ‘Part 5: Panel working methods’, in particular on: - where further clarification is required or where refinements could be made.

Members of the community have suggested that more details are needed about sub-panel working methods (for example in what order will different components be graded). There is a sense that in the last REF UoAs chose different orders, which caused some difficulties reconciling the grades with cross-referrals.

Also, will the sub-panels be using a finer grade scale than the integer 0-4 (there’s a sense again that last time this differed between UoAs). Clarity on this would help.

Please comment on the balance between consistency and allowing for discipline-based differences between the main panels.

The credibility of REF depends on equitable practices and behaviours across panels and sub-panels. We urge REF 2021 to be as transparent as possible in the approached and methods used to achieve this outcome.