REF2021

As a national exercise, the robust evaluation of research and impact requires academic peer contribution which is provided by members of different universities and research organisations (including research users) participating as REF2021 panellists. a number of Exeter colleagues played a vital role through this process. This included:

Although descriptions of REF procedures were still accurate in recordings, key dates changed due to Coronavirus.‌

The UoE REF2021 Code of Practice was submitted to Research England for approval by the REF Equality and Diversity Advisory Panel. It was first submitted in June 2019 with a revised version submitted for final approval in September 2019. The final completed University of Exeter REF2021 Code of Practice was submitted on 30th July 2021‌‌.

The Code of Practice detailed the policies and processes which governed how the University developed its REF submission. The University of Exeter REF2021 Code of Practice focused on:

• Processes for understanding who was eligible to be submitted for REF and how that was determined (Section 2 and Section 3)
• Processes for the selection of outputs, including those of former staff (Section 4)
• Processes for reporting individual circumstances

This‌ also includes other related documents and policies in the annexes:

• Governance structures and membership
• Roles and responsibilities for decision-making and key REF2021 recommendations
• Appeals Policy and Process
• Conflicts of Interest Policy
• Confidentiality Policy
• EDI Training policy and approach
• Policy to determine UoA allocation
• Approach to selecting Impact Case Studies
• Determining Independent Researcher status: forms
• Individual Circumstances Disclosure: form

Only members of staff who were recorded by the Higher Education Statistics Agency as having Education and Research, or Research-Only contracts, could have been considered for REF2021 submission. All staff who meet the REF2021 definitions of having ‘significant responsibility for research’ or of being an ‘independent researcher’ had to be submitted to REF. These staff are sometimes referred to as Category A Eligible staff.

As detailed in the Code of Practice, all staff on Education and Research (E&R) contracts at the University were deemed to have met the REF definition of having ‘significant responsibility for research’ and, therefore, ALL staff on E&R contracts must have been returned in the submission. The processes for determining whether staff on Research-only contracts meet the REF2021 definition of ‘independent researcher can be found in the Code of Practice. ALL research-only staff who meet the definition of ‘independent researcher’ must have been returned in the submission.

The University had undertaken a preliminary process for identifying independent researcher status and those who were eligible were notified in January 2020. New staff who had joined since June 2019 or for whom further information about their eligibility is required received an email asking them to complete a form.

If you have any queries please contact REF-2021-Eligibility@exeter.ac.uk or your Director of Research or Head of Department.

REF2021 Eligibility Review Group membership

The dedicated REF Eligibility Review Group oversees the review of Independent Researcher forms and using the information provided did inform individuals of their eligibility/ineligibility for the REF2021 submission.

The Eligibility Review Group was an independent group representing all of the colleges at the University of Exeter. It was guided only by Research England's instructions, and not by outputs or any other influences. Any information disclosed was only seen by the REF Eligibility Review Group. The REF Eligibility Review Group met periodically and consisted of the following members:

• Linda Peka (Chair) – Deputy Registrar, Chair of the University Inclusivity Group
• Professor Andrew McRae – Dean of the Doctoral College 
• Dorcas Cowan – Equality, Diversity & Inclusion Manager
• Andrew Johnson – Assistant Director (HR Policy & Reward)
• Professor Annette Broderick – Academic Member 
• Professor Laura Salisbury – Academic Member 
• Professor Obi Ukoumunne – Academic Member 
• Professor Pete Vukusic – Academic Member 
• Drs Astrid Wissenburg - Director of Research Services

 Eligibility Review Group Secretariat:

• Kerry Hope - Deputy REF Manager
• Dr Ruth Cooper - REF Policy Officer (Individual Circumstances)
• Natasha Walker - REF Policy Officer (Independent Researchers)
• Dr Sumi David - Head of Research and Impact (Policy & Performance)/REF Manager

Appeals

The decisions of the REF Eligibility Review Group could be appealed if there had been a failure to follow the policies and/or procedures for determining researcher independence (found in the Code of Practice), or if there had been a failure to comply with the University’s legal responsibilities.

If you had wished to appeal an eligibility decision made by the Eligibility Review Group, you would have referred to the REF Code of Practice. Appeals on the grounds of disagreement with the decision of the Eligibility Review Group, or on the basis of insufficient evidence having being submitted when the request was first put forward were not permitted. Appeals were also excluded from this process if they involved a matter which was still subject to litigation or legal proceedings; if the matter had previously been through the appeals procedure in line with this policy and had been closed; or if appeals were made by staff with a role capable of determining the shape of a submission.

Appeals had to be addressed to REF-2021-Appeals@exeter.ac.uk by 31st October 2020.

Research outputs counted for 60% of the overall quality rating for each Unit of Assessment (UoA).

There was an established tariff, based on the number of FTE submitted to a UOA, for the number of outputs that must be submitted. For every 1.0 FTE submitted, 2.5 outputs were required. A member of staff was eligible if they met the REF definition of having ‘significant responsibility for research’ or the REF definition of “independent researcher”. Each member of staff submitted had one output attributed to them, and no single member of staff could have more than five outputs attributed to them. Our policies on eligibility and how outputs will be selected could be found in our Code of Practice.

Former University of Exeter staff employed during the eligible period and who met the eligibility criteria were also returned alongside appropriate outputs.

Outputs were assessed on the basis of ‘Originality’, ‘Rigour’ and ‘Significance’. The University’s annual Research Monitoring exercise played an integral role in supporting the review and selection of outputs for REF.

In recognition of the effect individual circumstances could have on researcher productivity, and therefore their ability to contribute to the pool of outputs at the same rate as other staff, the requirement of a minimum of one output could have been waived for those with complex circumstances. The reporting of individual circumstances could have, in exceptional cases, also resulted in the entire UoA having a reduction to the total number of outputs that it needed to submit.

For REF2021, Research England introduced an Open Access policy against which all journal articles and conference proceedings published since 1 April 2016 were required to be compliant, although some exceptions are possible. For details of the policy and how you might get funding to support Open Access publishing please see the ORE website.

For the purposes of the REF, impact was defined as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.

Impact case studies

The Impact element of REF carried a weighting of 25 per cent of the overall assessment. The number of case studies required in each UoA submission was determined by the number (full-time equivalent) of staff returned in the submission. If a submission included fewer than the required number of case studies, a grade of unclassified was awarded to each required case study that was not submitted. Submissions may not have included more than the required number of case studies.

Each Impact case study was designed to capture: 

• The nature and extent of specific impact occurring during the reporting period - 1 August 2013 to 31 July 2020. This included a clear explanation of both the impact’s reach and its significance. 
• An outline of specific research-based activity within the unit that contributed to the impact, and an explanation of how it made a contribution
• Details of the underpinning research, produced by the submitting unit in the period 1 January 2000 to 31 December 2020, and justification that this was of high quality.
• Evidence to corroborate the claims made

Impact case study guidance was an extremely important resource to help complete this section. The University developed an overview of how to write an Impact case study and outlined the Best Practice for this.

In addition, a template for writing Impact case studies, was available in the REF Draft Guidance on Submissions (view Annex G, page 113)

Examples of Impact indicators

For more information on this please see Annex A, page 94 of the REF Consultation on the draft panel criteria and working methods. Please also see the REF Impact evidence taxonomy.

The environment element of the REF counted for 15% of the overall quality rating and was assessed in terms of a UoA’s 'vitality and sustainability', including its contribution to the vitality and sustainability of the wider discipline or research base during the assessment period (1 August 2013 to 31 July 2020).

Typically, the Director of Research for each UoA co-ordinated and completed a template that covered the following areas:

• Section 1: Unit context and structure, research and impact strategy
• Section 2: People
• Section 3: Income, Infrastructure and facilities
• Section 4: Collaboration and contribution to the research base, economy and society

To protect against direct or indirect discrimination, the University had an Equality Analysis process that helped consider the effects of policy, process or decision on anyone identifying with protected characteristics*.

Consultation with staff and students was imperative to the Equality Analysis process. It was important to liase with staff and students from different backgrounds in order to effectively meet the needs of the diverse groups of people at the University. By capturing this feedback, we hoped to guide and appraise policy formation within the University. 

For details of the governance arrangements for the development of the University’s REF2021 submission, and the membership of groups, please see the Code of Practice.

Deputy Vice-Chancellor, Research & Impact, Professor Neil Gow had overall responsibility for the University’s REF2021 submission. Other decision making bodies include:

• The Vice-Chancellor’s Executive Group (VCEG)
• The Research and Impact Executive Group
• The University Research Monitoring Panel

The University of Exeter’s preparations at an institutional level were also supported by the REF Strategic Advisory Group and the REF Project Management Group.

The appeals policy covered appeals relating to:

• The determination of independent researcher status; 
• The selections of outputs and impact case studies for submission to REF2021; 
• Internal decisions relating to the requests for waiving of the minimum requirement for outputs as a result of individual circumstances (including those requiring judgement) which are submitted to REF; 
• Internal decisions relating to the requests for applying the published tariffs for the reduction of the output pool due to ECR status, family-related leave or secondments/career breaks.

Appeals could have been brought forward on the following grounds:

• Failure to follow the policies and/or procedures for determining researcher independence or outputs selection as laid out in the Code of Practice;  
• Failure to follow the University’s legal obligations; 
• And, incorrect assessment of the Research England guidelines in relation to individual circumstances (including those requiring judgement), the removal of the minimum output requirement and any reductions in the UoA’s output pool.  

The following types of appeal were excluded from the procedure set out in this policy:

• Appeals regarding a disagreement with a decision of the Eligibility Review Group, or similar decision-making body;  
• Appeals on the basis of insufficient evidence having being submitted when the request was first put forward;  
• A matter which is currently subject to litigation or legal proceedings;
• The matter has previously been through the appeals procedure in line with this policy and has been closed;
• Appeals made by staff with a role determining the shape of a submission to overturn a decision made by the Eligibility Review Group and similar decision making body or role. 

Who could have used this policy and when? 

• Individuals: This policy covers members of staff on E&R and R-only contracts of either permanent or fixed-term basis who wish to appeal decisions based on the areas listed above. The policy applies to members of staff in relation to their independent researcher status, the selection of outputs and decisions regarding individual circumstances.
• Appeals must be submitted in writing using the following email address: REF-2021-Appeals@exeter.ac.uk
• The decision of the 2021 REF Appeal Panel will be final within the procedures of the University and all appeals will be concluded (where possible) before the REF submission deadline of 31st March 2021.

An appeals panel had been established and was comprised of the following members:

• Professor David Hosken - Pro-Vice-Chancellor, Cornwall (Chair)
• Dr Michael Wykes - Director of Policy, Planning and Business Intelligence (Deputy Chair)
• Holly Hellier - Senior HR Business Partner (People Services Representative)
• Professor Claire Dunlop - Professor of Politics and Public Policy (Academic Member)
• Professor Krasimira Tsaneva-Atanasova - Professor of Mathematics for Healthcare (Academic Member)
• Dr Sumi David - Assistant Director of Research and Impact (Secretariat)
• Kerry Hope - Deputy REF Manager (Secretariat)

The appeals panel met as and when neccessary and appeals meetings may:

• Uphold the decision taken by the Eligibility Review Group or similar decision body, determining also, where relevant, if the appeal is trivial/vexatious/invalid. In this case there shall be no further right of appeal. 
• Determine that further information is required and invite the member of staff to supply further information, and to invite them to appear in person and may be accompanied by a trade union representative or fellow worker.
• Determine that the decision taken by the Eligibility Review Group or similar decision body is not upheld on the basis of not following the policies and procedures of the Code of Practice, of not fulfilling the University’s legal obligations or, in the case of complex circumstances, an incorrect assessment of the Research England guidance. The relevant REF decision-making body will be notified and required to review and reconsider the decision taken.  

The individual raising the appeal was notified within 10 working days of the relevant Appeals Panel meeting.

The decision of the REF2021 Appeal Panel was final within the procedures of the University and staff did not have recourse to the Grievance Procedure for grievances related to REF2021. The REF2021 Appeal Panel was advised by the Equality and Diversity Team where appropriate. Please see the REF Code of Practice for further details.

The University of Exeter (the "University") is a data controller and is committed to protecting your personal data and working in accordance with all relevant data protection legislation.

The University’s Research Services Directorate, specifically the Research and Impact (Policy and Performance) team (RIPP or REF team) captures and detains data in order to submit them to the Research Excellence Framework (REF) 2021. This is the UK’s system for assessing the quality of research in UK higher education institutions, and it takes place every 6-7 years.

What data do we hold?

The submission will contain a common set of data comprising:

• Information on all staff in post with significant responsibility for research on the census date, including information on individual circumstances. These information are: Unit of Assessment, HESA staff identifier code, date of birth, Open Researcher and Contributor ID, contract details (FTE and dates), and some information to support the independent researcher assessment (grants, funds attribution, leading a research group). You can find further information about what data are being collected on the REF website, at www.ref.ac.uk in particular publication 2019/01, ‘Guidance on submissions’.
• Details of outputs produced in the submitted unit during the publication period or by those employed on the census date that fall within the publication period.
• Impact Case studies describing specific examples of impacts, the nature of the related underpinning research and underpinning evidence corroborating the impact. See also the Impact Case Study privacy notice relating to external partners or those providing corroborating evidence.
• Data about research doctoral degrees awarded, research income and related income-in-kind.
• An institutional-level environment statement, and a completed template for each Unit of Assessment describing the submitted unit’s research and impact environment.
• Documentation, policies and procedures at Department-, College- and University-level.
• Working document supporting the preparations for the REF2021 submission.

How do we use your data?

The data will be held by the University of Exeter’s REF team, and your data will be used and stored for the purpose of the REF2021 submission and in compliance with the Research Services data retention policy. Data will be reviewed by ad hoc REF-specific internal governance structures. University of Exeter will also store the data you have provided, to assist Research England who administer REF2021 with any auditing queries they may have. Your data will be kept in accordance with GDPR law and our retention schedule.

For the submission we are required to provide Research England with information about eligible staff for the purpose of the REF2021. The information will include your name and details such as your date of birth, Open Researcher and Contributor ID (ORCID), research groups, and contract dates will be provided along with details of your research.

In addition to these we are also required to provide Research England with copies of relevant extracts from staff contracts (or full contracts with relevant information highlighted and sensitive data redacted) for Audit purpose only. 

The results of the assessment exercise will be published by UKRI, on behalf of the four UK higher education funding bodies, in April 2022. The published results will not be based on individual performance nor identify individuals.

How do we protect your data?

Your data is held securely on secured network drives and the University’s internal databases. Access to the secured network drives and databases is restricted to University staff who are undertaking REF2021 related roles with access granted based on their specific remit, role and responsibilities. All staff who access the secured network drives and databases have completed the University’s data protection training.

Dedicated procedures had been set up to ensure an extra care in dealing with confidential materials including personal data, data relating to individual circumstances or eligibility and, where relevant, confidential impact case studies or outputs.

How long do we keep your personal information?

We may retain your personal information for the assessment and audit phases of the REF2021 submission (November 2021), or for two cycles of the national REF exercises (i.e. potentially 2028). If you want to know how long we keep your information, please write to our Data Protection Officer. Or for more information, please consult ourResearch Services retention scheme.

Your rights and legal basis

We ensure we maintain the Confidentiality, Integrity and Availability to University information as well as compliance with relevant legislation including:

• The General Data Protection Regulation (GDPR),
• The Data Protection Act 2018 (DPA2018),
• The Freedom of Information Act (FOIA).

This Privacy Notice will be kept under review. Any changes will be updated on our website This Privacy Notice was last updated in January 2020.

You have the right to:

• Ask us to see, correct or delete the data we hold about you
• To request access to any data about you
• Object to specific data uses, as described above

The University’s Data Protection Officer is responsible for monitoring compliance with relevant legislation in relation to personal data and can be contacted at dataprotection@exeter.ac.uk, you can also contact the DPO if you have any queries or concerns about the University’s processing of your personal data. You have the right to lodge a complaint with the Information Commissioner’s Office.

For more information on any of the above or the REF in general, please contact the University of Exeter REF team on Exeter-REF-2021@exeter.ac.uk.