RAE 2001: comments of the Biochemical Society on the draft criteria and working methods of the Biological Sciences (UOA 14) panel

Most Heads of Department and Committee members who responded to the Biochemical Society considered that the criteria and proposed working methods of the Biological Sciences panel were generally satisfactory, and the respondents usually sought only clarification of particular issues. However, the Society would like to highlight two particular concerns.

  1. A number of respondents questioned whether the panel membership, working without sub-panels having co-opted members, would have the breadth of expertise to cover the wide range of subject matter likely to be submitted. A number of other comments (see below) arose from this basic question. Publication of the panel membership in recent days has shown that the cell and molecular biosciences are well catered for, but the Society has received complaints that whole organism biology, ecology, behaviour, and plant biology are under-represented.
  2. The question as to what extent departments/schools will receive feedback on the gradings of sub-groups contributing to the submission. Paragraph 2.10.18 implies that a single grade will be given to an entire submission, although departments are recommended to sub-divide submissions into research groups or research areas. It was uncertainty on this point that led to the earlier correspondence of the Society with Dr Rogers on whether a department could make more than one submission to the same panel. In the 1996 RAE several large university departments sub-divided submissions to the Biological Sciences panel and received separate official RAE gradings for the sub-groups. It was not clear to the whole community beforehand that this option was available. It is essential that for RAE 2001 the rules are explicit and applied consistently. If this is the case there will be little need for institutions to apply for multiple submissions. The Society recommends that where departments have coherent reasons for sub-dividing submissions, and where the sub-groups achieve a certain critical size, the department should be able to specify to the panel those groupings for which it wants to receive separate published gradings.

The following additional comments were made.

  1. Paragraphs 2.10.2 and 2.10.5. Clarification was sought on how the panel would seek to ensure comparability of assessment for work, including elements of cross-disciplinary submissions, that was referred to other panels.
  2. Paragraphs 2.10.3 and 2.10.37. Sub-parts of submissions would be allocated to two panel members for detailed review. Some respondents considered that there should be cross- representation between these sub-panel groups to ensure consistent and equitable treatment of all submissions. This practice is intended by the panel for UOAs 5-8, for instance, which is operating a not dissimilar system. Concern was also expressed that the decision of the panel not to use a larger sub-group system could introduce an element of randomness into rankings depending on the match between a department's research activity and the expertise of the panel.
  3. Paragraph 2.10.8. Clarification was sought on the way that quality of output and numerical indicators would be used to assign gradings. In any fine-tuning it was important that quality of output should always outweigh numerical indicators.
  4. Paragraph 2.10.10. The intention to read only 'a very small proportion of the works cited' risks making assessment unduly subjective. In these circumstances it is difficult to see how the panel can claim to judge all work on its merit rather than using journal reputation or numerical parameters such as citation indices as guides. The process might be aided if those submitting were requested to rank their work in the order that they wanted it to be examined.
  5. Paragraph 2.10.13. Ranking of journals by individual panel members lacks transparency, and judgements could be affected by any lack of expertise of panel members in particular areas.
  6. Paragraphs 2.10.11-2.10.16. Clarification was sought on the relative rankings likely to be assigned to the types of work that could be submitted. For example, full papers, minor versus senior/first author, reviews in quality journals, book chapters, patents. Respondents felt they needed to know 'where the goal-posts were'.
  7. Paragraph 2.10.27. Additional indicators of esteem such as industrial consultancies should be taken into account.
  8. Paragraph 2.10.30. Stating that research studentships should be externally funded in order to be included in numerical measures of research activity is a stricter requirement than other panels apply.
  9. Paragraph 2.10.38. To command the confidence of the community, it is, of course, essential that the lead reviewers who report back preliminary assessments to the main panel, and who can exert considerable influence, are seen to have appropriate expertise in the particular areas of bioscience concerned.

The Biochemical Society,
59 Portland Place,
London W1B 1QW;
Tel: 020 7580 5530; Fax: 020 7637 3626;
E-mail: genadmin@biochemistry.org