RAE 2001: comments of the Biochemical Society on the draft criteria and working methods of the Biological Sciences (UOA 14) panel
Most Heads of Department and Committee members who responded to the Biochemical Society
considered that the criteria and proposed working methods of the Biological Sciences panel
were generally satisfactory, and the respondents usually sought only clarification of
particular issues. However, the Society would like to highlight two particular concerns.
- A number of respondents questioned whether the panel membership, working without
sub-panels having co-opted members, would have the breadth of expertise to cover the wide
range of subject matter likely to be submitted. A number of other comments (see below) arose
from this basic question. Publication of the panel membership in recent days has shown that
the cell and molecular biosciences are well catered for, but the Society has received
complaints that whole organism biology, ecology, behaviour, and plant biology are under-represented.
- The question as to what extent departments/schools will receive feedback on the gradings of
sub-groups contributing to the submission. Paragraph 2.10.18 implies that a single grade will
be given to an entire submission, although departments are recommended to sub-divide
submissions into research groups or research areas. It was uncertainty on this point that
led to the earlier correspondence of the Society with Dr Rogers on whether a department could
make more than one submission to the same panel. In the 1996 RAE several large university
departments sub-divided submissions to the Biological Sciences panel and received separate
official RAE gradings for the sub-groups. It was not clear to the whole community beforehand
that this option was available. It is essential that for RAE 2001 the rules are explicit and
applied consistently. If this is the case there will be little need for institutions to apply
for multiple submissions. The Society recommends that where departments have coherent reasons
for sub-dividing submissions, and where the sub-groups achieve a certain critical size, the
department should be able to specify to the panel those groupings for which it wants to
receive separate published gradings.
The following additional comments were made.
- Paragraphs 2.10.2 and 2.10.5. Clarification was sought on how the panel would seek to
ensure comparability of assessment for work, including elements of cross-disciplinary
submissions, that was referred to other panels.
- Paragraphs 2.10.3 and 2.10.37. Sub-parts of submissions would be allocated to two panel
members for detailed review. Some respondents considered that there should be cross-
representation between these sub-panel groups to ensure consistent and equitable treatment of
all submissions. This practice is intended by the panel for UOAs 5-8, for instance, which is
operating a not dissimilar system. Concern was also expressed that the decision of the panel
not to use a larger sub-group system could introduce an element of randomness into rankings
depending on the match between a department's research activity and the expertise of the panel.
- Paragraph 2.10.8. Clarification was sought on the way that quality of output and numerical
indicators would be used to assign gradings. In any fine-tuning it was important that quality
of output should always outweigh numerical indicators.
- Paragraph 2.10.10. The intention to read only 'a very small proportion of the works cited'
risks making assessment unduly subjective. In these circumstances it is difficult to see how
the panel can claim to judge all work on its merit rather than using journal reputation or
numerical parameters such as citation indices as guides. The process might be aided if those
submitting were requested to rank their work in the order that they wanted it to be examined.
- Paragraph 2.10.13. Ranking of journals by individual panel members lacks transparency, and
judgements could be affected by any lack of expertise of panel members in particular areas.
- Paragraphs 2.10.11-2.10.16. Clarification was sought on the relative rankings likely to be
assigned to the types of work that could be submitted. For example, full papers, minor versus
senior/first author, reviews in quality journals, book chapters, patents. Respondents felt
they needed to know 'where the goal-posts were'.
- Paragraph 2.10.27. Additional indicators of esteem such as industrial consultancies should
be taken into account.
- Paragraph 2.10.30. Stating that research studentships should be externally funded in order
to be included in numerical measures of research activity is a stricter requirement than other
panels apply.
- Paragraph 2.10.38. To command the confidence of the community, it is, of course, essential
that the lead reviewers who report back preliminary assessments to the main panel, and who can
exert considerable influence, are seen to have appropriate expertise in the particular areas of
bioscience concerned.
The Biochemical Society, 59 Portland Place, London W1B 1QW;
Tel: 020 7580 5530; Fax: 020 7637 3626; E-mail:
[email protected]
|