Tourism

3-15-2009 22:29

Source: Adapted from QAA
Table 7: Summary of scores from subject review in hospitality leisure, recreation, sport
and tourism 2000–2001 (%).

[ by Tourism at 3-15-2009 22:29 edited ]

Tourism

The strongest elements of provision were Curriculum Design, Content and Organisation
(CDCO) and SSG. Under CDCO the overview report highlighted a range of positive elements
including:
● multidisciplinary curricula with effective application of theory to practice;
● opportunities for work placement in around 70% of the provision;
● outward-looking and dynamic provision with good external links to industry.
One criticism to emerge was the need for staff to strengthen the underpinning of the curricula
with enhanced research and scholarly activity. For SSG, the positive points highlighted
clear information about the programmes, good academic guidance, effective tutorial support
and the wide range of central welfare and support services in the institution.
LR were also highly rated with 88% of HE provision awarded a grade 4 (in contrast to
42% in FE colleges). Favourable comment was made on library provision including effective
liaison between library and academic staff, IT resources and specialist facilities. Student
Progression and Achievement (SPA) was also overall good with “high achievements in final
awards attained when seen against modest student entry qualifications” (QAA, 2001a, p. 7)
although reference was made to 25% of cases where there were concerns about low retention
and progression rates. The biggest concerns emerged in the Teaching, Learning and
Assessment (TLA) and Quality Management and Enhancement (QME) categories. Whilst
“the quality of teaching observed was with few exceptions, judged to be of high quality”
(QAA, 2001a, p. 5) with many examples of good practice, a significant number of providers
dropped a grade because of weaknesses in assessment including:
● a mismatch between learning outcomes of the unit and the assessment criteria;
● a mismatch between the assessment criteria and the marking;
● variability of the quality of written feedback to students;
● variability in the length of time taken to give feedback.
QME displayed well-developed and comprehensive quality systems supported by good
documentation, but the reviews revealed that “institutional arrangements are not always
being applied consistently at the subject level” (QAA, 2001a, p. 9). Specific reference is
made to the annual programme monitoring for which it is suggested that reports are “insufficiently
evaluative in 20% of providers” (QAA, 2001a, p. 9) and there are deficiencies in
the systems for the consideration of external examiners reports in 30% of providers (QAA,
2001a, p. 9). Given that the Institutional Audit arrangements, which have now replaced
Subject Review, are based on the application of quality systems at subject level this is a
serious point.

Tourism

The subject review process is not without its critics. The most commonly cited criticism is
that the benefits were outweighed by the very high cost (Tribe, 2003; Macleod, 2001). In
addition to the high cost of each visit (fees and expenses of the review chair plus subject
reviewers) there are the preparation costs incurred by the institution. Many lecturers
regarded the process as unnecessarily bureaucratic involving considerable time and paperwork
which could have been spent more profitably on other activities (Tribe, 2003).
The real value of Subject Review was dependent on the extent to which it encouraged
institutions to be reflective about their provision and prompt system improvements or
whether it was an exercise in concealment. In brief, was it seen as an end in itself or the
beginning of an ongoing process? There was scope for the subject review process to be
used in a developmental way and although the paperwork required was significant, it did
not have to be wasted effort. The QAA recognise the considerable demands they make on
institutions but argue “that this is not wasted time or energy, and that the benefits to institutions,
as well as to students and others are considerable” (QAA, 2004a, p. 2).
Another comment made by lecturers was that it focused on procedure rather than
on the quality of teaching, research or even scholarship (Tribe, 2003). Such criticisms
seem unfair. Teaching was observed, albeit during a short intensive visit, and documentary
evidence to support claims for teaching was required and difficult for institutions to manufacture
(external examiner reports, student feedback). Scholarship was discussed in many
subject reviews and in the overview report and impacted on aspects of provision, most
notably related to the CDCO and TLA categories. Whilst quality assurance procedures
were central to the subject review process, it did try to measure how effectively institutions
used them to deliver provision. The weakest aspect, the TLA category, was so identified
because of what was delivered to the students. The criticisms identified under the QME
category may reflect procedure but shortcomings in the application of quality assurance at
subject level does impact directly on the student experience.
Perhaps the most significant criticism is that Subject Review did not use benchmark standards.
Judgements were made against the aims and objectives set by the institution, making
comparability between departments unfair if not impossible. Indeed departments making
lesser claims may have achieved full marks by meeting them (Tribe, 2003; Macleod, 2001).
However, these aims and objectives are published by QAA in the subject review report and
institutions cannot afford to set claims and objectives of mediocrity. Nevertheless the
overview did find that many SADs had been cautious in their construction and around 40%
were unduly descriptive rather than evaluative (QAA, 2001a, p. 7). Subject benchmarks
were being developed and published around the time of the subject review process which
made their use for subject review impractical.
Despite these criticisms, research by Tribe (2003) found that the majority of tourism
lecturers regarded their subject review score as fair although rigorous as more felt it
underestimated the quality of provision than overestimated it as shown in Table 8. Tribe
found the largest discrepancies between lecturer perceptions and the subject review
results in the TLA category, where 46% of lecturer respondents graded their institution at
4 (significantly more than subject review), and in LR and SSG where lecturers felt outcomes
overscored their provision. The TLA result has already been discussed and despite
many positive comments on teaching and on innovative assessment, collectively the subject
review reports highlighted a quite specific weakness in assessment practice from
which the sector would do well to learn. The lecturers’ more pessimistic assessment of
LR is partly explained by the nature of the category. A score of 4 does not reflect perfection,
and in this aspect there is always scope to provide more. It is a fine judgement
whether resources make a “full contribution” (hence 4) in an area where there is always
“scope for improvement”.

Tourism

3-15-2009 22:29

Source: Tribe (2003).
Table 8: Lecturer perceptions of QAA score.

[ by Tourism at 3-15-2009 22:29 edited ]

Tourism

The universal Subject Review process has now been replaced by Institutional Audit.
However, academic review of subjects is continuing in all FE colleges directly funded by
HEFCE. The QAA claims that academic review at subject level helps build a record of
review for these colleges but there seems little doubt that this more stringent regime is
partly driven by the lower scores in the FE sector.

Tourism

QAA undertook a review of 33 foundation degrees (FDs) in 2003 representing around
33% of the total number of registered students. This included two programmes in Travel
or Tourism and one in Hospitality Management. Overall, the review gave FD provision a
positive review although 4 of the reviews resulted in judgements of no confidence. Key
findings are summarised in Table 9.

Tourism

_______________________________________________________________________
Strengths
● Well-designed curricula and teaching and learning;
● Contribution to widening participation in HE;
● Involvement of employers in the development and delivery.
However reviewers noted
● Clear articulation of progression routes to honours degrees in only around 30% of
sample;
● Scope for the development and assessment of knowledge, understanding and skills.
Reviewers found that many achieved practical and vocational skills but not the higher
level, intellectual, analytical and reflective outcomes;
● Few cases where employers were full members of a consortium;
● Significant development of work-based learning practice.
_________________________________________________________________________
Table 9: Summary of findings from overview of foundation degrees.

Tourism

The Institutional Audit process for England was launched in the summer of 2002 with the first
visits taking place in January 2003. All English universities and colleges will be audited
between 2003 and 2005 and from 2006 audits will be on a 6-year cycle. This involves
“scrutiny of internal quality assurance systems at institutional level, with a more detailed
investigation at discipline level of whether those systems are operating in the manner
intended”. (QAA, 2003, p. 5). These disciplinary audits are undertaken by subject review specialists
included in the QAA team (rather like the specialists used for subject review) and are
limited to around 10% of an institution’s programmes. If significant weaknesses are found at
the discipline level this may result in a recommendation for a separate full subject review
under the process set out in the Academic Review Handbook (QAA, 2000e; Tribe, 2003). The
new system can be seen to respond to a number of the criticisms of Subject Review.

Tourism

The QAA have promoted the new institutional audit as a lighter touch for the process of
external scrutiny. Certainly institutions will receive far fewer visits. It is also claimed there
will be less duplication of effort preparing for an audit. The institution will prepare an SED
for the visit but the underlying principle is to undertake an external peer review of the
internal quality assurance systems and thus “deliver fair and secure judgements without
undue burden” (QAA, 2004a, p. 1). Therefore much of the required documentation for the
visit is that already produced for internal quality assurance.

Tourism

Reference has already been made to the lack of agreed benchmarks under subject review
(Macleod, 2001). Institutional Audit judgements are made against the nationally agreed
benchmark standards developed by QAA over the last 4 years and discussed above (QAA,
2000a, c, 2001b).