The assessment and verification of National Vocational Qualifications [NVQs]: a European quality perspective
Leeds Metropolitan University UK
Paper presented at the International Lifelong Learning Conference, 9-11 July 1999, University College Worcester
This paper is located within the policy context where the UK government is seeking to align its policies and systems closer to those of the European Union [EU], where that does not challenge vital national interests. The EU, following the Treaties of Union [Maastricht] and Amsterdam is seeking to develop Employment and Human Resource Development as one of the "pillars" of the EU. A number of significant recent developments in the field of surveys and analysis across the EU are identified to provide a specific focus for analysing the specific issues of assessment and verification of work place competencies.
This presentation reviews the problems encountered in the development of National Standards and Qualifications [NVQs] in the United Kingdom and identifies briefly some of the practical problems with this development. These issues are located within the European Quality Foundation excellence model for Total Quality Management. This model is then analysed against the current approaches to assessment, verification of NVQs, and the approach of developing "communities of practice" is used to focus on the weaknesses of the existing system. The specific issues of simulation and sampling are analysed are covered in some depth, with improved approaches suggested. Finally, recommendations are made for the continuing professional development of VET professionals.
Principal research questions
The vision put forward by the UK government in its New Training Initiative of 1981 still resonates in the European Union at the end of the 1990s. With the widening of the EU through the accession of new member states, the improvement of learning in the workplace has become a significant issue for the next decade.
"... if societies of the future are to become economically competitive, they have to become learning societies and individuals have to become lifelong learners. ... However, in the learning organisation, learning society and lifelong learning literature, very little attention has been given to the process of learning itself. ... Following cultural anthropologists such as Jean Lave, we are interested in the potential of the concept of apprenticeship as the basis for a social theory of learning." (Guile & Young 1998: 173-174)
The purpose of this paper is to propose a strategy for situated learning in the communities of practice made up of those engaged in the assessment of learning in the workplace. As I have argued in a separate paper (Konrad 1999), the present model of "competence" used in existing UK National Standards is based on an individualist and behaviourist model of learning which is rooted in the US models of competence based training of the 1970s. The critical literature on the delivery of NVQs has been reviewed elsewhere (Konrad 1998). This paper proposes a strategy to improve the reliability and validity of the assessment and verification of NVQs using this collaborative model of learning in the "cognitive apprenticeship" of assessors and verifiers. A Total Quality Management approach will be used to provide a framework. The approach selected in the model of the European Foundation of Quality Management [EFQM]
Situated Learning - I
"Knowledge is contextually situated and is fundamentally influenced by the activity, context and culture in which it is used." (Brown, Collins & Duguid, 1989)
Stories "Narratives play a vital role in the transfer of information and discoveries. And stories help people keep track of their discoveries providing a meaningful structure for remembering what has been learned. ... Stories are very important for situated learning and especially the social construction of knowledge." (McLellan 1996: 7-8)
In the case of NVQ Internal Verifiers, a preliminary investigation revealed that:
"I learned the practice through an initial familiarisation with am experienced colleague. Once I was qualified, I gathered experience and expertise over two years on a number of programmes, until I became lead Verifier for a small Centre"
"My job as an Internal Verifier is to check on the work of the Centres assessors. Where I find issues, I then discuss them in the regular team meeting so that we can all develop our expertise based on a sharing of experience."
These two preliminary examples indicate the potential of stories in this area of work. Further research will seek to develop this approach within a network of satellite sites attached to a single Centre.
There is a well-developed literature on the important role of reflexive and reflective practice in the continuing development of a range of professionals. (Boud et al1985; Holly 1989; Schon 1983, 1990; Elliott 1996)
Situated Learning - II
Cognitive Apprenticeship involves sharing the culture of real practice using knowledge as a tool for students to use in collaborative social interaction. Effective experience involves:
- a high intensity of interaction and feedback
- specific goals and established procedures
- high motivation
- continual challenge at a realistic level
- directly working on a real task
- avoiding disruptions and distractions
(McLellan 1996: 10)
This approach would provide an interesting approach to the development of assessment and verification processes. I believe that it would provide a more coherent and effective approach to training and development than the limited approach based on the current "D Units".
Collaborative working emphasises peer support, small group and team work. An approach to NVQ assessment which sees Assessment and Verification as a set of roles rather than a hierarchy of positions is likely to reproduce this important aspect of situated learning.
Similarly, a process of Coaching can be an important part of this type of team approach. Practice in a number of locations, with different types and levels of students provide an opportunity to refine individuals practice so that interpersonal and decision-making skills are more effectively used.
Communities of Practice
Some brief points:
In applying the concept of "Communities of Practice", it is important to take account of the socio-historical context. People who have no history of common goals, culture and practices will not be located within this context solely by virtue of their geographical setting. In practice, we all recognise that communities are not necessarily homogeneous or ethically desirable. In the context of this paper, I would argue that a group of assessors and verifiers may be regarded as a Communities of Practice if they share common values, purposes and practices that have a direct practical effect on their professional and personal actions. For example, I would see that a common articulation, problematisation and practical consequences would be crucial in resisting attempts to alter the result of an assessment solely on the grounds of funding or other contractual issues involving institutional compliance.
In this context, the ways in which actions are agreed and justified are, in my opinion, at least as important as the actions themselves. In another context, it might well be appropriate to describe these actions as strategies of coping or even resistance.
Factors affecting practice
The work of assessors and verifiers is affected by a number of contextual factors:
- Formal procedures laid down by QCA, the NTO and the Awarding Body
- The actual practical impact of these decisions
- The ways in which a community adds members
Assessors as a "community of practice"
The NVQ assessment guidance document provides a structure for the assessment process [NCVQ 1997:5] covering the main stages of planning, collecting and judging evidence, making decisions and giving feedback. The guidelines make it clear that while assessment should be jointly planned by assessors and candidates, management responsibility rests with a team who have advisor, assessor and internal verifier roles within an Approved Centre recognised by an Awarding Body and supported by an External Verifier.
Recent work on the ways in which professionals learn, emphasises the importance of "developing legitimate peripheral participation where learners inevitably participate in communities of practitioners and that the mastery of knowledge and skills requires newcomers to move towards sociocultural practices of a community. Legitimate peripheral participation provides a way to speak about the relations between newcomers and old-timers, and about activities, identities, artefacts, and communities of knowledge and practice. It concerns the process whereby newcomers become part of a community of practice. A person's intentions to learn are engaged and the meaning of learning is configured according to the process of becoming a full participant in a sociocultural practice. This social process includes, indeed it subsumes, the learning of knowledgeable skills." [Lave & Wenger, 1991:29]
Experience as a practising Assessor and Internal Verifier and anecdotal evidence from other colleagues indicate that these complex roles are often under-resourced and those carrying out these important roles have barely adequate initial education and training. Despite the efforts of individuals, there is little evidence of the systems that would be required to maintain "an up-to-date understanding of the NVQ system, including current best practice in assessment and internal verification [emphasis added] you need management skills which enable you to develop good working relationships with assessors, while at the same time maintaining control, and providing constructive feedback." . [QCA 1998a:9]
This approach to internal verification tends to locate the process within a hierarchy of individual positions, rather than a set or roles within a workgroup. The writer's experience is that if all internal verifiers are also practising as opposed to merely qualified assessors, the interplay of roles within the team is likely to lead to the development of expertise within the group. In this way, the framework provided by NVQs could, and should become a basis for evolving and growing learning in the workplace. In the light of the earlier discussion, it is argued that the most effective way of improving practice would be to develop an approach to the continuing professional development of internal verifiers, which was based on situated learning within a community of practice. The use of a total quality management approach, such as that represented by the EFQM model would also be important.
This view is consistent with the advice to Assessors to encourage integration of a number of activities across a series of Elements to ensure that the necessary evidence of competence and expertise should result in effective and efficient assessment. In such cases, it will also be clearer when sufficient evidence of expertise and competence is available. The main issue of sufficiency arises from the requirements that the competence demonstrated is comprehensively based on application of knowledge and understanding demonstrated consistently over an adequate period. [NCVQ 1997:17-18]
In the experience of this writer, it is difficult to see how this could be demonstrated without involving the work group who would then be checking and confirming the relationship between membership of the community of practice and the level of knowledge of the work process required. There is considerable scope for the Continuing Professional Development of assessors and verifiers in ways that promote the concept of knowledgeable skills based in communities of practice.
The EFQM excellence model
Summary of Quality Assurance roles and responsibilities
|Candidate||Individuals seeking recognition of their achievements and current competence against national occupational standards||Demonstrate their ability perform to these standards in the workplace and support their claim to continuing competence by evidence of knowledge and understanding. This process will normally involve the candidate playing an active role in the planning of this assessment process and in the quality of the evidence provided.|
|Adviser||A person sufficiently competent against the occupational standards involved and capable of advising candidates. [The Unit D36 Accreditation of Prior Learning Adviser is a relevant qualification]||Enabling the candidate to identify the appropriate standards and level against which their competence may be assessed. This process will involve an explanation of the assessment planning process and providing support for the identification of appropriate evidence to meet the requirements of the standards.|
|Assessor||A person sufficiently competent against the occupational standards involved and capable of assessing candidates. Appointed by an Approved Centre [see below]. Qualified as a Vocational Assessor by holding Units D32 & D33.||Judging a candidates performance and knowledge evidence, deciding whether the candidate has demonstrated competence against the chosen standards and providing clear and constructive feedback to the candidate. Maintaining the level of their professional competence.|
|Internal verifier||A person sufficiently competent against the occupational standards involved and capable of ensuring the consistency and quality of assessment. Appointed by an Approved Centre [see below]. Qualified as a Internal Verifier by holding Unit D34. Qualification and experience as a Vocational Assessor is preferable.||Ensuring the quality of the assessment process and assessors through undertaking a leadership role within the Approved Centre. This involves the selection, training, provision of continuing development of Advisers and Assessors; monitoring assessments including sampling of evidence; maintaining and developing the quality of documentation; authorising requests for award of NVQs; ensuring equality of access to assessment for all candidates. Managing their own continuing professional development.|
|External verifier||Appointed by Awarding Bodies to ensure consistency and quality of assessment. Qualified as a Internal Verifier by holding Unit D34. Qualification and experience as a Vocational Assessor is preferable||Ensuring reliability and validity of assessment and the quality assurance process across centres. Providing guidance and support for centres, normally through Internal Verifiers. Sampling assessment and verification practice in centres and providing feedback to both centres and awarding bodies. Managing their own continuing professional development.|
|Approved Centre||Organisations approved by awarding bodies to assess and verify candidates for specified qualifications||Maintain staff and operating procedures to meet awarding body requirements. Develop a culture of continuous improvement to meet the requirements of the VET market.|
|Awarding Body||Organisations approved by QCA to award NVQs and related vocational qualifications||Assure the reliability and validity of assessments. Provide guidance and criteria for centre operations. Approve and monitor the operation of centres.|
Applying the EFQM model to assessment and verification
|Quality standards||Suggested evidence|
|3a resources are planned and improved||The organisation's development plan should allocate sufficient finance to ensure that changes in assessment workloads are resourced and that sufficient time is available for the workgroup with assessor and verifier roles to carry out efficient and effective work.|
|3b capabilities are sustained and developed||Continuing professional development should be provided for the assessment/verification team and integrated into the organisation's human resources programme. This should provide for regular monitoring of assessment practice and for checking the reliability and validity of decisions.|
|3c agree targets and continually review performance||The outcomes of the monitoring process and especially the effectiveness of the sampling process should be reviewed with the quality system, and benchmarked against the performance of other organisations. The External Verifiers(s) should be closely associated with this process.|
|3d are involved, empowered and recognised||The education and training of this group of VET professionals should support the development of a community of practice. The development of work process knowledge in this area of the organisation should be supported, promoted, and celebrated.|
This paper has indicated an approach to the assessment and verification which, mindful of the strengths and weaknesses of the existing systems, seeks to build on the strengths and reduce the weaknesses. The main change that is advocated is to develop an approach to continuing professional development of those involved at an Approved Centre in the process of situated learning in a community of practice. This will require a radically different approach to the individualistic market-based model that QCA has inherited from its predecessor. Specifically, this will require radically different quality criteria, with built-in continuing professional development based on collaborative learning. It is unlikely that these changes and improvements can be realised without substantial modernisation of existing policy and practice and allocation of appropriate resources. The most important aspect of such development will be the piloting of these approaches to change.
Boud D et al (1985) Reflection: turning experience into learning, London, Kogan Page
Brown J S, et al. (1989) "Situated Cognition and the Culture of Learning" inMcLellan H (Ed.), Situated Learning Perspectives, Eaglewood Cliffs NJ, Educational Technology Publications, 19 - 44
EFQM (1999) The EFQM Excellence Model,http://www.efqm.org/ June 1999.
Elliott G (1996) "Teaching in post-compulsory education: profession, occupation or reflective practice" paper given at BERA conference September 1996¸ available at Education-Line: http://www.leeds.ac.uk/educol/documents/000000104.htm
Guile D & Young M (1998) "Apprenticeship as a conceptual basis for a social theory of learning", Journal of Vocational Education and Training, pages 173 - 192
Holly M L (1989) Writing to grow, London, Heinemann
Konrad J L (1998) "Review of the theory of workplace learning [WPL] covering the range of approaches present in the research literature: Social and Political perspectives on WPL", Education-Line,http://www.leeds.ac.uk/educol/documents/000000672.htm September 1998. Submitted to European Journal of Vocational Education and Training, June 1999
Konrad J L (1999) "Assessment and Verification of NVQs: policy and practice", submitted to the Journal of Vocational Education and Training July 1999
McLellan H (1996) "Situated Learning: Multiple Perspectives" in McLellan H (Ed.), Situated Learning Perspectives, Eaglewood Cliffs NJ, Educational Technology Publications, 5-17
Schon D A (1983) The Reflective Practitioner: How Professionals think in action, NY, Basic Books.
Schon D A (1990) Educating the Reflective Practitioner, San Fransisco, Jossey-Bass
Wenger E (1998), Communities of Practice: Learning, Meaning and Identity, Cambridge, Cambridge University Press
This document was added to the Education-line database 27 July 1999