Education-line Home Page

Students' perceptions about assessment in higher education: a review

Katrien Struyven , Filip Dochy & Steven Janssens
University of Leuven (K.U.Leuven), Centre for Research on Teacher and Higher Education

Paper presented at the Joint Northumbria/ Earli SIG Assessment and Evaluation Conference: Learning communities and assessment cultures, University of Northumbria at Newcastle, August 28 - 30 2002.

Katrien Struyven, K.U.Leuven, Centre for Research on Teacher and Higher Education, Vesaliusstraat 2, 3000 Leuven, Belgium. Telephone: +32 16 32 60 25. Fax: +32 16 32 62 74. Email: Katrien.Struyven@ped.kuleuven.ac.be

Abstract

Learning, from a constructivist point of view, is seen as a constructive act of the learner. Along with the changes in learning theory, several instructional innovations and alternative assessment methods found their ways into educational practice. But are these innovations as successful as they promise to be? In this review, the characteristics and impact of assessment are examined from the student's point of view. Research findings suggest that students' perceptions about assessment, have considerable influences on students' approaches to learning. But also vice versa, students' approaches influence the ways in which students perceive assessment. Furthermore, it was found that students hold strong views about different formats and methods of assessment. For example, within conventional assessment, multiple choice format exams are seen as favourable assessment methods in comparison to essay type questions. But when conventional assessment and alternative assessment methods are compared, students perceive alternative assessment as being more 'fair' than the traditional 'normal' assessment methods.

Structure and purpose of this review

Learning, according to the latest 'constructivist' learning theories, is essentially: (1) constructive, (2) cumulative, (3) self- regulated, (4) goal- oriented, (5) situated, (6) collaborative, and (7) individually different (De Corte, 1996). The learner is an active partner in the process of learning, teaching and assessment. S/he selects, perceives, interprets, and integrates new information to form a coherent and meaningful whole with her/his prior knowledge and former experiences (Dochy, Segers, & Buehl, 1999). These changes in learning theory go together with innovations in instruction and evaluation: new instructional methods are introduced in educational practice, the latest technologies and media are used, and alternative modes of assessment are implemented (Birenbaum, 1996). This belief in the active role of the learner and Entwistle's (1991) finding that it is the student's perceptions of the learning environment that influence how a student learns, not necessarily the context in itself, both gave rise to this review that focuses on the influences of conventional and alternative assessment methods from the students' perspective.

Methodology for the review

The Educational Resources Information Center (ERIC), the Web of Science and PsychINFO, were searched online for the years 1980 until 2002. The keywords: 'student* perception*' and 'assessment' were combined. This search yielded 508 hits in the databases of ERIC and PsycINFO and 37 hits within the Web of Science. When this search was limited with the additional keywords 'higher education', only 171 and 10 hits respectively remained. Relevant documents were selected and searched for in the libraries and the e- library of the K.U.Leuven: 35 documents met our criteria, in which 36 empirical studies are discussed. For a summary of this literature we refer to the overview (see Table 1), of which the full version is available on request. Theoretical and empirical articles are both included. First, a specific code is given to each article, e.g. 1996/03/EA, which refers to the publication year/ number/ publication type (EA: empirical article/ TA: theoretical article/ CB: chapter of book). Second, the author(s) and title of the publication are presented. For each empirical study, the overview reports on: (1) the content of the study, (2) the type and method of the investigation, (3) the subjects and the type of education in which the study is conducted, (4) the most important results that were found, (5) the independent and (7) dependent variables studied, (6) the treatment which was used, and (8) the type and (9) method of analyses reported in the research. Both qualitative and quantitative investigations are discussed.

Table 1: Extract of the overview: an example of a reviewed document, discussing two empirical studies.

2001/02/EA

Mien Segers & Filip Dochy

New assessment forms in problem- based learning : the value- added of the students' perspective.

Quant. & Qual. data gathered from a research project which is focused on different quality aspects of two new assessment forms in problem-based learning.

S1: written examination (overall test) and students' perc's on this ass. mode. S2: peer assessment and idem.

S1: TOC1: 34, & TOC2: 45, st. eval. quest.: 100, interviews: 33 students. S2: 27 Students in two groups.

Both ass. forms have acceptable qualities. The OA test =acceptable curricular, instructional and criterion validity. Peer marks correlate well with tutor and final exam. scores. Students perceive gap between tutorial groups and assessment.

S1: High, Moderate and low achievers S2: two groups

S1: Overall test, two Topic checklists (TOC1, TOC2), think aloud protocols and student eval. quest. and semi structured interviews (= 3 group discussions) S2: Peer ass. (tutor, self, peer and total scores) & peer ass. quest.

S1: students' results; curricular, instructional & criterion validity and students' perc's of OA test and relation to learning. S2: Results & students' perc's about peer & self assessment.

S1: Quant. & Qualitative analysis S2: Quantitative analysis.

S1: Means, SD, % S2: Reliability through generalisability scores for both groups; Pearson correlation coefficients: percentages, means and SD.

Students' perceptions about assessment

The repertoire of assessment methods in use in higher education has expanded considerably in recent years. New assessment methods are developed and implemented in higher education, for example: self and peer assessment, portfolio assessment, simulations, and OverAll assessment. The latest constructivist theories and practices go together with a shift from a 'test' or 'evaluation' culture to an 'assessment' culture (Birenbaum, 1996). The notion of 'alternative assessment' is used to denote forms of assessment which differ from the conventional assessment methods such as multiple- choice testing and essay question exams, and continuous assessment via essays and scientific reports (Sambell, McDowell, & Brown, 1997). The assessment culture, embodied in current uses of alternative assessment and fully compatible with constructivist ideas, favours: the integration of assessment, teaching and learning; the involvement of students as active and informed participants; assessment tasks which are authentic, meaningful and engaging; assessments which mirror realistic contexts, in contrast with the artificial time constraints and limited access to support available in conventional exams; focus on both the process and products of learning; and moves away from single test- scores towards a descriptive assessment based on a range of abilities and outcomes (Sambell, McDowell & Brown, 1997).

In this part of the review, the literature and research on students' perceptions about assessment are reviewed. The relation between and impact of (perceived) characteristics about assessment on students' approaches to learning and vice versa, are examined and discussed. Next, students' perceptions about different assessment formats and methods are presented, including: open- ended format versus multiple choice format examinations; portfolio assessment; self-, peer and overall assessment; simulations; and finally, also more general perceptions of students about assessment are investigated.

Assessment and Approaches to learning

Assessment is one of the defining features of the students' approaches to learning (e.g. Marton & Säljö, 1997; Entwistle & Entwistle, 1991; Ramsden, 1997). In this part of the review, an attempt is made to gain insight into the relations between (perceived) assessment properties and students' approaches to learning and studying.

Approaches to learning

When students are asked for their perceptions about learning, mainly three approaches to learning occur. Surface approaches to learning describe an intention to complete the learning task with little personal engagement, seeing the work as an unwelcome external imposition. This intention is often associated with routine and unreflective memorization and procedural problem solving, with restricted conceptual understanding being an inevitable outcome (Entwistle & Ramsden, 1983; Trigwell & Prosser, 1991; Entwistle, McCune, & Walker, 2001). Deep approaches to learning, in contrast, lead from an intention to understand, to active conceptual analysis and, if carried out thoroughly, generally result in a deep level of understanding. This approach is related to high quality learning outcomes (Entwistle & Ramsden, 1983; Trigwell & Prosser, 1991). And finally, because of the pervasive evidence of the influence of assessment on learning and studying an additional category was introduced, the strategic or achieving approach to learning, in which the student's intention was to achieve the highest possible grades by using well-organised and conscientious study methods and effective time- management (Entwistle & Ramsden, 1983; Entwistle, McCune & Walker, 2001).

Assessment in relation to students' approaches and vice versa

The research on the relation between approaches to learning and assessment is dominated by the Swedish Research Group of Marton and Säljö. These two researchers (Marton & Säljö, 1997) conducted a series of studies in which they tried to influence the students' approaches to learning towards a deep approach to learning. A prerequisite for attempting to influence how people act in learning situations, is to have a clear grasp of precisely how different people act. The learner/ reader, using a deep approach to learning, engages in a more active dialogue with the text. One of the problems with a surface approach is the lack of such an active and reflective attitude towards the text. As a consequence, a fairly obvious idea was to attempt to induce a deep approach through giving people some hints on how to go about learning (Marton & Säljö, 1997).

In his first study, Marton (1976) let students in the experimental group answer questions of a particular kind while reading a text. These questions were of the kind that students who use a deep approach had been found to ask themselves spontaneously during their reading (e.g. can you summarise the content of the whole section in one or two sentences?). This attempt to induce a deep approach, yielded interesting but contra- intuitive results. At one level, it was obvious that the approach taken was influenced by the treatment to which the experimental group was exposed. However, this influence was not towards a deep approach: instead, it seemed to result in a rather extreme form of surface learning. The control group, which had not been exposed to any attempts at influencing approach, performed significantly better on both the immediate and delayed retention measurements.

What happened was that the participants invented a way of answering the interspersed questions without engaging in the learning, characteristic of a deep approach. The task was transformed into a rather trivial and mechanical kind of learning. What allowed the participants to transform the learning in this way, was obviously the predictability of the task. They knew that they would have to answer questions of this particular kind, and this allowed them to go through the text in a way which would make it possible to comply with the demands without actually going into detail about what is said. The outcome of this study raises interesting questions about the conditions for changing people's approach to learning. The demand structure of the learning situation again proved to be an effective means of controlling the way in which people set about the learning task. Actually it turned out to be too effective. The result was in reality the reverse of the original intention when setting up the experiment (Marton & Säljö, 1997).

A second study (Säljö, 1975) followed. The factor varying between two groups, was the nature of the questions that the groups were asked after reading each of several chapters from an education text-book. One set of questions was designed to require a rather precise recollection of what was said in the text. In the second group, the questions were directed towards major lines of reasoning. The results show that a clear majority of the participants reported that they attempted to adapt their learning to the demands implicit in the questions given after each successive chapter. The crucial idea of this study was that people would respond to the demands that they were exposed to. In the group that was given 'factual' questions, this could be clearly seen. They reacted to the questioning through adopting a surface approach. However, in the other group, the reaction did not simply involve moving towards a deep approach. Some did, others did not. Only about half the group interpreted the demands in the way intended. The other students 'technified' their learning, again concentrating solely on perceived requirements. They could summarize, but could not demonstrate understanding (Marton & Säljö, 1997).

Both studies (Marton, 1976; Säljö, 1975) illustrate that although in one sense it is fairly easy to influence the approach people adopt when learning, in another sense it appears very difficult. It is obviously quite easy to induce a surface approach, however, when attempting to induce a deep approach the difficulties seem quite profound. The explanation is in the interpretation (Marton & Säljö, 1997).

In a third study, Marton and Säljö (1997) asked students to recount how they had been handling their learning task and how it appeared to them. Besides the questions about what they remembered of the text its content, students were also asked questions designed to discover how they tackled this reading task. The results show an astonishingly simple picture: the students who did not get 'the point' (i.e. understand the text as a whole) failed to do so, simply because they were not looking for it. The main difference in the process of learning concerned whether the students focused on the text itself or on what the text is about: the author's intention, the main point, the conclusions to be drawn. In the latter case the text is not considered as an aim in itself, but rather as a means of grasping something which is beyond or underlying it. It can be concluded that there was a very close relationship between process and outcome. The depth of processing was related to the quality of outcome in learning (Marton & Säljö, 1997).

The students' perceived assessment requirements seem to have a strong relation with the approach to learning a student adopts when tackling an academic task (Säljö, 1975; Marton & Säljö, 1997). Similar findings emerged from the Lancaster investigation (Ramsden, 1981) in relation to a whole series of academic tasks and also to students' general attitudes towards studying. Students often explained surface approaches or negative attitudes in terms of their experiences of excessive workloads or inappropriate forms of assessment. The experience of learning is made less satisfactory by assessment methods which are perceived to be inappropriate ones. High achievement in conventional terms may mask this dissatisfaction and also hide the fact that students have not understood material they have learned as completely as they might appear to have done. Inappropriate assessment procedures encourage surface approaches, yet varying the assessment questions may not be enough to evoke fully deep approaches (Ramsden, 1997).

Entwistle and Tait (1990) also found evidence for the relation between students' approaches to learning and their assessment preferences. They found that students who reported themselves as adopting surface approaches to learning preferred teaching and assessment procedures which supported that approach, whereas students reporting deep approaches preferred courses which were intellectually challenging and assessment procedures which allowed them to demonstrate their understanding. A direct consequence of this effect is that the ratings which students make of their lecturers will depend on the extent to which the lecturer's style fits what individual students prefer (Entwistle & Tait, 1995).

Assessment format and methods

During the last decade, an immense set of alternative assessment methods was developed and implemented into educational practice as a result of new discoveries and changing theories in the field of student learning. Students are supposed to be 'active, reflective, self- regulating learners'. Alternative assessment practices must stimulate these activities, but do they? Are these new assessment methods (e.g. self, peer, portfolio and overall assessment) different and/ or better, than the conventional assessment methods such as multiple choice tests and essay question examinations? An attempt is made to answer these questions from the students' perspective.

Open- ended format versus Multiple- choice format of examination

When students' perceptions and expectations about open- ended formats are compared to those about multiple choice formats of examination, some remarkable results occur. Within this section, the impact of students' expectations and perceptions about these two formats on students' approaches to learning is discussed and furthermore, students' preferences and attitudes towards both formats are compared and contrasted.

Deep, surface or strategic approach to learning?: Different types of assessment seem to encourage deep or surface approaches to learning. Entwistle and Entwistle (1991) found that students are very strongly influenced by the form of assessment they expect. Multiple choice formats, or an emphasis on detailed factual answers, push students towards a surface approach, while open, essay- type questions encourage a deep approach (Entwistle & Entwistle, 1991).

Also Thomas and Bain (1984) found that the most influential feature of the learning environment is the nature of the assessment procedures. The results of their study showed clearly how a change from multiple- choice to essay- type examinations had shifted the overall tendency of the students from a surface approach towards a deep approach. However, Eisenberg (1988) stressed that any component within the learning environment which contradicted the direction of influence of the other components might prevent the intended effect from being achieved. Thus, a clear implication for effective teaching is that all aspects of a course must convey the same message to students regarding what will be rewarded through assignments and examinations (Entwistle & Tait, 1995).

As we have seen, the strategic approach to learning is sensitive to the assessment procedures used and/ or expected. Because teachers have the final say on such indicators of academic success as student grades, it seems reasonable that students seek information and form opinions about 'what the teacher wants'. 'Figuring out the teacher' enables them to tailor study strategies that fit the task. For example, Rickards and Friedman (1978) found that reading notes taken by students expecting an essay examination were qualitatively, but not quantitatively, different than those taken by students expecting a multiple- choice test. The latter focused their note-taking efforts on facts and details, while those expecting essay tests concentrated on information of higher structural importance, such as main ideas and topic sentences (Nolen and Halady, 1990).

Students' preferences, attitudes and perceptions: Zeidner (1987) found that multiple choice type exams are generally perceived more favourably than essay type items along most dimensions of the inventory (i.e. perceived difficulty, anxiety, complexity, success expectancy, feeling at ease), by both boys and girls. The smallest differences between the formats were evidenced on the dimensions of trickiness, perceived interest, and perceived value. Furthermore, it was found that students perceive essay type exams to be somewhat more appropriate than multiple choice exams for the purpose of reflecting one's knowledge in the subject matter tested (Zeidner, 1987).

Birenbaum and Feldman (1998) examined the relationships between the students' learning patterns and their attitudes towards open- ended (OE) and multiple choice (MC) examinations, among students in higher education. The results reveal two patterns of relationships between the learning- related variables and the assessment attitudes. Students with good learning skills, who have high confidence in their academic ability, tend to prefer the constructed response type of assessment over the multiple choice type. And vice versa, students with poor learning skills, who tend to have low confidence in their academic ability, prefer the choice over the constructed- response type of assessment. The other pattern shows that low test anxiety measures were related to positive attitudes towards the OE format. Students with high test anxiety, have more unfavourable attitudes towards the OE format and a preference to the choice- response type, probably because it puts less demanding requirements on their information processing capacity during the testing situation where that capacity is occupied by worries and test- irrelevant thoughts (e.g. Hembree, 1988). In addition, this study also indicated gender differences, with males having more favourable attitudes towards the choice response format than females (Birenbaum & Feldman, 1998). These gender differences were attributed to a personality dimension of risk- taking, with females being more reluctant than males to guess on MC questions and being more likely to leave items blank (e.g. Ben- Shakhar & Sinai, 1991).

Birenbaum (1997) also evidenced that differences in assessment preferences are to a relatively large extent related to learning strategies and orientations. It may therefore be argued that such factors affecting students' assessment preferences, might also affect performance on different assessment types and thus cause bias in the interpretation of the assessment scores. The implication of this study is a need for adapting the assessment to the examinee's affective as well as cognitive characteristics in order to enhance the validity of his/ her test score interpretation. The question concerning assessment preferences ought to be: who prefers what?, rather than: what is preferred by most? (Birenbaum, 1997).

Traub and MacRury (1990) reviewed the empirical research on Multiple choice (MC) and Free response (FR) tests since 1970, and indicated several tendencies. Overall, students report that they are influenced by the expectation that a test will be in MC or FR format. In particular, students report more positive attitudes towards MC tests on grounds that these tests are easier to prepare, are easier to take, and hold forth hope for higher relative scores. As to whether the expectation that a test will be in MC or FR format influences learning, several studies show significant effects. In these studies, the performance of a MC test by students expecting MC is not significantly different from that of students told to expect a FR test, but students expecting a FR test performed on the FR test significantly better than students told to expect a MC test. Thus, studying for an 'expected' FR test seemed to have prepared students equally well for a MC or a FR test, whereas studying for an 'expected' MC test did not prepare students to take a FR test (Traub & MacRury, 1990).

Portfolio Assessment

The overall goal of the preparation of a portfolio is for the learner to demonstrate and provide evidence that he or she has mastered a given set of learning objectives. Portfolios are more than thick folders containing student work. They are personalized, longitudinal representations of a student's own efforts and achievements, in which the integration of numerous facts to form broad and encompassing concepts is actively performed by the student instead of the instructor (Slater, 1996). Additionally, portfolios have an important impact in driving student learning and they have the ability to measure outcomes such as professionalism that are difficult to assess using traditional methods (Friedman Ben- David et al., 2001).

Slater (1996) found that overall, students like this alternative procedure for assessment. Portfolio assessment seems to remove the students' perceived level of 'test anxiety'. This reduction shows up in the way students attend to class discussions, relieved of their traditional vigorous note- taking duties. Students thought that they would remember what they were learning much better and longer than they would the material for other classes they took, because they had internalised the material while working with it, thought about the principles, and applied concepts creatively and extensively over the duration of the course. The most negative aspect of creating portfolios is that they spend a lot of time going over the textbook or required readings. Students report that they are enjoying time spent on creating portfolios and that they believe it helps them learn concepts (Slater, 1996).

Janssens, Boes and Wante (2002) investigated student teachers' perceptions of portfolios as an instrument for professional development, assessment and evaluation. The student teachers felt portfolios stimulated them to reflect and demonstrated their professional development as prospective teachers. They felt engaged in the portfolio- creating- process, but portfolio construction in itself was not sufficient. Students thought that supervision, in addition, was desirable and necessary. They saw portfolios as an instrument for professional development and personal growth, but advantages were especially seen in relation to evaluation. When students did not get grades for their portfolios, much lesser efforts were made to construct the portfolio. Although portfolios are an important source of personal pride, students thought portfolios were very time- consuming and expensive. Portfolios appear very useful in learning environments in which instruction and evaluation form integrated parts (Janssens, Boes, & Wante, 2002).

A critical comment is given by Challis (2001) who argues that portfolio assessment simply needs to be seen in terms that recognize its own strengths and its differences from other methods, rather than as a replacement of any other assessment methods and procedures.

Self assessment

Orsmond and Merry (1997) implemented and evaluated a method of student self- assessment. A comparison between the tutor and the student self- assessed mark on the poster assignments revealed an overall disagreement of 86%, with 56% of students over- marking and 30% under- marking. It is noticeable that poor students tend to over- mark their work, whilst good students tend to under- mark. If the individual marking criteria are considered, than the number of students marking the same as the tutor ranged from 31% to 62%. The agreement among students marks ranged from 51 to 65%. The individual evaluation forms revealed that students acknowledged the value of the self- marking task. They thought that self- assessment made them think more and felt that they learned more. Most of the students reported that self- assessment made them more critical of their work and they felt that they worked in a more structured way. Self- assessment is perceived as challenging, helpful and beneficial (Orsmond & Merry, 1997).

Mires, Friedman Ben- David, Preece and Smith (2001) undertook a pilot study to evaluate the feasibility and reliability of undergraduate medical student self- marking of degree written examinations, and to survey students' opinions regarding the process. In contrast to the study of Orsmond and Merry (1997), a comparison between the student's marks and the staff's marks, for each question and the examination as a whole, revealed no significant differences. Student self- marking was demonstrated to be reliable and accurate. If student marks alone had been used to determine passes, the failure rate would have been almost identical to that derived from staff marks. The students in this study, however, failed to acknowledge the potential value of self- marking in terms of feedback and as a learning opportunity, and expressed uncertainty over their marks. Students perceived many more disadvantages than advantages in the self- marking exercise. Disadvantages included: finding the process stressful, feeling that they could not trust their own marking and having uncertainties on how to mark, being to concerned about passing/ failing to learn from the exercise, worrying about being accused of cheating and hence having a tendency to under-mark, having the opportunity to 'cheat', finding the process tedious, considering it time consuming and feeling that the faculty were 'offloading' responsibility. Advantages included the feeling of some students that it was useful to know where they had gone wrong and that feedback opportunity was useful (Mires et al., 2001).

These two studies revealed interesting but quite opposite results. The different task conditions could serve as a plausible explanation. A first task condition that differs in both studies is the clarity of the marking criteria. In the second study, for each question the agreed correct answer was presented, while in the first study, only general marking guidelines were given. Another important task condition that differed, was the level of stress experienced in the situation. In the first study, the task formed a part of the practical work that students had to produce during laboratory time. This is in strong contrast to the second study in which the self marking task concerned an examination, of which the evaluative consequences are more severe. Students' primary concern was whether they failed or passed the examination. This stressful pre- occupation with passing and failing, is probably the reason why students could not acknowledge the potential value of the self- marking exercise for feedback purposes or as a learning opportunity.

Peer assessment

Segers and Dochy (2001) gathered quantitative and qualitative data from a research project which focused on different quality aspects of two new assessment forms in problem- based learning: peer assessment and the OverAll test. In the peer assessment case at Louvain, the pearson correlation values indicated that peer and tutor scores are significantly interrelated. The student self- scores are, to a minor extent, related to peer and tutor scores. These findings suggest that students especially experience difficulties in assessing themselves. Students' perceptions reveal that, on one hand, they are positive about self- and peer assessment as stimulating deep- level thinking and learning, critical thinking, and structuring the learning process in the tutorial group. On the other hand, the students have mixed feelings about being capable of assessing each other in a fair way. Most of them do not feel comfortable in doing so (Segers & Dochy, 2001).

Overall assessment

In the Maastricht case of the Segers and Dochy (2001) investigation, a written examination, namely the OverAll test, was used to assess the extent to which students are able to define, analyse, and solve novel, authentic problems. It was found that the mean score on this test was between 30% and 36%, which implies that the students master on average only one- third of the learning goals measured. Staff perceived these results as problematic. Two topic checklists reveal that the overall test seems to have a high curriculum (curriculum as planned) and instructional (curriculum as implemented) validity. Through the analysis of think- aloud protocols of students handling real- life problems, confirmatory empirical evidence of criterion validity (i.e. whether a student's performance on the overall test has anything to do with professional problem- solving) was found. For staff, the central question remained why students did not perform better on the OverAll Test. Therefore, students' perceptions of the learning- assessment environment were investigated. The students' negative answer to the statement "the way of working in the tutorial group fits the way of questioning in the OverAll Test" particularly struck the staff as contradictory. Although empirical evidence of curriculum validity was found, students did not perceive a match between the processes in the tutorial group and the way of questioning in the OverAll Test. Students felt that for the OverAll Test, they had to do more than reproducing knowledge, they had to build knowledge. The tutorial group was perceived as not effectively preparing students for this skills. Too many times, working in the tutorial groups was perceived as running from one problem to another, without really discussing the analysis and the solution of the problem. Additionally, the students indicated that they had problems with the novelty of the problems. During the tutorials, new examples, with slight variations to the starting problem are seldom discussed. The students suggested more profound discussions in the tutorial groups, and that analysing problems should be done in a more flexible way. In one of the modules, a novel case was structurally implemented and discussed in the tutorial groups on the basis of a set of questions similar to the Overall test questions. Students valued this procedure, and felt the need to do this exercise in flexible problem analysis, structurally in all modules (Segers & Dochy, 2001).

From both cases, the Louvain and the Maastricht case, it can be concluded that there is a mismatch between the formal learning environment as planned by the teachers and the actual learning environment as perceived by the students. Students' perceptions of the learning- assessment environment, based on former learning experiences and their recent experiences, have an important influence on their learning strategies and affect the quality of their learning outcomes. Therefore, they are a valid input for understanding why promises are not fulfilled. Moreover, looking for students' perceptions of the learning- assessment environment seems to be a valid method to show teachers ways to improve the learning- assessment environment (Segers & Dochy, 2001).

Simulation

Edelstein, Reid, Usatine and Wilkes (2000) conducted a study to assess how computer- based case simulations (CBX) and standardized patient exams (SPX) compare with each other and with traditional measures of medical students' performance. Both SPX and CBX allow students to experience realistic problems and demonstrate the ability to make clinical judgments without the risk of harm to actual patients. It was found that the CBX and the SPX had low to moderate statistically significant correlations with each other and with traditional measures of performance. Students' perceptions of the various types varied based on the assessment. Students' rankings of relative merits of the examinations in assessing different physician attributes evidenced that performance examinations measure different physician competency domains. Students individually and in subgroups do not perform the same on all tests, and they express sensitivity to the need for different purposes. A multidimensional approach to evaluation is the most prudent (Edelstein et al., 2000).

General perceptions about assessment

A series of studies do not focus on students' perceptions about a specific mode of assessment but more generally investigate students' perceptions about assessment. The study of Drew (2001) illustrates students' general perceptions about the value and purpose of assessment. Within the context of new modes of assessment, the Northumbria Assessment studies are often referred to. In these studies, different aspects of perceptions of students about new modes of assessment are elaborated upon: the consequential validity of alternative assessment and its (perceived) fairness, but also the relations between teacher's messages and student's meanings in assessment, and the hidden curriculum are investigated.

What helps students learn and develop in education: Drew (2001) describes the findings of a series of structured group sessions, which elicited students views on their learning outcomes, and what helped or hindered their development. The findings suggest that there are three areas (i.e. three contextual factors) that, together, comprise the context in which students learn, and which have a strong influence on how and if they learn: (1) course organization, resources and facilities, (2) assessment, and (3) learning activities and teaching. Set within this context is the student and his use of that context (i.e. four student- centred factors), relating to (a) students' self- management, (b) students' motivation and needs, (c) students' understanding and (d) students' need for support.

Within the context of 'assessment', these student- centred factors occur as follows: (a) students valued self management and, generally examinations were seen as less supportive of its development. Deadlines were not seen in themselves as unhelpful. They developed self- discipline, the ability to work under pressure and increased determination, but they were also seen as indicating when to work, rather than when work was to be completed. (b) Assessment, seen by the students as a powerful motivator, was regarded as a major vehicle for learning. However, a heavy workload could affect the depth at which they studied and on some courses students thought it should be lessened so that 'work doesn't just wash over students'. (c) In order to help them learn, students wanted to know what was expected- clear briefs and clear assessment criteria. (d) Students closely linked the provision of feedback with support. Effective feedback was critical to 'build self confidence, help us evaluate ourselves' and students wanted more of it. Students preferred 1:1 tutorials as a method to provide effective feedback, but they knew that staff pressures made this difficult. They disliked one- line comments and saw typed feedback sheets as excellent (Drew, 2001).

But is it fair: consequential validity of alternative assessment: Sambell, McDowell and Brown (1997) conducted a qualitative study of students' interpretations, perceptions and behaviours when experiencing different forms of alternative assessment, in particular its consequential validity (i.e. the effects of assessment on learning and teaching). They employed the case study methodology (Sambell, McDowell & Brown, 1997).

Effects of student perceptions of assessment on the process of learning: Broadly speaking, students often reacted very negatively when they discussed what they regarded as 'normal' or traditional assessment. Many students expressed the opinion that normal assessment methods had a severely detrimental effect on the learning process. Exams had little to do with the more challenging task of trying to make sense and understand their subject. By contrast, when students considered new forms of assessment, their views of the educational worth of assessment changed, often quite dramatically. Alternative assessment was perceived to enable, rather than pollute, the quality of learning achieved. Many made the point that for alternative assessment they were channelling their efforts into trying to understand, rather than simply memorize or routinely document, the material being studied. Yet, some students recognized that there was a gap between their perceptions of the type of learning being demanded and their own action. Several claimed they simply did not have the time to invest in this level of learning and some freely admitted they did not have the personal motivation (Sambell, McDowell & Brown, 1997).

Perceptions of authenticity in assessment: Many students perceived traditional assessment tasks as arbitrary and irrelevant. This did not make for effective learning, because they only aimed to learn for the purposes of the particular assessment point, with no intention of maintaining the knowledge in any long- term way. Normal assessment was seen as something they had to, a necessary evil that allowed them to accrue marks. The accompanying activities are described in terms of routine, dull artificial behaviour. Traditional assessment is believed to be inappropriate as a measure, because it appeared, simply to measure your memory, or in case of essay- writing tasks, to measure your ability to marshal lists of facts and details. Alternative assessment was believed to be fairer, because by contrast, it appeared to measure qualities, skills and competences which would be valuable in contexts other than the immediate context of assessment. In some of the cases, the novelty of the assessment method lay in the lecturer's attempt to produce an activity which would simulate a real life context, so students would clearly perceive the relevance of their academic work to broader situations outside academia. This strategy was effective and the students highly valued these more authentic ways of working. Alternative assessment enabled students to show the extent of their learning and allowed them to articulate more effectively precisely what they had digested throughout the learning program (Sambell, McDowell, & Brown, 1997).

Student perceptions of the fairness of assessment: The issue of fairness, from the student perspective, is a fundamental aspect of assessment which is often overlooked or oversimplified by the staff. To students, the concept of fairness frequently embraces more than simply the possibility of cheating: it is an extremely complex and sophisticated concept which students use to articulate their perceptions of an assessment mechanism, and it relates closely to our notions of validity. Students repeatedly expressed the view that traditional assessment is an inaccurate measure of learning. Many made the point that end- point assessment or evaluations, particularly examinations which took place only on one day, were actually considerably down to luck, rather than accurately assessing present performance. Often students expressed concern that it was too easy to leave out large portions of the course material, when writing essays or taking exams, and still do well in terms of marks. Many students felt quite unable to exercise any degree of control within the context of the assessment of their own learning. Normal assessment was done to them, rather than something in which they could play an active role. In some cases, students believed that what exams actually measured was the quality of their lecturer's notes and handouts. Other reservations that students blanketed under the banner of 'unfairness', included whether you were fortunate enough to have a lot of practice in any particular assessment technique in comparison with your peers (Sambell, McDowell & Brown, 1997). When discussing alternative assessment, many students believed that success more fairly depended on consistent application and hard work, not a last minute burst of effort or sheer luck. Students use the concept of fairness to talk about whether, from their viewpoint, the assessment method in question rewards, that is, looks like it is going to attach marks to, the time and effort they have invested in what they perceive to be meaningful learning. Alternative assessment was fair because it was perceived as rewarding those who consistently make the effort to learn rather than those who rely on cramming or a last- minute effort. In addition, students often claimed that alternative assessment represents a marked improvement: firstly in terms of the quality of the feedback students expected to receive, and secondly, in terms of successfully communicating staff expectations. Many felt that openness and clarity were fundamental requirements of a fair and valid assessment system. There were some concerns about the reliability of self and peer assessment, even though students valued the activity (Sambell, McDowell & Brown, 1997).

It can be concluded that students' perceptions of poor learning, lack of control, arbitrary and irrelevant tasks in relation to traditional assessment contrasted sharply with perceptions of high quality learning, active student participation, feedback opportunities and meaningful tasks in relation to alternative assessment (Sambell, McDowell & Brown, 1997).

The hidden curriculum: messages and meanings in assessment: Sambell and McDowell (1998) focus upon the similarities and variations in students' perspectives on assessment. At surface levels, there was a clear match between statements made by staff and the 'messages' received by students. Several themes emerged, indicating shifts in students' typifications of assessment. First, students consistently expressed views that the new assessment motivated them to work in different ways. Second, that the new assessment was based upon a fundamentally different relationship between staff and students, and third, that the new assessment embodied a different view of the nature of learning. At deep levels of analysis, it was found that students have their individual perspectives, all of which come together to produce many variants on a hidden curriculum. Students' motivations and orientations to study influence the ways in which they perceive and act upon messages about assessment. Students' views of the nature of academic learning influence the kinds of meaning they find in assessment tasks and whether they adopt an approach to learning likely to lead to understanding or go through the motions of changing their approach (Sambell & McDowell, 1998). Students' typifications of assessment, based on previous experience, especially in relation to conventional exams, also, strongly influence their approach to different assessment methods. In an important sense this research makes assessment problematical, because it suggests that students, as individuals, actively construct their own versions of the hidden curriculum from their experiences and typifications of assessment. This means that the outcomes of assessment as 'lived' by students are never entirely predictable, and the quest for a 'perfect' system of assessment is, in one sense, doomed from the outset (Sambell & McDowell, 1998).

Conclusion and discussion

Methodological reflections

Traditionally research with regard to human learning, was done from a first order perspective. This research emphasized the description of different aspects of reality, reality per se. Research on students' perceptions turned the attention to the learner and certain aspects of his world. This approach is not directed to the reality as it is, but more to how people view and experience reality. It is called a second- order perspective. The accent of this second- order perspective is on understanding and not on explanation (Van Rossum & Schenk, 1984). Both qualitative and quantitative research have been conducted to reveal this second- order perspective. The majority of the studies concerning students' perceptions about assessment, is quantitative in nature. More than half, 23 out of 36, of the studies were solely analysed by quantitative, mostly relational, methods. Only 11 investigations, not yet one third of the reviewed studies, are qualitative in nature. Two of the 36 reviewed studies do both use quantitative and qualitative methods. Popular data collection procedures within the quantitative research are the Likert type questionnaires (n= 35) and inventories (n= 7), for example: the Reaction To Test questionnaire (RTT) (Birenbaum & Feldman, 1998; Sarason, 1984), Clinical Skill Survey (Edelstein et al., 2000), Assessment Preference Inventory (API) and the Motivated Learning Strategies Questionnaire (MLSQ) (Birenbaum, 1997). Only a relatively small number of surveys (n= 7) was done in response to a particular task or, as it fits a subject such as assessment, in response to a test or examination. The most frequent used methods for data collection within the qualitative research, were open questionnaires or written comments (n= 4), think- aloud protocols (n= 1), semi- structured interviews (n= 10), and focus group discussions (n= 4). Observations (n= 5) and research of document sources (n= 7) were conducted to collect additional information. The method of 'phenomenography' (Marton, 1981) has been frequently used to analyse the qualitative data gathered. Most studies have a sample of 101 to 200 subjects (n= 11) and from 31 to 100 persons (n= 9). A relatively high number of studies (n= 6) has a sample size of less than 30 students. Three, and five investigations have respectively a sample of 201 to 300 subjects and more than 300 persons. The sample size of the two case studies (n= 13 cases), is unknown. As a consequence, one can state that reiterative conclusions from this set of studies can be seen as robust.

Overall conclusions and further research

The reviewed studies evidenced that students' perceptions about assessment and their approaches to learning are strongly related. The perceived characteristics of assessment seem to have a considerable impact on students' approaches, and vice versa. These influences can be both positive and/ or negative, however the study of Trigwell and Prosser (1991) suggests that deep approaches to learning are especially encouraged by assessment methods and teaching practices which aim at deep learning and conceptual understanding, rather than by trying to discourage surface approaches to learning. As a consequence, teaching methods and educational policy play an important role in creating this 'deep learning'. Students' perceptions about conventional and alternative assessment can equip us with valuable ideas and interesting tip- offs to bring this deep conceptual learning into practice.

Within conventional assessment practices, namely multiple choice and essay typed examinations, students perceive the multiple choice format as more favourable than the constructed response/ essay items, especially students' perceptions on the perceived difficulty, lower anxiety and complexity, and higher success expectancy give preference to this examination format. Curiously, over the past few years, multiple choice type tests have been the target of severe public and professional attack on various grounds. Indeed, the attitude and semantic profile of multiple choice exams emerging from the examinee's perspective is largely at variance with the unfavourable and negative profile of multiple choice exams often emerging from some of the anti- test literature (Zeidner, 1987). However, within the group of students some remarkable differences are found. For example, students with good learning skills and students with low test anxiety rates, both seem to favour the essay type exams, while students with poor learning skills and low test anxiety have more unfavourable feelings towards this assessment mode. It was also found that this essay type of examination goes together with deep(er) approaches to learning than multiple choice formats. Some studies found gender effects, with females being less favourable towards multiple choice formats than to essay examinations (Birenbaum & Feldman, 1998).

When students discuss alternative assessment, these perceptions about conventional assessment formats, contradict strongly with the students' more favourable perceptions towards alternative methods. Learners, experiencing alternative assessment modes, think positive about new assessment strategies, such as: portfolio assessment, self and peer assessment, simulations. From students' point of view, assessment has a positive effect on their learning and is 'fair' when it: (1) Relates to authentic tasks, (2) Represents reasonable demands, (3) Encourages students to apply knowledge to realistic contexts, (4) Emphasis the need to develop a range of skills, and (5) Is perceived to have long- term benefits (Sambell, McDowell & Brown, 1997). Although students acknowledge the advantages of these methods, some of the students' comments put this overall positive image of alternative assessment methods into perspective. Different examination or task conditions can interfere. For example, 'reasonable' work- load is a pre-condition of good studying and learning (Chambers, 1992). Sometimes, a mismatch was found between the formal curriculum as intended by the educator and the actual learning environment as perceived by the students. Furthermore, different assessment methods seem to assess various skills and competences. It is important to value each assessment method, within the learning environment for which it is intended, and taking its purposes and skills to be assessed into consideration.

The literature and research on students' perceptions about assessment is relatively limited. Besides the relational and semi- experimental studies on students' approaches to learning and studying in relation to students' expectations, preferences and attitudes towards assessment which is well known, especially the research on students' perceptions about particular modes of assessment is restricted. Most results are consistent with the overall tendencies and conclusions. However, some inconsistencies and even contradictory results are revealed within this review. Further research can elucidate these results and can provide us with additional information and evidence on particular modes of assessment in order to gain more insight in the process of student learning.

This review has tried and hopefully succeeded to provide educators with an important source of inspiration, namely students' perceptions about assessment and its influences on student learning, which can be an important guide in their reflective search to improve their teaching and assessment practices, and as a consequence, to achieve a higher quality of learning and education.

References

Ashworth, P., & Bannister, P. (1997). Guilty in whose eyes? University students' perceptions of cheating and plagiarism in academic work and assessment. Studies in Higher Education, 22 (2), 187-203.

Ben- Shakar, G., & Sinai, Y. (1991). Gender differences in multiple- choice tests: the role of differential guessing tendencies. Journal of Educational Measurement, 28, 23-35.

Birenbaum, M. (1990). Test anxiety components: comparison of different measures. Anxiety research, 3, 149-159.

Birenbaum, M. (1996). Assessment 2000: towards a pluralistic approach to assessment. In M. Birenbaum & F. J. R. C. Dochy (Eds.), Alternatives in assessment of achievements, learning processes and prior knowledge. Evaluation in education and human services (pp. 3-29). Boston, MA: Kluwer Academic Publishers.

Birenbaum, M. (1997). Assessment preferences and their relationship to learning strategies and orientations. Higher Education, 33, 71-84.

Birenbaum, M., & Feldman, R. A. (1998). Relationships between learning patterns and attitudes towards two assessment formats. Educational Research, 40 (1), 90-97.

Birenbaum, M., Tatsuoka, K. K., & Gutvirtz, Y. (1992). Effects of response format on diagnostic assessment of scholastic achievement. Applied psychological measurement, 16 (4), 353-363.

Boes, W., & Wante, D. (2001). Portfolio: het verhaal van de student in ontwikkeling/ Portfolio: the story of a student in development [Unpublished dissertation]. Katholieke Universiteit Leuven, Department of Educational Sciences.

Challis, M. (2001). Portfolios and assessment: meeting the challenge. Medical Teacher, 23 (5), 437-440.

Chambers, E. (1992). Work- load and the quality of student learning. Studies in Higher Education, 17 (2), 141-154.

De Corte, E. (1996). Active learning within powerful learning environments/ Actief leren binnen krachtige leeromgevingen. Impuls, 26 (4), 145-156.

Dochy, F., Segers, M., & Buehl, M. M. (1999). The relation between assessment practices and outcomes of studies: the case of research on prior knowledge. Review of educational research, 69 (2), 147-188.

Drew, S. (2001). Perceptions of what helps learn and develop in education. Teaching in Higher Education, 6 (3), 309-331.

Edelstein, R. A., Reid, H. M., Usatine, R., & Wilkes, M. S. (2000). A comparative study of measures to evaluate medical students' performances. Academic Medicine, 75 (8), 825-833.

Eizenberg, N. (1988). Approaches to learning anatomy: developing a programme for preclinical medical students. In P. Ramsden (Ed.), Improving learning: new perspectives. London: Kogan Page.

Entwistle, N., & Entwistle, A. (1997). Revision and experience of understanding. In F. Marton, D. Hounsell, & N. Entwistle (Eds.), The experience of learning. Implications for teaching and studying in higher education [second edition] (pp. 146-158). Edinburgh: Scottish Academic Press.

Entwistle, N. J. (1991). Approaches to learning and perceptions of the learning environment. Introduction to the special issue. Higher Education, 22, 201-204.

Entwistle, N. J., & Entwistle, A. (1991). Contrasting forms of understanding for degree examinations: the student experience and its implications. Higher Education, 22, 205-227.

Entwistle, N. J., & Ramsden, P. (1983). Understanding student learning. London: Croom Helm.

Entwistle, N. J., & Tait, H. (1990). Approaches to learning, evaluations of teaching, and preferences for contrasting academic environments. Higher Education, 19, 169-194.

Entwistle, N., McCune, V., & Walker, P. (2001). Conceptions, styles, and approaches within higher education: analytical abstractions and everyday experience. In Sternberg and Zhang, Perspectives on cognitive, learning and thinking styles (pp. 103-136). NJ: Lawrence Erlbaum Associates.

Entwistle, N., & Tait, H. (1995). Approaches to studying and perceptions of the learning environment across disciplines. New directions for teaching and learning, 64, 93-103.

Flint, N. (2000, December). Culture club. An investigation of organisational culture. Paper presented at the Annual Meeting of the Australian Association for Research in Education, Sydney.

Franklyn- Stokes, A., & Newstead, S. E. (1995). Undergraduate cheating: who does what and why? Studies in Higher Education, 20 (2), 159-172.

Friedman Ben- David, M., Davis, M. H., Harden, R. M., Howie, P. W., Ker, J., & Pippard, M. J. (2001). AMEE Medical Education Guide No. 24: Portfolios as a method of student assessment. Medical Teacher, 23 (6), 535-551.

Hembree, R. (1988). Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58, 47-77.

Hounsell, D. (1997a). Contrasting conceptions of essay- writing. In F. Marton, D. Hounsell, & N. Entwistle (Eds.), The experience of learning. Implications for teaching and studying in higher education [second edition] (pp. 106-126). Edinburgh: Scottish Academic Press.

Hounsell, D. (1997b). Understanding teaching and teaching for understanding. In F. Marton, D. Hounsell, & N. Entwistle (Eds.), The experience of learning. Implications for teaching and studying in higher education [second edition] (pp. 238-258). Edinburgh: Scottish Academic Press.

Janssens, S., Boes, W., & Wante, D. (2001). Portfolio: een instrument voor toetsing en begeleiding/ Portfolio: an instrument for evaluation and coaching. In F. Dochy, L. Heylen, & H. Van de Mosselaer (Eds.), Assessment in onderwijs/ Assessment in Education. Utrecht, Netherlands: LEMMA.

Kniveton, B. H. (1996). Student perceptions of assessment methods. Assessment and Evaluation in Higher Education, 21 (3), 229-238.

Lomax, R. G. (1996). On becoming assessment literate: an initial look at preservice teachers' beliefs and practices. Teacher educator, 31 (4), 292-303.

Marlin, J. W. Jr. (1987). Student perception of End-of-Course-Evaluations. Journal of Higher Education, 58 (6), 704-716.

Marton, F. (1976). On non- verbatim learning. II. The erosion of a task induced learning algorithm. Scandinavian Journal of Psychology, 17, 41-48.

Marton, F. (1981). Phenomenography- describing conceptions of the world around us. Instructional Science, 10, 177-200.

Marton, F., & Säljö, R. (1997). Approaches to learning. In F. Marton, D. Hounsell, & N. Entwistle (Eds.), The experience of learning. Implications for teaching and studying in higher education [second edition] (pp. 39-59). Edinburgh: Scottish Academic Press.

Meyer, D. K., & Tusin, L. F. (1999). Pre-service teachers' perceptions of portfolios: process versus product. Journal of Teacher Education, 50 (2), 131-139.

Mires, G. J., Friedman Ben- David, M., Preece, P. E., & Smith, B. (2001). Educational benefits of student self- marking of short- answer questions. Medical Teacher, 23 (5), 462-466.

Nolen, S. B., & Haladyna, T. (1990). Personal and environmental influences on students' beliefs about effective study strategies. Contemporary Educational Psychology, 15 (2), 116-130.

Orsmond, P., Merry, S., et al. (1997). A study in self- assessment: tutor and students' perceptions of performance criteria. Assessment and Evaluation in Higher Education, 22 (4), 357-369.

Ramsden, P. (1981). A study of the relationship between student learning and its academic context [Unpublished Ph.D. thesis]. University of Lancaster.

Ramsden, P. (1997). The context of learning in academic departments. In F. Marton, D.,Hounsell, & N. Entwistle (Eds.), The experience of learning. Implications for teaching and studying in higher education [second edition] (pp. 198-217). Edinburgh: Scottish Academic Press.

Richardson, J. T. E. (1995). Mature students in higher education: II. An investigation of approaches to studying and academic performance. Studies in Higher Education, 20 (1), 5-17.

Rickinson, B. (1998). The relationship between undergraduate student counseling and successful degree completion. Studies in Higher Education, 23 (1), 95-102.

Säljö, R. (1975). Qualitative differences in learning as a function of the learner's conception of a task. Gothenburg: Acta Universitatis Gothoburgensis.

Sambell, K., & McDowell, L. (1998). The construction of the hidden curriculum: messages and meanings in the assessment of student learning. Assessment and Evaluation in Higher Education, 23 (4), 391-402.

Sambell, K., McDowell, L., & Brown, S. (1997). 'But is it fair?': an exploratory study of student perceptions of the consequential validity of assessment. Studies in Educational Evaluation, 23 (4), 349-371.

Sarason, I. G. (1984). Stress, anxiety and cognitive interference : reactions to tests. Journal of Personality and Social Psychology, 46 (4), 929-938.

Schmelkin, L. P., Spencer, K. J., & Larenberg, L. J. (1997). Students' perceptions of the weight faculty place on grading criteria. Perceptual and Motor Skills, 84 (3), 1444-1446.

Segers, M., & Dochy, F. (2001). New assessment forms in Problem- based Learning: the value- added of the students' perspective. Studies in Higher Education, 26 (3), 327-343.

Slater, T. F. (1996). Portfolio assessment strategies for grading first- year university physics students in the USA. Physics Education, 31 (5), 329-333.

Thomas, P. R., & Bain, J. D. (1984). Contextual dependence of learning approaches: the effects of assessment. Human Learning, 3, 227-240.

Treatwell, I., & Grobler, S. (2001). Students' perceptions on skills training in simulation. Medical Teacher, 23 (5), 476-482.

Trigwell, K., & Prosser, M. (1991). Improving the quality of student learning: the influence of learning context and student approaches to learning on learning outcomes. Higher Education, 22, 251-266.

Traub, R. E., & MacRury, K. (1990). Multiple choice vs. free response in the testing of scholastic achievement. In K. Ingenkamp & R. S. Jager (Eds.), test und tends 8: jahrbuch der pädagogischen diagnostik (pp. 128-159). Weinheim und Base: Beltz Verlag.

Van Rossum, E. J., & Schenk, S. M. (1984). The relationship between learning conception, study strategy and learning outcome. British Journal of Educational Psychology, 54 (1), 73-83.

Zeidner, M. (1987). Essay versus multiple-choice type classroom exams: the student's perspective. Journal of Educational Research, 80 (6), 352-358.

Zoller, U., & Ben-Chaim, D. (1988). Interaction between examination-type anxiety state and academic achievement in college science: an action- oriented research. Journal of Research in Science Teaching, 26 (1), 65-77.

This document was added to the Education-line database on 28 October 2002