"Be more critical!": Rethinking Assessment Feedback
School of Cultural Studies, Sheffield Hallam University, UK
Paper presented at the British Educational Research Association Conference, Cardiff University, September 7-10 2000
This paper develops themes emerging from an ongoing three year research project investigating the meaning and impact of assessment feedback on written course work assignments for students in Higher Education. This is an important issue at present, reflected in a growth in debates on the purpose of assessment and feedback and its impact on student learning. Early findings and a review of the literature suggest that feedback involves a unique process of communication situated within a particular social context. This process and needs to be understood as involving a complex interplay of power relations, competing discourses, emotion and identity.
"Be more critical!", "Your argument needs to be more academic", "More use of analysis would have helped", "You need to think carefully about your structure" - these are the kinds of written comments often confronting higher education (HE) students. But what do they mean? How do students make sense of them?
Despite a rise in innovative assessment methods, in many contexts the written essay is still used (McCune, 1999) and written feedback comments continue to be the main (if not only) source of formative feedback for the majority of today's students. This feedback is important because it is essential for learning and can play a significant role in students' development by providing knowledge required for improvement. (Hinett and Weeden, forthcoming; Gibbs, 1999; Ecclestone, 1998; Hyland, 1998).
There is some evidence that most students read feedback comments. For example, the majority of the students involved in a study at Lancaster University (Turner, 1993) valued feedback and tried to make sense of tutors' comments. This is a finding supported in our own research (Higgins et al., 2000). Therefore, 'if students are going to take their tutors' responses seriously, then it matters very much what they contain' (Ivanič et al., 2000: 60).
However, students can experience difficulties when trying to use assessment feedback (for example, see Ding, 1998; Turner, 1993). They are a variety of reasons for this. For instance, students' heavy workloads may give little time to reflect on tutors' comments or feedback may not be sufficiently timely to be of any use. But I believe the most significant reason is that many students are simply unable to understand feedback comments and interpret them incorrectly. The purpose of this paper is not to identify these interpretations but develop a framework for accounting for them, since it is more important to understand what produces conceptions than the character of the (mis)conception itself (Strike and Posner, 1992, cited in Breen, 1999).
The process of giving and receiving feedback is a particularly complex and problematic form of communication, which takes place within a social context. Students produce work and tutors provide feedback on that work and this process is defined 'through implicit assumptions about what constitutes valid knowledge ... and the relationships of authority that exist around the communication of these assumptions' (Lea and Street, 2000: 45). Furthermore, feedback carries messages to students 'about university values and beliefs, about the role of writing in learning, about their identity as a student, and about their own competence and even character' (Ivanič et al., 2000: 47). Consequently, students' feelings and identities need to be recognised as playing a significant role in this process alongside notions of authority and implicit knowledge. Below is a discussion of the assessment feedback process in terms of notions of 'discourse', 'power', 'emotion' and 'identity'. This provides a framework for exploring student (mis)conceptions of tutors' written feedback comments.
Dai Hounsell found differences in conceptions of essay writing among students which have affinities with Marton and Säljö's deep and surface approaches to learning. (Hounsell, 1987). By trying to explain these differences and why undesirable conceptions seem to persist, Hounsell suggests that rather than blaming it on a lack of guidance on what essays involve, the problem is a deeper one. He appeals to Rommetveit's notion of an 'architecture of inter-subjectivity'. Rommetveit (1979) pays attention to the social context of communication and argues that communication involves shared understandings which tend to be tacit and taken-for-granted. Hounsell applies this idea to his own work on feedback and suggests that it may fail because of the differential between students' and tutors' conceptions; students 'do not have a grasp of the assumptions about the nature of academic discourse underlying what is being conveyed to them' (Hounsell, 1987: 114). Therefore:
'conventional attempts to guide students ... may founder because of the exigencies of communication ... Such characteristic comments as 'you do not make your points clearly enough', 'this essay lacks structure' or 'too much irrelevant detail', do not have a meaning which is self evident' (Hounsell, 1987: 118).
We recognise the notion of a tacit, academic discourse, but argue that there is not likely to be one such discourse but many. Discourses underpinning feedback comments will vary by discipline and refer to different sets of ideas and beliefs about what can be said about a topic area and how it may be said (Baynham, 2000). However, other discourses are likely to affect the feedback provided thereby generating differences within disciplines also. For example, these discourses may reflect tutors' different roles within the university and their status.
Discourses can be seen as resources available to individuals to draw upon in order to formulate behaviour (Layder, 1997). The problem for students is that they may have difficulty accessing the discourses underpinning assessment and feedback (McCune, 1999). But why? The roles of tutors, students and external bodies may provide the answer.
Tutors develop and hold values and beliefs about ways of 'knowing' and 'doing' which are specific to each discipline and contribute to the discourse of that discipline. In the context of teacher training, Tomlinson (1999) suggests that teachers bring with them to their course an implicit knowledge of classroom life and 'normal' practices (based on personal experience such as once being students themselves), a 'tacit' knowledge of what to do and how to respond, and consciously held concepts and views regarding teaching. We would argue that such values and beliefs are developed by tutors in HE. Moreover, by constituting part of each discipline's discourse, they are enduring and relatively stable since they are reproduced as tutors draw on these discourses and the social practices associated with them.
But as we have already suggested, values and beliefs may vary within disciplines (see Barnett, 1990; Lea and Street, 2000). Therefore, one tutor's advice may contradict another's within the same department. This is evidenced by Ivanič et al's (2000) analysis of selections of written feedback comments that varied in terms of the pattern of the comments, where they were written, and what they were written with. This finding is also supported by our own analysis of samples of feedback, which found comments to vary extensively in terms of type, quantity and tone (Higgins et al., 2000). Although this could partly be a result of differences in the accuracy of student texts, it is suggested that other contextual factors play a role (Hyland, 1998). We suspect that these factors relate to particular values and beliefs and dispositions developed from individuals' personal experiences and unique biography, and tutors drawing on, to some extent, other discourses to formulate their behaviour. Therefore, while we argue that the way feedback is provided and the nature of that feedback is predominantly underpinned by a dominant, subject specific discourse, personal and individual vales and beliefs, and the employment of other discourses play a mediating role.
ii) External bodies
Assessment criteria which underpin the language of feedback are increasingly falling under the influence of external bodies. The values and beliefs underpinning them (both explicitly and implicitly) are therefore not just those of tutors, but of political bodies (such as the Quality Assurance Agency). These values and beliefs are likely then to also constitute part of each discipline-specific discourse.
It is not simply the case, however, that assessment criteria and the language and form of feedback are underpinned by discourses which are constituted (at least in part) by the values and beliefs of staff and external bodies, and by tutors' idiosyncratic traits. Rather, students' own actions may initially compound their misconceptions of feedback. This point will be explored in more detail in following sections of this paper, but the point for now is that the student body is increasingly heterogeneous and students are likely to come to university with a wide variety of past experiences and dispositions. They will therefore bring with them a variety of conceptions of learning, writing and assessment. Students in a study by McCune (1999) exhibited different conceptions of essay writing and these seemed to partly mediate their interpretations of advice in feedback. Therefore, we should expect that different students will inevitably interpret tutors' comments differently and that no matter what feedback is provided, a number of students will misunderstand comments and fail to connect with them.
Feedback then, is likely to be underpinned, to a large extent, by tacit, subject specific discourses and mediated by implicit values and beliefs. Many students' (inexperienced in the ways of HE in general, and different disciplines in particular) may misunderstand the messages that tutors' comments convey since they will struggle to access these discourses due to their 'taken-for-granted' and 'hidden' nature. But to develop a better understanding of this, we need to look further.
If we locate the process of giving and receiving feedback in a social context, then we must also consider the role of power. Power is ubiquitous in social life and can be viewed as closely linked to discourses and associated practices (Layder, 1997). If discourses involve ideas and beliefs about what can be said and how things should be done within a subject area, and associated practices are the patterns of behaviour associated with these ideas and beliefs, then they must inextricably be linked to power by 'conferring capabilities on those who deploy them as well as those who are "subjected" to their deployment' (Layder, 1997: 84). For example, if we imagine a discourse of physics or physics teaching, then there will be particular values and beliefs regarding what counts as valid knowledge of the physical world and how it is acquired and then demonstrated. The associated practices will include the active seeking, interpreting and demonstrating of relevant knowledge. Different levels of access to this discourse and its associated practices distinguish individuals in terms of levels of expertise and authority. Discourses therefore articulate and effect social positions and relations (and provide markers of normality) (Layder, 1997).
Through their education, training and experience (and 'official' recognition of this), 'expert' tutors have a level of access to appropriate discourses and practices that 'amateur' students do not. Tutors are able to exercise power. They have the legitimate means to exert 'control' over students through their academic authority. This power differential is recognised and by most in HE. For example, students readily perceive tutors to be 'experts' who posses a level of 'know what' and 'know how' over and above that of the undergraduate. In the main, the exercise of power by tutors is not directed to sinister ends, but to help students learn successfully (although some might argue that the power of the tutor is used to protect their authority over knowledge). Layder (1997) argues, however, that power is not uni-directional and a feature of structural differences in social positions. As well as being able to resist the exercise of power, students are able to deploy power through the personal resources they have at their disposal (Layder, 1997).
The central point to make is that the process of giving and receiving feedback needs to be understood within the context of a particular tutor-student relationship based on an unequal distribution of power where the tutor is accepted by students as an authority figure and expert (Hinett and Weeden, forthcoming; Hyland, 2000). The implications of this relationship for understanding the feedback process are discussed below in relation to notions of emotion and identity.
According to Layder (1997), emotion is frequently allied to power and this is certainly the case in the context of learning. Adopting the view of Bloom (1957) and Barnett (1997), Hinett and Weeden argue that 'learning is a complex relationship between cognition, action, intuition and emotion' (forthcoming, italics added), while Boud and Walker (1998: 194) claim that 'emotions are central to all learning'. Receiving feedback is also an emotional business. For example, Hyland argues that 'writing is an intensely personal activity, and students' motivation and confidence in themselves as writers may be adversely affected by the feedback they receive' (Hyland, 1998: 279), while McCune suggests that the effects of feedback - where they are a change in students' conceptions of learning - can be linked to notions of confusion, anxiety and crisis of confidence (1999).
It is of little surprise that this is the case since students occupy the role of 'novice' within the learning context. Students read feedback with an implicit understanding of the power differential (outlined above) (Hyland, 2000), and by perceiving the tutor as a powerful, authority figure, they are likely to connect evaluations of their work to evaluations of themselves. For example, the findings of Turner's (1993) research suggested that discouraging feedback affected the student subjects' self-esteem, confidence, and whole approach to a course. Students are likely to be affected emotionally by feedback on work which is a presentation of their own 'amateur' knowledge, ideas, understandings (and so on) to a perceived 'expert' judge.
We can support this idea through the work of Layder (1997), Ingleton (1999), Barbalet (1998), and Scheff (1990; 1991; and 1997). For Scheff (1990), 'pride' and 'shame' are the primary emotions. Inherent in social life is the 'emotion-deference system' which functions continuously, yet almost at an invisible level. It is a form of social control in which individuals seek the pleasure of the emotion of pride and avoid the displeasure of the emotion of shame.
Scheff (1990) asserts that the emotion-deference system is a powerful one. Scheff (drawing on the work of Helen Lewis (1971) who suggests that anger can quickly and often follow shame) argues that when an individual perceives a rejection, form of criticism or insult from another, the system might produce 'a chain reaction of shame and anger between and within the interactants. This explosion is usually very brief, perhaps a few seconds. But it can also take the form of a bitter hatred lasting a lifetime' (Scheff, 1990: 76). Yet normally, the system operates by aligning the thoughts, feelings and actions of people in society because 'mutual conformity and respect lead to pride and fellow-feeling and further conformity, and so on' (Layder, 1997: 60). So for Scheff (1991), shame generates alienation while pride - its opposite - partners solidarity and togetherness (Ingleton, 1999).
We can elaborate on the role of emotion in learning and feedback by returning to the ideas of Dai Hounsell. As suggested elsewhere, Hounsell's (1987) work is concerned with the nature of academic discourse. He suggests that students may become:
'locked into a cycle of deprivation as far as constructive feedback is concerned. Since feedback fails to connect, it comes to be viewed as insignificant or invalid, and so is not given considered attention. At the same time the activity within which it is offered is seen increasingly as unrewarding, and so it is approached perfunctorily, thus rather lessening the likelihood that a more appropriate conception might be apprehended' (Hounsell, 1987: 117):
In a similar way, students may become locked in an emotional cycle of deprivation. Consider the following. A student misunderstands assessment criteria (due to the tacit nature of the academic discourse underpinning it), and produces work deemed by the tutor to be poor. The consequent feedback is prceived by the student to constitute a negative criticism or even rejection of both his work and even his character. The student considers himself to have been judged as inadequate by an expert tutor and as a result, feels embarrasment, humiliation and guilt (which are all feelings included in Scheff's (1990) definition of 'shame'). Following this experience, he is less willing to engage with the tutor and the discipline and partly 'withdraws' from the learning situation in an attempt to avoid emotional experiences of a similar nature. Consequently, the 'gap' between tutor and student in terms of access to discourse, power and authority is widened by a somewhat 'hidden' emotional dimension. This is not a generalisation of all student behaviour. Rather it illustrates the central role of emotion in the process of giving and receiving feedback and its relationship to power and discourse.
Notions of discourse, power and emotion are, however, also essentially tied to the concept of identity formation. Drawing on the work of J. Turner (1988), Layder (1997) argues that individuals have the capacity to present themselves to different audiences in different ways. How they do this will be affected by a number of factors. A person's life experiences and individual development over time suggests that students' experiences will be mediated by, and incorporated into current and past experiences. For example, particular findings reported by Hyland (1998) (although based on only two case studies) suggested that differences in students' uses of feedback seemed to be a result of past experiences, present writing ability and their attitudes toward writing.
Students constitute an increasingly heterogeneous body, with diverse backgrounds, so their predispositions and approaches will vary greatly. However, at a more macro level, the structural conditions that actors experience will mediate the development of their identities. For example, an individual's class position might affect access to writing skills and this may produce feelings of shame and foster a lack of self-confidence. That is, if a student is made to feel that they have poor writing skills then this may affect their self confidence and therefore their self-identity and the way they present themselves to others in future. Therefore, students positions within a power relationship, their use of discourses and social practices, the constraints of the university, their individual predispositions and their life experiences, will all affect how they both perceive and present themselves. Furthermore, there is not a one-way influence of the projection of different selves on audiences, but a two-way interaction whereby audiences will have their own mediating demands and expectations (Layder 1997, adapted from Goffman).
This understanding of student identity suggests a number of implications for the feedback process which relate closely to those identified in earlier parts of this paper. From an anthropological perspective, Strathern (2000) argues that what is presented to audiences is what is perceived (by those doing the presenting) to lead to successful judgements rather than the true state of the individual. Similarly, as a result of the power differential between tutor and student and the variety of emotional responses that ensue, students may present themselves to the 'audience' or tutor in ways that are designed to avoid feelings of shame (rather than revealing their true state) and effect feelings of pride.
This suggests particular difficulties for the giving and receiving of feedback. Tutors may be unaware of particular student difficulties or misconceptions which remain hidden. Feedback is therefore less likely to connect if it fails to address certain problems or concerns. At the same time, if students fail to grasp the nature of the tacit discourse underpinning the language of feedback, perhaps viewing it as meaningless and without use, they are likely to enter into a cycle of 'emotional' and 'interpretative' deprivation (as suggested above), where they become increasingly detached from the assessment process (thus failing to develop more sophisticated conceptions) and become even more guarded about how they present themselves in order to avoid negative emotional responses. But of course, such avoidance will only serve to keep further weaknesses 'hidden' and leave tutors providing feedback which seems even less relevant.
Understanding the process of giving and receiving feedback as a problematic form of communication situated within a particular social context allows us to focus on a central problem of students' misconceptions of tutors' comments, and to explore the internal dynamics of this process in relation to concepts of discourse, power, emotion and identity. While some of the implications for students' reactions to feedback are speculative, this paper has considered the interlocking roles of these concepts and, it is hoped, contributed to the development of a framework for understanding the nature of the assessment feedback process within HE which may provide interesting and, ultimately, useful insights.
If we accept the problematic nature of assessment feedback and some of the difficulties inherent within the process outlined in this paper, then one implication for improving practice is to question and discuss the values and beliefs underpinning the language of assessment. While it may be difficult for tutors to provide feedback that is useful for all students, since individual students will have different preferences and expectations (relating to a number of factors such as cultural differences) (Hyland, 1998), the work of Hounsell (1987) suggests that assumptions underpinning feedback must be shared before it can be effective. One solution could be to 'explicitly discuss issues of discipline-specific discourse in tutorial classes' (McCune, 1999: 22) involving a discussion between tutors and students re expectations and aims of feedback in order to avoid 'miscommunication' (Hyland, 1998). Moreover, Hyland (2000) argues that clear and comprehensive communication between students and tutors, not only on specific writing problems, but also approaches to learning and writing and feedback strategies, is essential for effective feedback since the stances that tutors and students adopt with relation to these issues are usually implicit rather than explicit and so not understood by either party. These strategies may help students to understand what tutors may really mean by comments such as "Be more critical!".
Barbalet, J. (1998) Emotion, Social Theory and Social Structure (Cambridge: Cambridge University Press).
Barnett, R. (1990) The Idea of Higher Education (Buckingham: SRHE and Open University Press).
Baynham, M. (2000) Academic Writing in New and Emergent Discipline Areas, in: Lea, M. and Stierer, B. (Eds.) Student Writing in Higher Education: New Contexts (Buckingham: SRHE and Open University Press).
Boud, D. and Walker, D. (1998) Promoting Reflection in Professional Courses: the challenge of context Studies in Higher Education, Vol. 23 (2), pp. 191-206.
Brass, K. (1999) Using timely feedback on student progress to facilitate learning, paper presented to: HERDSA Annual International Conference, Melbourne, 12-15 July.
Breen, R. (1999) Student motivation and conceptions of disciplinary knowledge, paper presented to: HERDSA Annual International Conference, Melbourne, 12-15 July.
Ding, L. (1998) Revisiting assessment and learning: implications of students' perspectives on assessment feedback, paper presented to: Scottish Educational Research Association, University of Dundee, September 25-26.
Ecclestone, K. (1998) "just tell me what to do" barriers to assessment-in-learning in higher education, paper presented to: Scottish Educational Research Association Annual Conference, Dundee, 24-26 September.
Gibbs, G. (1999) Using Assessment Strategically to Change the Way Students Learn, in: Brown, S. and Glasner, A. (Eds.) Assessment Matters in Higher Education: Choosing and Using Diverse Approaches (Buckingham: SRHE and Open University Press).
Higgins, R., Hartley, P. and Skelton, A. (2000) What do students really learn from tutors' comments?, in: Graal, M. and Clark, R. (Eds.) Writing Development in Higher Education: Partnerships across the curriculum, Proceedings of the 6th Annual Writing Development in Higher Education Conference, University of Leicetser, April 1999 (Leicester: Teaching and Learning Unit, University of Leicester).
Hinett, K. and Weeden, P. How am I doing? Developing Critical Self Evaluation in Trainee Teachers, forthcoming in: Quality in Higher Education November, 2000.
Hinkle, M. A. (1997) Transcriptional and compositional responses to student writing: designing courses with social climates supportive of written expression, in: Rust, C. and Gibbs, G. (Eds.) Improving Student Learning Through Course Design (Oxford: Oxford Centre for Staff and Learning Development).
Hounsell, D. (1987) Essay writing and the quality of feedback, in: J. Richardson, M.W. Eysenck and D.W. Piper (Eds.) Student Learning: research in education and cognitive psychology (Milton Keynes: Open University Press).
Hyland, F. (2000) ESL writers and feedback: giving more autonomy to students Language Teaching Research, 4 (1), pp. 33-54.
Hyland, F. (1998), The Impact of Teacher Written Feedback on Individual Writers Journal of Second Language Writing, 7 (3), pp. 255-286.
Ingleton, C. (1999) Emotion in learning: a neglected dynamic, paper presented to: HERDSA Annual International Conference, Melbourne, 12-15 July.
Ivanič, R., Clark, R. and Rimmershaw, R. (2000) What Am I Supposed to Make of This? The Messages Conveyed to Students by Tutors' Comments, in: Lea, M. R. and Stierer, B. (Eds.) Student Writing in Higher Education: New Contexts (Buckingham, SRHE and Open University Press).
Layder, D. (1997) Modern Social Theory: Key Debates and New Directions (London, UCL Press).
Lea, M. R. and Street, B. (2000) Student Writing and Staff Feedback in Higher Education: An Academic Literacies Approach, in: Lea, M. R. and Stierer, B. (Eds.) Student Writing in Higher Education: New Contexts (Buckingham: SRHE and Open University Press).
McCune, V. (1999) Development of First Year Students' Conceptions of Essay Writing, paper presented to: 8th European Conference for Research on Learning and Instruction, August 24-28, Göteborg, Sweden.
Rommetveit, R. (1979) On the Architecture of Intersubjectivity, in: Rommetveit, R. and Blakar, R. M. (Eds.) Studies of Language Thought and Verbal Communication (London: Academic Press).
Scheff, T. J. (1997) Emotions, the social bond and human reality, (Cambridge: Cambridge University Press).
Scheff, T. J. and Retzinger, M. (1991) Emotions and Violence (Massachusetts: Jossey-Bass Inc.).
Scheff, T. J. (1990) Microsociology: discourse, emotion and social structure (Chicago: University of Chicago Press).
Strathern, M. (2000) The Tyranny of Transparency British Educational Research Journal, Vol. 26 (3), pp. 309-321.
Thompson, C. (1999) Critical thinking: what is it and how do we teach it in English for Academic Purposes (EAP) programs?, paper presented to: HERDSA Annual International Conference, Melbourne, 12-15 July.
Tomlinson, P. (1999) Conscious Reflection and Implicit Learning in Teacher Preparation. Part II: implications for a balances approach Oxford Review of Education, Vol. 25 (4), pp. 533-544.
Turner, K. (1993) You do not seem to have understood the question Unpublished MA Dissertation, Department of Educational Research, Lancaster University.