Paper presented at the British Educational Research Association Annual Conference
(September 11-14 1997: University of York)
In 1994 the Thinking Through Geography (TTG) Group (a curriculum development group of university-based teacher educators and geography teachers) was engaged by the Schools Curriculum and Assessment Authority (SCAA) to design one of the optional tasks and tests for teacher assessment of Key Stage 3 (KS3) National Curriculum (NC) Geography (SCAA, 1996). It was inferred by SCAA that we should feel able to explore non-traditional forms of assessment, as more traditional formats would be generated by other consultant groups. We decided to use a format that we have come to term a mystery. The characteristics of mysteries as developed by the group are as follows:
The teacher plays a very important role in introducing a mystery, as they are different from much classroom work. Two critical features of the introduction are, therefore, getting the pupils interested and establishing an expectation towards the task i.e. that it will need effort, it will not be done in 2 minutes and the group needs to listen to each other.
We were encouraged to use plate tectonics (part of the KS3 Programmes of Study) as a possible context and using the template above one of teachers wrote The Kobe Earthquake Mystery. The mystery and other materials were extensively trialled by the group and written responses to the question were collected from several hundred pupils and analysed inductively to derive three levels of performance, a parameter set by SCAA.
Mysteries in the Classroom
Mysteries have become a popular teaching strategy among humanities teachers in North East England (Northumberland County Council, 1997) and to a lesser extent nationally through the writing and in-service endeavours of the group. More than 20 have been written for a variety of topics and subjects. Many teachers have commented with extreme enthusiasm on the collaborative work, motivation and understanding engendered. Davies (1997) reporting on a SCAA evaluation of the use of materials reported:
"The teacher at school A believed that the materials would appeal to teachers at schools such as his own where there was a high proportion of 'challenging' pupils and to teachers who actually want to look at how pupils develop in terms of their learning and want to discover more about that process." Equally however we are aware that some teachers do not feel comfortable using them.
Discussion among TTG members repeatedly included the observation that there seemed to be a link between the facility with which pupils moved the pieces of paper and quality of the eventual individual written outcome. Put another way, lower achieving sets were often slow and hesitant in finding and imposing any order in the data and would often need considerable support. Group members and other correspondents have, however, also pointed out such pupils usually show great perseverance and motivation in this endeavour.
It was from this grassroots generalisation that we began, with the assistance of our classroom colleagues, to speculate about the nature and significance of this apparent relationship. As we watched pupils sorting data physically on table tops it began to dawn on us that the manipulation process was a window on cognitive process and as such a potentially powerful diagnostic tool. Furthermore if physical manipulation reflects cognitive processes, was it possible that mysteries could be an intervention medium? Would it be possible to accelerate cognitive development by scaffolding pupils as they learn to handle the data items? Would they be able to internalise procedures for handling the data as mental models (Johnson-Laird, 1983, Halford, 1993). What do teachers have to do to make this a reality?
The Vygotskyan perspective
There has been an evident renaissance of interest in Vygotsky's work (e.g. Daniels, 1996), not least because of the success of CASE (Adey & Shayer, 1994). One of the alluring features of Vygotsky's work is the prospect that teaching can be an intervention in cognitive development. Whilst little direct evidence survives of Vygotsky's teaching methods, others have developed practical and/or theoretical manifestations. Thus we have mediated learning (Feuerstein 1980) scaffolding (Wood et al., 1976 ), assisted performance (Tharp & Gallimore, 1988) and reciprocal teaching (e.g. Brown & Campione, 1990). In each case there is a particular concern to integrate assessment with teaching, so that assessment adopts a diagnostic mode that will inform the teacher about the learning of her pupils and what might be the most appropriate next step.
Vygotsky's view was that instruction created the Zone of Proximal Development, stimulating development and crucially therefore:
From this point of view, instruction cannot be identified as development, but properly organised instruction will result in the child's intellectual development, will bring into being an entire series of such developmental processes, which were not possible without instruction. (1982, p.121)
In Hedegaard's (1996) interpretation initial activities must be oriented towards concrete exploration. Thereafter children must be able to symbolise the relations they perceive into a forms of words, which will allow them to extend their everyday concepts to include theoretical concepts. In Hedegaard's teaching concrete exploration includes exploratory analysis of objects, museum visits followed by the use of drawing and modeling of the initial findings as the step to formulating the relations they perceived. The modeled concept relation should be a tool for the child in daily life; it becomes a psychic tool. Mysteries maintain many of the features of such an approach.
Terms like the ZPD and scaffolding have begun to lose clarity, however, because they are so widely and in some senses carelessly used. Teachers can legitimately ask "How can I use the ZPD in my teaching today?" or "How can I scaffold the understanding of 9Y?" - this of course assumes that they are familiar with the terms! Unless the research community (which has to include teacher researchers) can provide some guidance we perhaps should be more careful in our use of the terms. The current research is an investigation into providing a theoretical base and practical guidance for humanities teachers for intervening in cognitive development.
Our aim in the classroom research has been to investigate how pupils do mysteries; it is therefore naturalistic. This has two aspects: what do they physically do with the pieces of paper and secondly what cognitive process underpin these physical actions? In relation to the first aspect we have been concerned to define the limits of performance by identifying how mysteries are tackled by both high and low achieving groups. A supplementary concern has been to observe the interactions of teachers with groups as a start to understanding how they might most effectively support pupils within the limitations of the classroom context.
In pursuit of our aims therefore we have:
It was planned that after three of the video sessions, interviews would be conducted with the target group, representing broadly upper, middle and lower ability. We were aiming to sketch out the range in strategies and processes employed across the ability range. The interview with the low ability group was effectively stillborn because this group had needed so much intervention that it was impossible to ask them about their strategies whilst sorting. In our original protocol it was planned that pupil-teacher interaction, after initial task instructions, should be kept to an absolute minimum whilst pupils were doing mysteries but this quickly broke down as the needs of classroom life intervened. There have been a number of important context variables in the research, notably the relationship between the teacher and the class and the type of mystery.
The Display Stage
Presented with an unsorted mixture of data, not all of which may be useful, the groups need to familiarise themselves with data items before any further meaningful action can proceed. Some groups elect to distribute the slips as in a card game to be read aloud in turn, before laying them out. Others just spread them out, neatly or otherwise. The purpose appears to be to allow group members to read the text or at least to register the presence of a slip that someone else has read aloud. Some groups do not successfully complete this stage unaided as they, for example, accumulate a pile of read slips without being able to infer any meaning.
The Setting Stage
Groups are inevitably unaware of the relative significance of any particular data item (it has to be remembered also that there is no one definitive right answer). Groups often therefore begin to organise the data into sets on the basis of perceived common characteristics, suggesting a general strategy based on association. Yet even at this early stage a variety of strategies are adopted which may reflect cognitive ability.
Low achieving groups frequently form sets on the basis of common vocabulary such as the names of characters, animals or places. Most groups assemble broad thematic sets, for example 'anything to do with earthquakes' or data items suggesting a chain of events. On the tables, these sets are usually arranged as columns and blocks. This stage indicates, we believe, a developing familiarity with the events and circumstances. A refinement is the creation of subsets, which may be triggered by one thematic set growing so large that the group begins to reclassify them . It is generally the more able groups who form sets based on the background data, although at this stage they may not appreciate its full significance.
As a generalisation it seems that the number of data items that each group can attach no meaning to is inversely related to ability. Nevertheless all groups form a reject pile. There appears to be a further contrast between less and more able groups in that the more able groups are more inclined to reconsider their reject pile, while some less able groups never look at them again.
The Sequencing and Webbing Stage
Some groups do not go beyond the setting stage, even though the sets that they have formed may be quite ineffective in terms of producing an answer to the mystery. However the majority of groups begin to identify relationships between sets or between single data items. In some instances this is in lines representing the construction of a causal explanation (sequencing), whilst in others it is a non-linear pattern representing multiple inter-relationships (webbing). Inference is common in the production of these patterns.
The Reworking Stage
This stage can be radical or modest and can take many forms. It may start with moving one slip from a set to another, but can go on to include reject slips being worked in or wholesale movement and regrouping. These reworkings appear to represent the establishment of new sets of relationships, which are increasingly abstract and likely to include the background data items. In the reworking process data items which are moved are cumulatively taking on new meaning. It is our impression, at this stage, that the more the data is rearranged, the better the quality of any subsequent written work. High ability groups show little reluctance in breaking their original sequences and webs, whilst other groups may be very frustrated by having to do so. Many groups do not rework.
The Abstract Stage
For a few groups, the physical manipulation ceases but the discussion continues. It is possible that they have internalised the data to a point where they can explore new relationships and hypotheses without recourse to the concrete format of the data slips.
Stimulated Recall Interviews
We include some extracts from these interviews to illustrate the stages that two groups have gone through. The first group are three Y9 girls from a middle set working on a mystery concerning stock losses from an organic farm.
Interviewer: So what are you doing here?
B: We're just sort of reading them - seeing what they were.
(The display stage)
Interviewer: What's that little grouping there?(The group are in the setting stage. However, before they finish this stage they start to make links that are characteristic of webbing)
C: .... It was the Leythorpe hunting things - linking those together ... but we read most of them first .... we tried to put them into little groups that linked together.
A: And the hedgerows and stuff
B: And we thought that the golfers on Massey's farm would be complaining (about the smell from the organic farm)(Thematic sets begin to appear, although the exact relevance is as yet unclear)
Interviewer: There's a new one beginning there ... What's happening there?( Now however they begin to make substantial progress as they begin to make connections characteristic of the sequencing and webbing stage)
B: That's the one where we didn't know which one ... but I think it was the herbicides and pesticides and that.
Teacher: Was that because you didn't know what they did?
B: Yeah and we were a bit lost.
B: But then we found the fact about the steep decline in birds and then we linked that together with the pesticides and herbicides and the fact that they ate little bugs ... that used to destroy the crops, then when the bugs were killed then the birds wouldn't have much to eat.(By establishing these relationships, they were linking sets in a logical causal sequence and at the same time weaving in some of the background data).
A: Plus the hedgerows were getting destroyed and their nests getting destroyed as well.
There is a susbstantial quality to the talk episodes, with examples of speculating and deductive reasoning:
Interviewer: This is clearly becoming a very important category. You are doing all the talking here.
B: I'm just saying that if the birds are being poisoned .... the chickens ... they would have found the bodies unless the fox had eaten them but then you'd have found feathers ....
However, at this point the majority of slips were still in the original three unhelpful sets and frustration had set in. The abandonment of the original sets is a turning point as they rework the data items and adopt a new strategy leading to a very different understanding:
A: We started to get stuck. We just like completely gave up for two minutes because it wasn't working, got bored then got back into it when we decided to split the groups up.This clearly illustrates that having gained sufficient familiarity with some of the themes in the data the group begin to organise sets which represent major concepts in the domain. They were able to discuss and explain the nature of the links in terms and phrases not found on the slips and demonstrate an impressive understanding.
C: We split them all up. We took one then looked to see which ones linked to that.
A: One of them was about the EMU, quotas and everything, we put them in a group, .... encouraging people to get rid of hedgerows and stop them growing so much.
The second interview involved two very able girls from a Y9 top set from another school doing the Kobe mystery. There are strong similarities in the pattern of the stages. However there are a number of points of difference worth noting. This pair never appeared to be as dependent on the manipulation of the data items for the development of their thinking, although they were aided by it. Secondly they were quicker to form thematic groups in the setting stage. Thirdly they were able to be much more explicit about metacognitive strategies that they employed in trying to impose order on the data (although this may be post hoc rationalisation), and in the process demonstrated impressive evidence of transfer of learning from other contexts.
Interviewer: You have one at the top there about Japan being a rich country?It is interesting to note that work in the US has shown that more able pupils required less help to transfer rules and principles to novel problems. In addition their relative advantage increased as the transfer distance increased (Brown et al, 1992.)
B: It was like a background (the interviewer and the teacher had not used this word). It was not an order, it was background, Japan is a rich country and the plates stuff, it's not in any order.
Interviewer: You are forming groups?
B: I thought that they should end up in a line like a storyboard, but they didn't all go (storyboards had not been mentioned).
Interviewer: Where have you used storyboards?
A & B: In English.
Interviewer: What didn't fit?
B: The backgrounds.
A: There were the buildings and things that contributed to her death, but not directly.
Interviewer: Have you done background before?
A (looking at B): Once or twice in Y8. We did not realise that we were doing it. New things kept cropping up and things changed so it changed. We were looking at evidence and sorting and resorting.
Interviewer: Have you looked at evidence before.
A & B: In history. In history we do sources, which sources are reliable and which are unreliable.
What begins to emerge is a pattern that bears close resemblance to the SOLO taxonomy (Biggs & Collis, 1982). In analysing the work of hundreds of students of different ages in a variety of subjects Biggs and Collis detected a recurring pattern that they described in 5 levels - Prestructural, Unistructural, Multistructural, Relational and Extended Abstract. The SOLO taxonomy has Piagetian origins and each level is linked to a Piagetian stage.
Prestructural responses do not address the question or task set. Answers at best may just restate the question. Those groups, who if left unaided, can make no sense of the data in relation to the question could be considered to be showing a prestructural response.
Unistructural responses use one piece of relevant data in a descriptive mode without a conclusion related to the data. Multistructural responses use two or more pieces of data, but without linking them. These responses can be matched to the display stage.
Relational responses are qualitatively different as the student links the data together in coherent manner and reaches a conclusion consistent with the data. There is a correspondence with the webbing and sequencing stage.
Extended abstract responses are characterised by the inclusion of data not given in the task, abstract concepts, a level of generalisation and the consideration of a number of competing hypotheses. This bears a resemblance to the reworking and abstract stages.
These findings are of profound significance. Firstly we suggest that when pupils tackle new problems they do not automatically switch into high order thinking, using formal operations in Piagetian terms (or analysing, synthesising and evaluating in Bloom's terms) they start with comprehending or making sense of individual data items. It is only from this foundation that they proceed to relational or extended abstract thinking. Some pupils falter early on in this sequence. Secondly these stages are evident in the physical manipulation of the data and the associated talk. This allows the possibility that teachers both identify the level of thinking being used and can intervene when they judge that pupils cannot proceed unaided.
We are immensely excited and encouraged by our initial experience:
Wood & Wood (1996), in seeking to clarify the meaning of scaffolding, have suggested that adults can serve a number of crucial tutoring functions in guidance and collaboration that promotes development ... "These included recruitment of the child's interest in the task, establishing and maintaining an orientation towards task-relevant goals, highlighting critical features of the task that the child might overlook, demonstrating how to achieve goals and helping to control frustration." (p.5). We have already watched teachers interacting with pupils as they engage with mysteries. Detailed analysis aimed to make these interactions more instructionally effective lies in the future but we can offer the following observations:
It needs to be repeated that these possible scaffolding moves are only tenable because of the physical manipulation of single data items, an advantage not available in block text. However we eschew certainty in relation to these possible scaffolding moves as much further work needs to be done to achieve the principles of 'contingent teaching', that is to "give more help when the learner gets into difficulty, and offer less help as they gain in proficiency" (Wood & Wood, 1996, p.7). Such principles are important not only because of the need to relinquish responsibility and control to pupils but also because of the demands of class teaching. This skill in the deployment of time and attention is acute because it is easy for teachers to be destructive in their visits to groups. Furthermore Bliss et al. (1996) have pointed to the difficulty of KS2 teachers have in implementing scaffolding strategies.
It is important within this period of review, analysis and speculation in relation to the NC leading to possible major revisionsin 2000, that greater attention is paid to teacher assessment. It is to some extent teachers' weaknesses in the sphere of diagnostic assessment that has allowed the summative mode in national testing, driven by accountability, to dominate. This is dangerous not only because of the evidence that suggests "teaching to the tests" is narrowing the curriculum (see for example Radnor et al, 1995) and reducing the validity of the tests in the process, but also because of the threat to the professionalism of teachers. Diagnostic assessment has to be one cornerstone of the knowledge base that delineates the teaching profession.
If teacher assessment is to demonstrate its importance it must develop a distinctive function that overshadows simple measures of accountability and it must be accepted as valid. We note that validity is a contested concept, but we find much encouragement in Wiliam's (1996) analysis based on Messick's (1980) proposals. Whilst the Kobe mystery may not be strong in supporting within-domain inferences, it should be more highly rated for within-domain consequences and is an exciting prospect in terms of both beyond-domain inferences and beyond-domain consequences. Wiliam proposes that 'a test is valid to the extent that one would be happy for teachers to teach towards the test' (1992, p.17). Using this yardstick, mysteries represent a profile of validity that more traditional tests lack. Guided assessment is a term used to describe assessment used to test a child's learning potential. This can be normative, seeking to identify pupils for referral, but it can also be more strongly diagnostic, through trying to find the approach that will help each such reach specified targets. Therefore developing assessment technology in the profession is crucial. One of the challenges of teachers professionalism is bringing assessment and teaching closer together, in words of Brown et al (1992, p.194) "Ideally children are not labelled and categorised by assessment they are diagnosed and helped."
David Leat, Department of Education, Newcastle University,
Tel: 0191 222 6578, E-mail: D.J.K.Leat@ncl.ac.uk
Adam Nichols, School of Education, Durham University,
Tel: 0191 374 7821, E-mail: Adam.Nichols@durham.ac.uk
Adey, P. & Shayer, M. (1994) Really Raising Standards, London, Routledge.
Biggs, J. & Collis, K. (1982) Evaluating the Quality of Learning : the Structure of Learning Outcomes, New York: Academic Press.
Bliss, J., Askew, M. & Macrae, S. (1996) Effective Teaching and Learning: scaffolding revisited, Oxford Review of Education, Vol. 22, pp. 37-61.
Brown, A.L. & Campione, J.C. (1990) Communities of learning and thinking, or a context by any other name, in D. Kuhn (Ed.) Developmental Perspectives on Teaching and Learning Thinking Skills, Contributions to Human Development Series, Basle, Karger.
Brown, A.L., Campione, J.C., Webber, L.S. & McGilly K. (1992) Interactive Learning Environments: A New Look at Assessment ans Instruction, in B.R. Gifford & M.C. O'Connor (eds) Alternative Views of Aptitude, Achievement and Instruction, Boston, Kluwer Academic Publishers.
Daniels, H. (Ed.) (1996) An Introduction to Vygotsky, Cambridge, Cambridge University Press.
Davies, P. A Report on Schools' Use of Teacher Assessment Guidance for Geography in Key Stage 3, unpublished report to SCAA.
Feuerstein, R. (1980) Instrumental Enrichment, Baltimore, University Park Press.
Johnson-Laird, P.N. (1983) Mental Models, Cambridge, Cambridge University Press.
Halford, G.S. (1993) Children's Understanding: the development of mental models, Hillsdale, New Jersey, Lawrence Erlbaum.
Hedegaard, M. (1996) The ZPD as a basis for instruction in H. Daniels (ed) An Introduction to Vygotsky, London, Routledge.
Leat, D. (1997) Getting Ambiguous, Educating Able Children, Vol. 1, pp. 17-25.
Messick, S. (1980) Test validity and the ethics of assessment, American Psychologist, Vol. 35, pp. 1012-1027.
Northumberland County Council (1997) Northumberland "Thiking Skills" in the Humanities Project, Morpeth: Northumberland County Council.
Radnor, H. A., Poulson, L. & Turner-Bisset, R. (1995) Assessment and teacher professionalism, The Curriculum Journal, Vol. 6, pp. 325-342.
SCAA, (1996) Key Stage 3 Optional Tests and Tasks: Geography Unit 2, London: SCAA.
Tharp, R.G. & Gallimore, R. (1988) Rousing Minds to Life, Cambridge, Cambridge University Press.
Vygotsky, L. (1978) Mind in Society, The Development of Higher Psychological Processes, M. Cole, V. John-Steiner, S. Scribner & E. Souberman (Eds), Cambridge, Cambridge University Press.
Wiliam, D. (1992) Some technical issues in assessment: a user's guide, British Journal of Curriculum and Assessment, Vol. 2, pp. 11-20.
Wiliam, D. (1996) National Curriculum Assessments and Programmes of Study: validity and impact, British Educational Research Journal, Vol. 22, pp. 129-141.
Wood, D., Bruner, J.S. & Ross, G. (1976) The role of tutoring in problem solving, Journal of Child Psychology and Psychiatry, Vol. 17, pp. 89-100.
Wood, D. & Wood, H. (1996) Vygotsky, Tutoring and Learning, Oxford Review of Education, Vol. 22, pp. 5-16.
This document was added to the Education-line database 19 November 1997