Examination and learning
An activity-theoretical analysis of the relationship between assessment and learning
Centre for Staff and Learning Development, Oslo University College
Paper presented at the Learning Communities and Assessment Cultures Conference organised by the EARLI Special Interest Group on Assessment and Evaluation, University of Northumbria, 28-30 August 2002
Improving student learning implies improving the assessment system. Teachers might assume that it is their teaching that directs the students' learning, but students orient themselves as learners in relation to what will be assessed. In spite of the underlying idea that assessment should serve to test what students have learned during a course of studying, it tends to define what is worth learning. This view, which basically says that assessment drives learning, and that assessment "overrules" teaching, has been advocated by many researchers in the field of learning and assessment over a longer period of time (e.g. Elton and Laurillard, 1979; Crooks, 1988: Biggs, 1996; Shepard, 2000). This can be seen as a paradox and the metaphor often used is that of the tail wagging the dog.)
The fact that students attune their learning practice according to the demands of examinations might well be trivial. It is a way of coping with the educational system. The significance of this "backwash effect" of assessment is widely acknowledged. But the impact of assessment on how teachers teach might be even more significant than its impact on how students learn, particularly if we take into regard the impact of teaching on student learning (Trigwell and Prosser, 1999?). In this paper the core issue is the impact of assessment on learning. But I will try to widen the scope of the discourse from an assessment-learning relationship to an assessment-education-learning relationship. In stead of setting up assessment and learning as the only variables, I will also look for how assessment pushes other aspects of the educational process; particularly teaching, production of textbooks and learning material, and, more generally, the design of the learning environment. Consequently, the unit of analysis expands from the level of individual learning to the level of educational programmes and the institutional structures under which students learn, teachers teach and educational administrators work. At this level of analysis assessment is an integral part of the process of education. We are not dealing with a one-directional impact from assessment to learning and teaching, but rather an interrelatedness between assessment and the various aspects of education. This is a more complex phenomenon than the question of how individual students orient themselves as learners and requires a theoretical basis for analysis that can account for systemic complexity and dynamics. In this paper I also hope to illustrate how socio-cultural (or cultural historical) activity theory can serve as a tool for coming to terms with the interrelatedness between assessment and learning.
I shall focus on these questions by discussing the relationship between assessment and the educational process in a specific educational context in Norwegian higher education. The analysis is based on an empirical study and with students as the main informants. My starting point was an interest in understanding student learning from the perspective of students. The empirical material is from Exam Philosophicum (ExPhil) at the University of Oslo (Havnes, 1996 and 1997)1. The ExPhil course was a one-semester programme and was meant to serve as an introduction to scientific reasoning and studying at the university. My main argument is that the assessment structure contributed to establish a learning context that worked contrary to the declared content of the study. These observations will be accounted for and discussed on the background of activity theory.
The dominant patterns of educational practices differ from one country to another, from one institution to another, and even from one educational programme to another. In this paper I address one specific programme an one Norwegian university and do not claim that this analysis accounts for higher education in general or in Norway. One the other hand, it is relevant to talk about "the Norwegian tradition" and even of some general characteristics of higher education. It is important to approach the analysis at the level of the singular, but look for general patterns of a wider socio-cultural tradition. OECD has criticised Norwegian higher education for investing a disproportionate amount of resources in an assessment system that is not productive with respect to learning. Universities are characterised as "research institutions conducting examinations". The strong emphasis that has been on summative assessment will come through in this analysis. Since the time this study was conducted assessment has become an issue in the public debate in Norway and there are now strong incentives from national authorities to emphasise learning, feedback and formative assessment. But the rationale underlying "the old tradition" is still strong and the danger of changes implemented being "old wine on new bottles" is absolutely there.
I will not discuss the impact of traditions extensively here, just point at three aspects. (1) There are mechanisms inherent in educational programmes that sustain a specific examination system and corresponding assessment practices (formative as well as summative) over long periods of time. These mechanisms can be seen as artefacts that represent condensed versions of the experience of generations of scholars in a particular field. Examples of such can be national, profession-specific or institutional regulations, practical arrangements for examinations, feedback and supervision, and other arrangements that sustain specific institutionally situated practices. (2) These artefacts represent stumbling blocks for innovation and thus secure continuation and stability. (3) The application of such artefacts restricts our thinking by implicitly defining something as obvious and mandatory, excluding alternatives that could be taken into considerations (Lauvaas et al, 2001). In the analysis I will account for how a wide set of artefacts closely linked to the students' learning practice (e.g. textbooks, information material, exercises, previous exam questions, organisational structures and feedback structures) formed the educational practice of the Exphil course. Changing educational practice is not only changing how we think about teaching, learning and assessment, but also - and even more - about changing the artefacts that we rely upon in our daily practices as teachers and students.
The ExPhil study is an ethnographic case study of 7 students, who were followed throughout a complete term. In addition to interviewing them several times during the semester - and after the exam - I also joined in on lectures and participated in seminars and colloquium meetings. My role was that of an active observer attempting to take part as authentically as possible in the activity of the students, reading the literature and working on the same study tasks as the students. I systematically made field notes about what took place in concrete teaching and learning situations. I also collected written material from and about the course of study. The project is thus based on data from various sources: written material describing the course of study; the literature; the sets of exercises that the students worked on, and examination tasks. My own experience as a participant observer is yet another set of data. The major part of the data is the students' stories about their learning as they were told to me in interviews and conversation.
Data from interviews and observations have been coded to identify patterns in the students' descriptions of their learning, the content of teaching and learning, the learning situation, and the students' participation in the educational programme. The actions and utterances of individuals have been analysed with respect to meaning expressed about the educational programme. The analysis has focused on patterns of participation and the interrelatedness between these patterns and contextual conditions for learning. Thus it is not the participation of individual students that has been the focus of interest, but modes of participation as social practice in a particular context. Individual actions have been analysed as a forms of individualisation and particularisation of collective practice.
The epistemological basis for putting the collective level in the foreground is associated with situated learning and socio-cultural activity theory (Vygotsky, 1978; Engeström, 1987; Lave and Wenger, 1991; Cole, 1996; Engeström and Miettinen, 1999; Säljö, 2000). Engeström and Miettinen, for instance, argue that collective and historical aspects of the cultural context are integral to the actions conducted by individual agents. To understand learning (and teaching) practices we have to go beyond the individuals and look at the cultural systems and mechanisms that sustain some action patterns rather than others.
"Individuals act in collective practices, communities, and institutions. Such collective practices are not reducible to sums of individual action; they require theoretical conceptualization in their own right. When individual action is the privileged unit of analysis, collective practice can only be added on as a more or less external envelope" (ibid., p. 11).
Educational programmes are part of cultural systems and what goes on there is not reducible to a sum of individual actions. Cultural systems have "...cyclic rhythms and long historical half-lives" (Engeström et al, 1995). As we shall see, collective practices can be heterogeneous and contradictory. The tensions in the context of learning are not external to or separate from the learning. The study of learning from the perspective of activity theory implies searching for tensions in the learning context and mechanisms by which these tensions make their way into the process of learning as a social phenomenon.
This approach bears similarities with the hidden curriculum approach (Jackson 1968). A distinction is made between the intended, the taught, and the learned curriculum (Cuban, 1992). The so-called hidden curriculum is identified by going beyond the intended curriculum and taking the content of teaching and learning as the primary basis for understanding the educational process. The hidden curriculum is identified as something different from the intended curriculum. From the perspective of activity theory the complexity that emerges from the wide scope of practices (and corresponding curricula) involved is problematised. The focus is not so much on different categories of educational practices and phenomena, but rather on different levels of functioning that data and observations can be analysed on. The smallest possible unit of analysis is the activity system, not specific categories of practices within it.
Unit of analysis
In the socio-cultural or cultural-historical tradition, learning is viewed as social practice situated in a specific historical and socio-cultural context. Rather than emphasising individual differences, the focus is on mechanisms that make people act in similar ways within a given context. The emphasis is thus on action patterns typical of specific social contexts, that is, ways of acting that tend to be consistent across individuals and over time. The core location of the particular practice is in an organisation, a context or a social structure, rather than in individuals. It is common to raise one's hand before speaking in a class setting or at a meeting, but rarely in a pub or another informal situation - not because the people are different, but the context is. We act contextually meaningfully (or not). The unit of analysis, then, is the person-in-the-situation, not the person as a separate entity.
Becoming a student involves adapting to, and to a greater or lesser degree accepting, certain institutional demands. Such demands are rooted in historical practice and institutional constraints and affordances, regardless of the individual person's presence or absence. They are not to be seen as the student's assumptions about the course of study but as affordances of the ambient environment (Gibson, 1979)2. In order to understand how students tackle their learning situation, I intend to examine the institutional influences inherent in the two educational programmes. It implies investigating student learning by turning the attention in the same direction as the students themselves have their primary focus; towards the surrounding environment, what it affords and demands.
Learning is also an individual action and as such is influenced by the person's background, experience and habits. In social situations a state of tension might arise between the person's individual skills, interests and wishes, on the one hand, and institutional demands, on the other. In this field of tension a dynamic in social practice is created. The situation is both given and at the same time created - or interpreted - by the agent. Also, there are opportunities to challenge the system. We are more or less able to help determine the social context in which we are involved. The extent to which we help create our own situation is both socially bound and dependent upon how we approach it. This line of reflection puts the notion of participation in the foreground, which means that the scope of our intention simultaneously goes in two directions; toward the context and toward the participants. Neither can be understood independently. This is a fundamental ideological basis in activity theory.
Levels of human practice
In activity theory (Vygotsky 1978, Engeström 1987) one differentiates between operation, action and activity. The simplest level of understanding of human behaviour is that of operations; routine activities which we carry out without any conscious intention. At some stage they might have been conscious, but not any more. An action, on the other hand, is individual, intentional and consciously performed in a manner unlike an automatic operation. An activity, however, is more abstract and more difficult to come to grips with. It reaches beyond the individual and is part of a wider, social context. It cannot be accounted for by referring to what individuals do or think. Relevant examples can be found in public services and functions.
Kindergarten activity, for example, is more than the actions of those working there. What I do is part of an activity, but not on the strength of my doing it. At the individual level it is an action. The cashier checking out purchases in a supermarket carries out a number of individual actions - and probably a series of habitual operations - but collectively the personnel are engaged in the activity of trade of everyday commodities. In order to understand the nature of this sales activity, we must look at the whole co-ordinated picture of a complex set of actions, not merely the actions or tasks carried out by the individual.
It is this activity concept that I intend to examine more closely in my endeavour to analyse the relationship between assessment and learning. At this activity level the focus is on relatively stable social practice which are upheld by something more than, or different from, the intentions, habits and preferences of those individuals involved. The question is what mechanisms sustain a social practice within a specific social and cultural setting.
The sustainability is not rooted in the intentions of the individuals. In activity theory one refers to a collective motive which, since it does not arise out of the individuals, must have its source elsewhere. The motive is first and foremost rooted in the object of the activity, in what the activity is directed toward. In the context of a discussion of assessment, it can be questioned if the object of the education practice is learning for future professional practice or the passing of exams.
Changing an educational practice e.g. moving lifelong learning in the foreground, rather than the preparing for exams is not basically a matter of redefining the object at the discursive level. The continuity of an activity system is also sustained through a series of mediating artefacts, i.e. tools, instruments, rules or other factors that afford certain ways of acting in a given situation. These artefacts are often rooted in institutional conditions and they afford some practice and constrain some other. For instance, the way auditoriums are constructed indirectly "invite" a certain type of teacher and student behaviour.
Tools can be material or psychological. For instance, when telling ourselves how to act, we base our decision on conceptions or models of what will work. Language is an example of a psychological tool, as are also models of what functions best and the techniques of memory. Paper and pencil are examples of material artefacts. We make notes in order to record what is being said. Some may use laptop computers or tape-recorders at lectures or meetings, while others mark their textbooks according to a personal colouring system. The introduction of new technology - that is; artefacts - in a company might change dramatically the action and interaction of the workers.
The concept of institutionally mediated practice is very wide and has many component parts. Engeström (1987) used the following model to illustrate the components of an activity system.
Figure 1. Based on Engeström's (1987) model of an activity system
I shall use this model in order to illustrate central aspects of the two educational programmes in question. The fundamental idea in this model is that
1. subjects with competence, preferences and goals act in relation to
2. an object which is partly socially given and partly constructed in relation to the subject
3. by using tools (artefacts) which afford some actions and constrain others and
4. agents act as participants in a larger social system.
As participants we are engaged in this social system (community) where certain rules apply and were the action and interaction between participants are regulated to some extent (division of labour). The upper part of the triangle is thus just the "top of the iceberg". Actions are anchored in social conditions and requirements that are not so clearly visible. The model depicts what we mean when we say that actions are culturally, institutionally and historically situated and opens up paths for empirical research based on a contextual approach to learning.
One consequence of this model is that strategies to improve study practice will have little effect, if aimed solely at the students, their conceptions and their ways of addressing learning tasks. Such a strategy assumes that the students' volition and control over their own actions constitute the primary mechanisms for mastery a given situation. But by focusing on learning strategies we deal only with the "top of the iceberg". A social approach to learning expands the attention away from individual persons and towards the social system and the surrounding institutional practice. That is where we find the most important factors contributing to upholding social patterns of behaviour.
Assessment procedures and examinations are institutionally created mechanisms that serve to shape the learning process among students. But only one set of mechanisms out of many. Other influences can be found e.g. in the forms of teaching and in different types of material, as well as the physical environment. Ways of teaching can, for example, either encourage rote learning or foster an understanding of the underlying premises. The physical environment can "invite" students to spend time at the institution and interact with other students or teachers. The opposite effect is also possible. Educational institutions are complex environments and one can expect to find influences pulling in a variety of directions.
Exam philosophicum - a counter-productive assessment system?
Objective and history
Exam Philosophicum was a preparatory course that all students at the four Norwegian universities had to pass, but is not a part of the programmes in the polytechnic colleges. The course included the history of philosophy, logic and argumentation theory, and philosophy of science. Since Norway had its first university in 1813 students had to pass a preparatory course as part of their university education. Over the years it had been revised several times. The course in its particular form at the time of the project had existed for about 10 years when this study was done. After this study was conducted changes has been made and at the time of writing some organisational structures have changed. But the main aspects that are emphasised in this analysis are basically the same.
The background for establishing a preparatory course for the academic studies was - and is - that students have not achieved adequate academic competence for such studies through primary and secondary education. The programme has always been labelled as a preparatory exam, but the content has changed. In the very beginning it was a two years programme and included more than philosophy, e.g. Latin and mathematics. These courses had to be completed before you could enter the disciplinary study. Over the years there has been different requirements for different academic studies. E.g. phonetics and theory of language has been requirements for studies in the Humanities, and mathematics for studying natural sciences. In other words, Norwegian universities have had the system of preparatory courses from the very beginning. The historical roots of the Exphil course are the philosophy part of this set of preparatory exams. In the process of history these requirements have become part of the different academic programmes and training in the Classical languages not regarded as necessary. Training in the history of philosophy and science and logic of scientific reasoning still exists through the Exphil course.
The "father" of the "modern" Exphil was the philosopher Arne Næss. His doctoral thesis from 1939 Interpretation and Preciseness has been an essential basis for the Exphil programme that has existed in the post second world war period. Preciseness in interpretation of verbal utterances and observations was still, at the time of the study, the core dimension of the course content. Knowledge about the history of philosophy was regarded at a key competence and a necessity for understanding present approaches in a historical and critical perspective. At the time of this study the course content had been basically unchanged for about ten years. In the beginning of the eighties there had been a major reorganisation by taking away optional courses within Exphil and making it more into a general introduction to historical and systematic dimensions of science.
In short version it could be said that through Exphil neophyte students would achieve the basic scientific competence that was needed to become an independent, critical student of science. Through learning the general basics of science one should be prepared for university studies. You should learn the basics that underlie science in general, before you go into the specific disciplinary knowledge. This preparatory aspect of Exphil is essential for understanding what takes place there. On the other hand, the course was open for students who were not going to study at the university and quite a lot of the students had no intention of university studies. Some students took the examination in order to earn credit points towards a place at a college of higher education, while others simply had no better alternative. Accordingly, teachers argued that the course had relevance beyond preparing for academic studies; it served as a platform for developing a critical and independent approach to everyday life situations, political and ethical discussion etc.
The course of study lasted for thirteen weeks, which is standard length for the teaching part of a term in Norwegian universities. Teaching was by lectures or seminars. Because of the huge number of students involved, the lectures assumed mammoth proportions with several hundreds of students attending.
Seminars were conducted for smaller groups of 20 to 60 students. The students were often organised into smaller groups (colloquia) in order to work on assignments in between seminars. In many cases working in these groups provided the students with their most important social relationships at the university.
The course was terminated by a 5-hour examination during which the students were asked to answer one question from each of the three subject areas. Every term a collection was published of all the examination questions from the previous ten years. The questions conformed by and large much to the same pattern of questioning. In the history of philosophy students were expected to write about philosophers and to compare one with another in relation so some specific issue. In logic and argumentation and in theory of science the students were given prototypical cases and asked to identify and discuss theoretical problems related to these cases.
All the students that I followed started with enthusiasm, apart from one female student who had taken the course before and failed. She was disillusioned from the start, as were the others gradually after a while. All the students' commitment to work increased again just before the examination.
Common to all the students was a trust in the importance of starting at university, not only from an academic point of view but also for what they deemed the personal value of studying. The range of expectations, however, was very wide, varying according to the way in which studying at university played a role in the future pattern of their lives. One student was a previous drug addict and for him university represented the start of a new life free of drugs. Another had suffered from eating disorders and felt unhappy in the local environment she came from. A third student was an immigrant from India who had completed Norwegian secondary school education and was now looking forward to studying his great interest, political theory. A forth student looked forward to study literature, which was her main interest. Another student looked forward to become a medical doctor and yet another to become a lawyer.
In spite of the different role university studies would have in their individual life projects, the one common feature they shares was the expectation of a more interesting course of learning than they had experienced at secondary school level. The general feeling was that at secondary school they had managed well by doing what the teacher had asked of them. University would surely offer something different, they expected. In what way it was going to be different was not clear for them.
Exphil unveils itself
The students received a welcome that suggested that more would be demanded of them than had been the case at school. University, they were told, represented search for knowledge rather than homework and tests. These new students were made welcome to a world where the emphasis was to be on academic freedom and the joy of learning. They would be confronted by a new way of thinking. The ExPhil course would introduce them to the principles of scientific reflection and enable them to apply these principles to research and to experiences from their own lives. The course was to be an introduction to free intellectual thought. There was also a "learning-to-learn" aspect. One teacher argued, "Gradually as the students become familiar with a critical, reflective and scientific philosophy, they will also learn the university way of learning." Stigen (1990, page 71) expresses the idea of the course when he emphasises that students learn show how previous generations oriented their lives. They must, however, each make up their own minds, as individuals. One teacher on the ExPhil course explained, "My aim is to make the students understand that what we take for granted today has been the subject of discussion throughout the centuries. I hope to make students look with a critical eye at what they so far have taken for granted."
The course began and much of the avowed intentions proved relevant to the students. One student, for example, who had appeared critical and analytical before the start of the course, expressed himself in the following way.
"This is a new way of thinking. It is challenging to tackle this material. I am familiar with a great deal but much of what lies behind it is new to me. You have to understand the structure in order to make use of it."
The students experienced a challenge to think in new ways. There was, however, another side to their studies. Looking back on the course, another student explained:
"It is often said of the ExPhil course that it provides an introduction to what it is like to be a university student, that it offers a kind of feedback as to whether you should go on to study at university or not. Since I am also already studying literature, I am aware that the real university is quite different from the ExPhil course. For me there is a somewhat false atmosphere about ExPhil. We pretend to be university students but in fact ExPhil is really somewhere between secondary school and university. Sometimes I have the feeling that we are simply being led through the course."
During the course the students expressed a growing frustration. A general theme was that they had hoped to learn more than just for the purpose of passing an examination. One student, expressing a preference for applying critical thought to his reading, made the following admission. He was frustrated about the history of philosophy:
"In the end I realised that I only needed to study the review book. So I did.I gave up reading thick books. The important thing was to learn as much as possible by heart."
Another student felt that the pace of the course was too fast:
"This is what it's like. There's no time to rest. We just have to cram it all into our heads. Take the history of philosophy, for example. It could be so interesting, if we only had more time."
"In a way, ExPhil is just nonsense, although it can be interesting once you get into it," said one of the students. This was the sharpest comment I heard and one that indicated the students' ambivalent feelings towards their studies. But the proportion of the dilemma goes beyond the feeling of the students. These tensions are embedded in the educational programme in variable and complex ways. They can be examined by turning our attention away from the intentions of the course and the students' experiences of what is often called "the learned curriculum". I will address the tensions by drawing the attention towards the teaching, learning and assessment practice. I here use an extended notion of teaching, which includes the learning material used and rationales that seems to lie under the production of such material. The first issue to address is the content of teaching and learning and the expected learning outcomes.
Course content and teaching
The problems to be dealt with in the ExPhil course were complex and fundamental and much of the material was difficult. Some simplification was clearly inevitable.
The students were confronted by varying philosophical beliefs through the ages, the nature of logical thought, different methodological perspectives, etc. The overriding aim was to make the students more aware of fundamental, scientific problems and thereby heighten their critical senses. At the same time, however, they were presented with specific, detailed learning outcomes, defining what they should study, and also with a number of tasks to be solved. The history of philosophy, for example, was divided into twenty or more guideline questions. At the time of writing, question 5 is as follows:
"Describe Aristotle's views on how perception and thought combine to create knowledge. Give an account of his teaching on substance/things, the four causes, the four types of change and his overall concept of nature (cosmology)."3
Learning goals in other parts of the course were specified by a series of key words. For example, in part 2b, philosophy of science, these goals are presented as follows:
Hypotheses. Theories. Models. Formation of hypotheses and theories. Knowledge and truth. Axiomatic methods. Hypothetical-deductive method: Reasoning from deduction of empirical consequences. Predictions. Types of hypotheses: Universal, static, singular. Causality hypothesis. Scientific laws.4
Teaching and seminars were to a great extent structured in accordance with these learning goals. The students received no reading lists or homework, but they were given assignments to work on and these provided the basis for the course. The seminars in particular were centred around these assignments. At first they were limited and fairly simple, but soon the students were required to answer parts of examination questions and later complete examination papers which had been set in previous years.
Lectures, on the other hand, were of a different nature. Those I attended were characterised by interpretation and reflections on the views of philosophers and on theoretical problems. However, it was interesting to sit in the lecture hall and to register the reaction among the students when the lecturer announced that what he was about to say could be important for the examination.
Teaching and learning practices: Examination up front
Not many days passed before the students started to discuss what they would have to write about at the examination. Examination anxiety came to the surface early on in the course. They were told that a 35% failure rate was normal. The students I followed throughout the course came to regard answering examination questions as a central part of their studies and seemed to agree that instead of reading the literature it was sufficient to concentrate on those main philosophers that the exam questions mostly focused on.
Several of the teachers emphasised the importance of taking part in the seminars, participating in colloquia and work on problem-solving. Students who did so rarely failed the examination, they said.
Parts of the course literature were also examination-oriented in a specific way. There was a tendency for previous examination papers to define the content of the textbooks. Some seemed to be written with these exercise books and examination questions as the starting point. This applied particularly to the books most often used in logic and philosophy of science (e.g. Baune 1991, 1993a). Other books had a wider perspective, using examples from daily life and from science without drawing those close parallels with exercises and examination demands (e.g. Follesdal et.al., 1985; Wormnæs, 1987). In the former type of literature the examples given had the character of prototype academic problems, whereas in the latter they tended to illustrate the existence of a dilemma and were used to provide a basis for reflection. The former type of material bore witness to a pedagogic rationale, the purpose being to help students with problem solving. In Baune (1991) this was achieved by setting questions to be answered alongside the presentation of the theory. Typically, there were references to where in the literature it was possible to find the answers to specific tasks. The (review and exercise-oriented) supplementary readings written by the textbook author strengthened this impression (e.g. Baune 1993b). Although without doubt this was a help to the students in dealing with this specific kind of problem-solving activity, it is difficult to see how this could promote critical reflection.5
These data illustrate that examination influenced the course of study in many ways; in the literature, in the teaching and in the students' learning practice. The focusing on how assessment drives learning is a too restricted approach to the problem. We have to take the educational programme as a whole into account and it is hard to tell which component is prior to the others. It seems that the way teachers teacher teach, students learn, exams are constructed, assignments is developed, feedback is given, and textbooks are written are all interrelated. The practice related in producing these various aspects of the educational programme seem to mutually define each other and merges into what can be framed the activity of education.
However, as we have seen, the course was contradictory. For instance, the students experienced a contrast between the stated "ideals" of the course and what was asked of them in actual everyday practice. There was a general pattern that students encountered a learning environment with affordances pulling their learning in different directions. The most important of these was not what was said about the course of study but the activity of which they became a part, it turned out to be a programme of continuous preparation for an examination. The examination gradually became the final objective of their studies, a situation described by many of the students as "meaningless". Nevertheless, this was their experience of the overriding demands of the course.
This "meaninglessness", however, was not absolute, as can be seen from such remarks as "It's exciting once you get into it". One student emphasised the difference between the prospects inherent in the study material and the direction taken in the actual study situation. Reframing one student's view it turned out like this: Whereas the latter aimed at "filling your head with as much as possible", the material also invited to "reflection on research and life in general, putting to the test one's own epistemological assumptions and all that one takes for granted".
Another student experienced the study material as relevant when she took part in discussions with other more advanced students in the cafeteria. When these more advanced students were engaged in serious discussion she could hear them use expressions that she was struggling with on the ExPhil course. She would think: "Wow, perhaps I've been learning something that may prove useful later." In these situations, however, she was outside the framework of the ExPhil course. Inside the course the main objective was the examination and the demands were for the students to tackle typical examination questions according to set patterns that they practised throughout the course.
Teachers also expressed dilemmas. The complexity of scientific research was considerably simplified. Principles in philosophy of science, scientific logic, and western philosophy in general were illustrated through a set of cases from the modern history of science and constructed study tasks. The danger of trivialisation was obvious. The learning of what can be called the generics of science underlying the variety of disciplinary research within a university context was challenged by the lack of disciplinary knowledge among the students. The high failure rate made it necessary to establish some realism in expectations of what the students could deal with conceptually.
My starting point in this study has been that learning is embedded in the social practice in which the students participate. We learn to participate in concrete social situations on the basis of what they demand, what they afford, and the significance we attribute to them. Learning implies a change in an aspect of our participation in the available social practice, although not independent of our own contribution. Learning is relational. It is relational to assessment, but assessment, again, is relational to other components of the complex system of educational programmes.
From an activity-theoretical perspective, the main emphasis is on identifying the object of the activity on an institutional or systemic level. In this case that would be, for instance, what the teachers' teaching and the students' learning is directed towards. On the one hand, the object is a given feature within a social and institutional context. On the other hand, the participants interpret or project meaning into the situation and thereby contribute to creating the object (Leont'ev, 1978). Something in the outer world comes into contact with a person's needs, wishes or assumptions. In such a meeting the person concerned selects certain aspects of a situation or an activity as essential, both on the basis of personal interests, qualifications and experience, as well as against the background of what appears as essential in the actual situation. The object of the activity arising from this meeting serves to create the motive for how the person acts. The motive lies thus in the situation, not in the person, although it arises from the meeting between the person and the situation. The nature of the learning process in the educational programme discussed here is, therefore, only to a limited degree the subject of the individual's personal choice.
The data from this study illustrate another crucial aspect of learning. The students' meaning making, or learning, cannot be understood only on the basis of an analysis of the students' interpretation of the course content. The teachers' meaning making of the course content, as it evolves through teaching, the production of textbooks and other learning material, and construction of contexts for assessment is an integrated part of an institutional meaning making process. The meaning making process of teachers can be seen as a driving force for the meaning making process of the students. How and what students learn, and how and what teachers teach, can then be seen as interrelated meaning making of certain phenomena in the world, that is, the construction of the object of teaching and learning.
In activity theory an important distinction is made between individual, purposeful actions and longer-lasting object-oriented activity (Engeström, 2000, p. 946). Individual actions are subordinate to activities and to be understood in the light of the whole activity system. The actions of students in an educational context must first and foremost be understood in relation to what the course of study demands or offers.
In an activity-theoretical perspective the emphasis is on mechanisms for continuity of social practice. At the action level, however, people can challenge the system, the potential for change lying in the tension between the person's intentions and preferences, on the one hand, and the socially, culturally and historically mediated6 object of activity on the other. The model employed demonstrates how social practice - in this case student learning - is mediated by a multiplicity of factors.
Using the activity-theoretical model, the ExPhil course of study can be presented in the following way, based on the data provided by the students I followed throughout the course and my own observations.
Figure 2. The ExPhil course as an activity system
In activity-theoretical terms the situation was as follows. At the intention level the object of learning was the fundamental questions and problems of scientific reasoning. Central here was a rational way of thinking. The problem-solving practice that the students were engaged in was meant to make them acquainted with the core principles of for scientific reflection. The teaching and teachers problem-solving in seminars were intended to be prototype examples of a rational, scientific approach to solving the problems that the students had worked on before the seminars. In practice, however, a change took place whereby the study tasks themselves and their specific solutions became more relevant and dominant than the ideas they were intended to illustrate. Instead of being tools to promote the learning of scientific reflection, they took on the character of being the object of learning. This situation was reminiscent of what Engeström (1987, page 101) identified as typical of much school activity. "The text becomes a closed world, a dead object separated from its living context".
In the ExPhil course of study the students were drawn into an institutional practice dominated by a specific kind of problem-solving. This practice manifested itself and was supported by the literature, by the course structure, by the focus on examination questions throughout the course and by specific learning objectives. This was the dominant tendency, but the picture was nevertheless somewhat ambiguous. Interestingly, as the examination drew nearer, the students showed a growing enthusiasm for a broader approach to the philosophical and theoretical themes. They had a greater motivation to study and they discovered aspects of the course material that went beyond the examination and their study assignments. They worked more intensely than earlier in the term and formed small groups where they co-ordinated their individual efforts. This interplay served to create new conditions for learning that were in line with the stated aims of the course. Many of the students acknowledged that the themes dealt with during the course were in fact an invitation to critical and rational reflection on their lives so far, to interpretation of social questions and to a more critical attitude towards the media, etc. The fact, however, that these aspects were not the dominant features of the ExPhil study situation was regarded by several of the students as a loss and a betrayal of what they had expected from a university.
The students were confronted with a fixed structure and a predefined set of tasks. The questions were the same for everybody and the students were presented with the "good solutions" both during the seminars and in various types of supporting material. The material was directly aimed at how best to deal with the solving of relatively standardised problems. Answering these questions, even though intended to be tools for learning, became the object of learning and assumed the character of a goal for the individual. There were several reasons for this: the way in which the students regarded the value of the course; the strong tendency of the seminars to concentrate on the given answers; the nature of the supporting material and the way in which some of the textbooks were written.
The study clearly shows that assessment and examination procedures can have a "backwash" effect. The examination questions and the structure of the final assessment exercised an influence on the basis for the students' learning. The assessment arrangements had an effect on the literature, as well as the teaching and the problems students worked on. They also supplied the premise for what the students interpreted as central to their studies.
Assessment can be regarded as a relatively concrete manifestation of the demands of a course of study. It serves to define the object of the learning activity. The questions to be dealt with on the ExPhil course were largely of a formal character. The quality of any answer depended on whether or not the student had correctly carried out formal operations or reasoning along the approved lines. This was especially true of the second part of the course, philosophy of science, logic and argumentation theory, where the examples were on the whole tailored (or specially selected) to illustrate specific, formal knowledge.
As said before, the picture presented of the ExPhil course here shows the dominant tendency that emerges from my study. The situation did, however, have other aspects. For example, the students found that the content had relevance for the students in their daily life, the way they reasoned and their conception of knowledge and science. The teachers also emphasised this aspect and described it as an essence aim of the course. However, the consequences of the teaching practice that I observed, was quite different. This illustrates the ambience of an educational situation - as learners the students can be pulled in different directions. It was interesting to note that in the period prior to the examination the students gave some evidence of learning that went beyond their preparations for the examination. This happened mainly when the students themselves took a grip on their studies, formed groups, co-ordinated individual work with peer students and allocated time to discuss the study material. In this way connections were identified between the course material and the students' experiences, knowledge and daily life. In the midst of intense preparation for the exam an alternative motive for learning emerged; this is interesting for the social life-world of the students.
The examination and assessment procedures stand out in this study as a significant source for the premise underlying the manner in which staff and students related to the course. The examination and assessment system has a retroactive force in that it tends to define the object of the activity, providing motivation for the teaching programme and the students' learning practice to be tailored accordingly.
The retrospective force of examination on learning was clearly demonstrated in this study. But it was equally clear that assessment had a similar backwash effect on teaching, textbooks and other learning material. It was particularly the interrelatedness between teaching, learning, learning material and summative and formative assessment that constituted a context for a particular educational practice. This was an educational context that the students entered, became part of and acted within in contextually situated ways. The fact that there were different tendencies or affordances within the situation that "pulled" the students way of learning in different and conflicting direction was also demonstrated. The situated character of learning was complex and students had to manoeuvre within a complex system. In spite of the complexity some feature of the educational practice are dominant, while others are subordinate. In this study it seems that the examination played a major role in defining what is dominant - for the way teachers taught as well as for the students' learning practice.
Activity theory seems to be an approach for analysing the interrelatedness between assessment, learning and teaching and coming to grips with the how the configuration of interrelated positions and practices constitute an educational. The emphasis on conceptualisation of the contextually embedded dilemmas or contradictions opens up for analysis of complexities and potentials for change and development.
Baune, Ø,, 1991, Vitenskap og metode [Science and Method], 7th edition, Oslo
Baune, Ø., 1993a, Språk, logikk og argumentasjonsteori [Language, Logic and Theory of Argumentation], 7th edition, Oslo
Baune, Ø., 1993b, Veiledning i oppgaveløsning. [Guide to Problem Solving], Temagruppe A & B, [Theme Group A & B] Oslo.
Biggs, J., 1998, "Assessment and classroom learning. A role for summative assessment?" Assessment in Education, Vol.5, No.1, pp. 103-110.
Cole, M., 1996, Cultural Psychology. A Once and Future Discipline. Cambridge, Mass., Harvard University Press.
Crooks, T.J., 1998, "The impact of classroom evaluation practices on students", Review of Educational Research, 58, pp. 438-481.
Cuban, A. P., 1992, "Curriculum Stability and Change" in Jackson, P. W., Handbook on Curriculum, New York, MacMillan Publishing Company, pp. 216-47.
Elton, L. & Laurillard, D., 1979, "Trends in student learning", Studies in Higher Education, 4, pp. 87-102.
Engeström, Y. & Miettinen,R. (1999): Introduction. In Engeström, Y., Miettinen,R., Punamäki, R.L. (Editor), Perspectives on Activity Theory (pp. 1-16). Cambridge: Cambridge University Press.
Engeström,Y., 1987, Learning by expanding. An activity-theoretical approach to developmental research. Helsinki, Orienta-Konsultit Oy.
Engeström, Y., 2000, "Activity theory as a framework for analyzing and redesigning work", Ergonomics, 2000, vol. 43, no.7, pp. 960-974.
Føllestad, D., Walløe, L. & Elster, J., 1985, Logikk, språk og vitenskapsteori, 2. utgave [Logic, Language and Philosophy of Science, 3rd edition], Oslo, Scandinavian University Press.
Gibson, J. J. (1979): The Ecological Approach to Visual Perception, Boston: Mifflin.
Havnes, A., 1996, "Læring som sosial praksis. Examen philosophicum som eksempel" [Learning as a social practice. Examen Philosophicum as an example] in Dysthe, O., Ulike perspektiver på læring og læringsforskning [Different Perspectives on Learning and Learning Research], Oslo, Cappelen Academic Publishers.
Havnes, A., 1997, Agency in learning. A case study of learning at university. Unpublished Doctoral thesis, Institute for Educational Research, University of Oslo.
Heggen, K., 1995, Sykehus som "klasserom" [The hospital as "classroom"], Oslo, Scandinavian University Press.
Jackson, P. W., 1968, Life in Classrooms, New York, Holt, Rinehart and Winston.
Leont'ev, A.N., 1978, Subject, Object, Cognition, Moscow, Progress.
Lave,J. & Wenger, E., 1991, Situated learning. Legitimate Peripheral Participation. Cambridge, Cambridge University Press.
Stigen, A., 1990, "Om målsettingen for Examen philosophicum" [On the intentions of Examen Philosophicum] in Skram,H.F. and Viestad, E., Hjelpehåndbok i filosofihistorie til Examen philosophicum [Supplementary manual in the history of philosophy for Examen Philosophicum], Oslo, Scandinavian University Press, pp.71-72.
Student Manual for Examen Philosophicum, Autumn 1993, University of Oslo.
Säljö, R., 2000, Lärande i Praktiken [Learning in Practice], Stockholm, Liber.
Vygotsky, L.S., 1978, Mind in society. The development of higher psychological processes. Edited by Michael Cole, Vera John-Steiner, Sylvia Scribner and Ellen Souberman. Cambridge. Cambridge University Press.
Wormnæs, O., 1987, Vitenskapsfilosofi [Philosophy of Science], Oslo, Gyldendal.
1. Research project financed by the Norwegian Research Council.
2. The term 'affordance' was created by Gibson to focus on aspects of the material and sosial environment that "invites" organisms to act in specific ways. A hammer affords hammering and a chair affords sitting.
3. http://www.uio.no/sthb/ExPhilfac/studieplan/ExPhil/presentasjon.html (updated 21.06.2001) 4. http://www.uio.no/sthb/ExPhilfac/studieplan/ExPhil/pensum/del2/temagruppeb.html (updated 21.06.2001). These are just part of the learning goals in this section of the syllabus, but the remainder also use the same system of key words.
5. This situation was true of the early/mid 1990s. I have not investigated to what extent ExPhil literature has changed since then.
6. In the language of activity theory one refers to mediated action and mediating factors (tools, norms, division of work, community - all components in the model I have used as a tool for analysis)