|
|
Promoting the use of feedback as a means of student and staff learning
Lauritz Brännström1
Linköping University, Sweden.
Email: laubr@ibv.liu.se
Paper presented at the European Conference on Educational Research,
University of Hamburg, 17-20 September 2003
Network 22 "Research in Higher Education"
Abstract. Feedback is a necessary prerequisite for the student or the group of students when they evaluate their progression in learning. The feedback can be provided by the student, by fellow students or by the teacher or supervisor. However, the forms and the content of feedback to the student often vary unpredictably in their distinctiveness and usefulness as a support to the learning process of the student. Thus, it can be hard to know what factors made learning more efficient in one situation as compared to another one. Since learning is associated with the student and/or fellow students, the follow up of their learning is often terminated as soon as the student has finished examination and delivered course evaluations. Parallel to the learning of the student there should be a learning associated with the teacher and the teacher's staff. The way the goals of a course are approached by educational means do contain much information for the teachers as to which means should be the most efficient ones. In order to safeguard information about the teaching events as presumptive causes of efficient student learning and staff development, more effort should be put on the documentation of teaching and other decided activities that were supposed to promote learning. The paper gives concrete examples of the use of different means to make the feedback to students more efficient. By beeing more explicit about the goals of the course at focus for the student's learning much uncertainty can be reduced on what is expected of the student as regards the area of content to be learned and mastered. It also means that the self generated feedback of the student will become more easily to assess as compared to feedback around more general and amorphous stated learning goals. By beeing explicit about the goals of the course the teacher also sets the standard for evaluating the student's achievements of the curriculum at focus. From a cognitive point of view the effect of making the constructs and the relations between them more explicit should facilitate the structuring and remembrance of the learning material. Also, staff learning and development should be positively correlated with the expliciteness of the goals stated and the formulated central constructs associated with them. However, much of the teaching activities designed and their impact on student learning are vulnerable to "positive entropy" since many of them will not be properly evaluated and incorporated in the teachers' conception of functionally and efficient means of teaching. Setting goals is also a question of balance: beeing to explicit and precise as regards course goals might result in a closing up strategy of learning while beeing to general and imprecise in goal formulation might give no hints of what is actually expected of the student to achieve. This paper discusses the use of different means to make feedback to students and the staff more efficient and a question of common interest if the ambition is to enrich the curriculum and support student learning as well as staff development. Data from an ongoing project on student and teacher feedback will also be presented.
The role of feedback in an academic context
In order for the student to evaluate progress in learning, whether it be a subject or a skill, he will need information that informs him of his progress toward the goal he is trying to achieve. The feedback information will normally come from his physical or interpersonal environment. In order to be efficient, the feedback, normally exposed as part of an interpersonal process, must be both accurate and focused. Otherwise the students efforts to learn might be reduced to trial and error. The feedback to the student can be self-generated, where the student himself compares information reflecting his aquired knowledge to information in terms of a stated goal for learning or some form thereof. The feedback can come from a single fellow student or a group of fellow students. In those cases the giving of feedback will be part of an interpersonal process where the relation between the student who receives feedback and the one/those presenting it will be crucial. The same holds, in principle, for the situation where the teacher is the one presenting feedback to the student. The interpersonal relation may in that case also be affected by the inequalities in power between the teacher and the student. Added to that kind of inequalities in power are the cultural rules for saving face which are present in any interpersonal situation.
The receiver of feedback must ask himself whether the giver of feedback will present him thruthful information which is guided by sincere motives (to help) and which are relevant for him with respect to what he is actually trying to learn. In turn, the giver of feedback must ask himself whether the receiver of feedback will actually listen to him, whether he will actually take seriously what he has to tell, and, whether the receiver will see the giver of feedback as beeing helpful rather than punishing.
Even if it seems to be relatively easy to stipulate some of the desired characteristics of feedback as a means of supporting the student's learning, we still might find that the form and the content of feedback in an academic context is not always standing in any unquestionable relation to how distinctive and useful it is experienced by the student although the aim should be to facilitate the learning process of the student. A meta-analyses on the effects of feedback interventions on performance (Kluger & De Nisi, 1996) suggest that feedback intervention studies might reveal very different outcome patterns – in a considerable number of the reviewed studies, feedback did even have detrimental effects on performance (one third of the feedback interventions). The results of the analysis also showed that "feedback intervention effectiveness decreases as attention moves up the hierachy closer to the self and away from the task. These findings are further moderated by task characteristics that are still poorly understood." (op cit., p. 254).
The forms and the content of feedback to the student normally do vary considerably over courses, examinations, teachers and so on. Since feedback, in order to be efficient, probably ought to be individually designed to the student in question the precondition for efficient feedback will not always be met. Often the time and effort needed by the teacher will probably exceed the resources at hand. For instance, providing individually designed feedback in a class of students will put great demands on the teacher in many aspects as regards the need for time, the mental effort to be exerted, the content to be well aquainted with, the comparisons to be made between standards of performance and actual performance, and so on. One reaction from the students could, then, concern the scarcity of feedback. However, asking students about the form of feedback they mostly reveal a positive attitude to the idea of a multiplicity of forms. Then, the important point, from the students point of view, is that they actually are getting feedback while the forms of that feedback seems to be a less controversial question (cf London & Smither, 1995).
Another way of arranging feedback is to rely on the information provided by the students of the group that the student is part of. Consulting literature on learning in groups gives rather exhaustive descriptions of how to give and receive feedback as part of a two-way communication process. As an example, one finds in one of them the following rules (Jaques, 2000): The rules for giving feedback are emphasizing that feedback should be invited, be descripitive, be revealing the giver's positions or feelings vis-á-vis the receiver, be specific, be sensitive to the needs of the receiver as well as the giver, be directed towards behavior the receiver can change or control, be timed, be checked to be understood by the giver, be checked with regard to accuracy by consulting group members to the student. The receiver of feedback should listen to the person giving feedback, acccept feedback as a gift from that person, check the feedback with another source of information, give the feedback serious consideration, communicate the decisions made to the giver and tell the giver what he might do to help the receiver change his behavior.
As can be seen from the above listing the feedback concept is referring to concrete behaviors of both the giver and receiver of feedback. It does not focus primarily on questions of uncertainty and dissonance that should be of importance in learning new material. By concentrating the feedback to questions of uncertainty and experienced dissonance as regards facts and principles in relation to already established knowledge etcetera, the value of the feedback probably should be positive to the learning process. Therefore, in order to incorporate the feedback on subject related issues it should refer to the knowledge structure of the individual receiving feedback (see the feedback concept summarized from a cognitive view in Kreitner, Kinicki & Buelens, 1999). By referring to a system of principles for deliberate feedback there should be benificial effects of it in terms of stimulating and facilitating the learning by the receiver of that feedback.
One such system of principles for deliberate feedback to stimulate and facilitate learning can be found in Schein (1999, p 140): "Goals must be clear and agreed to, the emphasis should be on description and appreciation, the feedback should be as concrete and specific as possible, both giver and receiver must have constructive motives, criticism should not be avoided if it is specific and focused on behavior, the giver should own his own feelings and reactions rather than resorting to impersonal generalities, and both giver and receiver should be psychologically ready for a feedback discussion."
Even if the forms of feedback do follow the general outlines presented above there is still a great uncertainty as regards the content of the feedback to the student; how distinctive is the feedback in relation to the content to be learned? And how useful is the feedback information as a support to learning? Is the content of the feedback information of the right kind with respect to the goals of learning? Does the feedback information serve to reduce the uncertainty of the student in his learning effort? In many cases the student will be left to himself to judge whether he has attended the goals of the curriculum. The judgments will mostly be based on rather sparse individually directed comments coming from the teacher. The scarcity of comments are mostly due to the fact that individually oriented feedback will be to costly to realize, thus, more standardized forms of feedback will be used, like feedback to groups of students instead of individual students. One illustration of the complexity of processing feedback is summarized in a model by Ilgen, Fisher & Taylor (1979).
In the following three realistic examples of the complexity of questions raised about providing and using feedback information in higher education will be presented. According to the advices given on how feedback information ought to be presented to individuals and members of groups the following examples will extend the analysis to an academic context and beyound the rather straightforward situations normally described as a kind of prototypical examples. Two of the examples will illustrate the complexity of providing feedback to the student and his fellow students in an academic context (very little effort has been directed to that question according to Anderson, 1995) but also its consequences for the teacher or to the staff of teachers (the third example) as regards their tasks and their learning,
First example - "Organizing work activities" is one course representing a part of the organizational perspective on the Programme for psychologists at our university. The students are working in study groups using problem based learning. The goals of learning as well as the central constructs to be learned are formulated and assembled in a written document. The goals of the course are on a relatively abstract level (like "aquiring knowledge about different models of organizing work activities and their applicability in practice"). Also the central constructs are at a general level (like "types of organizations", "decision levels", "coordination" and so on). During the study period of the course the choice of the study groups will be their interpretations of the goals formulated, thus, student learning is to a high degree dependent on what the students as a group decide to focus on. The individual student then can make comparisons whether he is in the vincinity of the goals stipulated by the group but also make comparisons to the goals of the written document. The perceived discrepancies between the student's knowledge of the subject and the goals stipulated will help the student make further efforts in directing his learning activities towards what is expected.
A still further adjustment of the goals of learning will be made by the teacher in order to be somewhat more specific about what achievements the students is expected to manifest in the examination of the course. The adjustment is a result of a feedback information from the students who, as the examination approaches, want more information about the criteria used to evaluate whether the officially formulated goal(s) are reached or not. By informing the students more specificially what they are expected to know and what performance to manifest, the student can make further adjustments of his learning efforts. More precisely a goal stipulated as (during the course the student should be) "aquiring knowledge about different models of organizing work activities and their applicability in practice" was given a more precise operationalised form in (the student should) "have such knowledge about different models of organizing work activities that he can apply the constructs of the chosen models on those (new) examples that form the examination task (at least two different models are requried)". Given those prerequisites the student may mobilize further efforts in order to adjust his knowledge to a closer match to the stipulated learning demands. As regards the central constructs of the course similar clarifications were given. Each construct should be defined and its relation to other central constructs of the course should be formulated.
The examination was organized around four short descriptions of real examples of "organizing work activities" (different wards at a university hospital). The task for each study group was to analyze the examples in accordance with the stipulated goals and the central constructs of the course as formulated in the written document over the course and the more precisely formulated criteria presented at the examination. For example a more precise goal could be formulated as "in the study group students must be able to show that they have the knowledge of different organization models to such a degree that they are able to apply the constructs of the models to the examples of 'organizing work activities' that are presented in the examination task". They were also told what was expected as regards knowledge of the central constructs of the course: to know the meaning of each single construct (as a definition or as part of a context understandable to others), to show how a single construct can be related to other main constructs of the course, and finally, to value the importance of the construct in order to describe the activities that are present in the examples of the examination task.
After the course the students were asked to rate how well the stipulated and more precisely formulated goals had been reached. Mostly the ratings of the students were very positive with small dispersion for all the goals to be achieved (a mean of around 5 on a 7th grade scale with a dispersion of approximately one scale unit; n=24).
The students reactions to the kind of examination used was quite positive with mean rating around 5 on a 7th grade scale and a dispersion somewhat exceeding a scale unit (n=24). The students verbal comments to the examination were also very positive and underlined the fact that examination is another opportunity to learn more about the subject at focus. Typical comments from the students were "enjoyable form of examination. An opportunity to apply the knowledge one has acquired and at the same time learn new things" (respondent A), "a very good opportunity for a new learning experience built on the intellligent use of the knowledge of the group members. The examination task was very relevant for the course." (respondent B), "an opportunity for further learning by using the knowledge of the group put into a relevant context" (respondent C).
Thus, in order to support the student learning there is a number of feedback situations needed and where the precision of the feedback information becomes gradually more articulated. By observing the discrepancy between the goals of learning and the student's own knowledge of the subject area, the student may subsequently direct his learning efforts more intelligently. From a teacher perspective there must be a parallel and ongoing processing of information from the students when they are trying to adjust their learning effort to match the goals of learning. The information from the student may come from a number of sources like questioning at lectures, discussions at seminars, from students in work group and so on.
Second example – multiple goals and complex feedback. On the Programme for Psychologists there is a course focused on 'leadership, followers and conflicts in groups'. The course is the last one in a sequence of four courses focusing on aspects of group psychology where the three foregoing courses are on group dynamics, the study of groups, and, how to work with groups. The final examination on the perspective of group psychology is fullfilling multiple goals of which some of the goals are linked specifically to the final course in question. The student is free to choose one or two content areas to focus on ( leadership, followers, conflicts or a combination thereof). Another goal of the course is the writing of a paper according to APA norms (APA, 2001) on the chosen subject(s) of the course. Still another goal is about presenting the paper at a 'conference' with individual contributions from all the students. This goal is combined with the listeners ability to present feedback on how the presentation of the paper was managed by the student. A small group of students are, in addition, made responsible for arranging and leading the conference. These students also have to document and critically evaluate their contribution to the conference as regards handling the process of realizing it (as an application of group dynamics and leadership). Thus, the learning goals associated with the conference are from the students' perspective quite differentiated.
The task of the teachers during the conference is to critically evaluate the presented papers "on-line", but also to evaluate the individual presentations of the students at the conference as well as make observations on the students leadership while managing the conference. The teachers also must critically evaluate the papers of the students and presenting that feedback "off-line" (content and form issues). Thus, from the perspective of the single student there is a number of sources of feedback information to judge the paper and the performance when presenting it.
The subjects chosen for the paper will be compared to the knowledge the student has about the subject areas at focus. Ideally, the student will search for new knowledge and try to combine it with those subject areas already known to him. Thus, alternative knowledge structures should be formed which consolidate what the student already knows with information reducing his uncertainty within new areas of inquiry. The freedom of choice for the student should make appeal to his study motivation in mastering these new areas of knowledge. By later presenting the results of the efforts to better understand the subject area chosen and writing them down, the student will make his efforts publicly accessible and possible to evaluate by other students and by the teacher. The outcome of that evaluation process will in turn be a support to the learning of the student by making it more or less obvious what aquired content is in accordance with the stipulated learning goals of the course and what is not.
The writing of a paper should be reflecting the fulfilment of goals related both to form and to content. In order to make it more easy to share the content of the paper with other students, the student should follow the recommendations of APA on how to structure the writings. A good structure reduces some of the reader's uncertainty about what a paper should look like and the reader might use more efforts to process the contents of the paper. Since there is a recurring task to write papers according to the APA norms the writing of this specific paper will represent another learning experience added to those earlier. However, in what respects will the presentation of the content represent new challenges to the student instead of being experienced as another routine task subsumed under the writing of articles? By writing a paper to be presented at a conference the student should be very much motivated to give considerable attention to the writings and to mobilize effort in it. At the stage of presentation there should be a kind of pay back to the student for his/her intense work on the paper. Since no product will be perfect he should also experience a dissonance between the investments made and the outcome observed in terms of reactions from fellow students and the teachers, often in terms of "why haven't you been more explicit on the relation between construct A and B etcetera". Each critical question serving as input for a deeper learning and understanding of the subject presented in the paper.
During the conference the students as listeners are to present feedback to the student presenting his/her paper. Before that event takes place the students are fully informed about the conditions as regards the aims and the rules of conduct for the feedback session (following the recommendations of Schein, 1999 as regards deliberate feedback). This kind of arrangement gives the student immediate feedback whether the message of the paper was presented orally in such a way that most members of the audience were able to follow the sequence of thoughts represented in the paper. A negative response from the auditory in one or another aspect should make it necessary for the student to reflect on the message presented – "why didn't it work as expected?". The experienced dissonance between the intentions and the actual behavior might be helpful indicators for the student as to what to improve would the presentation be done anew. Was the content of the presentation understood? Was the form accepted? Did the form of the presentation underline the content of the presentation?
A group of 4-5 students are given the task to organize the conference as regards the agenda from the beginning of the conference until it ends. The task is supervised by a senior lecturer and contains a number of issues to be solved, like "who is to present what in which subject context". Since the conference is dealing with a great number of papers on leadership, followers, and conflicts in groups the time for the presentations will exceed the time at disposal if every student should attend to every presentation, the day of the conference will therefore be organized in three parallel sessions each focusing on mainly leadership issues, issues concerning the followers in a group, and, issues of conflicts in groups. The hard task of the organizing committe is to order all the papers such that they are forming interesting themes to be followed in a session.
The students of the conference were free to move between sessions during the conference day with the exception for those occasions when the student acted in the role as respondent and opponent. Those in the organizing committe chaired the sessions. In order to serve with feedback on the contents of each paper presented there was an appointed opponent whose task was to prepare for scrutinizing the paper to be presented as regards its content and its form. After the opponent had finished his task other students were free to ask questions to the respondent. Added to that task which serves to give feedback primarily on content issues, all students who attended a specific presentation had as a standing task (i e the same instruction) to provide the student who presented his paper with feedback on aspects of that presentation. The feedback was given as a kind of cooperative act where the respondent got answers to the question "how did I manage to present the content of my paper to my audience?" from his fellow students interested in his presentation (by attending a specific session it could be assumed that the students also were interested in or motivated for the content issues to be dealt with).
Besides the task of chairing the parallel sessions the members of the organizing committe were to document how they managed to realize the undertaking of the conference. Each student summarized his observations from each presentation in a kind of "diary" but with focus on how well the conference issues were handled at any moment.
On each parallel session there was a teacher following the presentation of the students. After each presentation and the feedback session from fellow students the teacher summarized and presented his observations and opinions about the paper presentation. That summary was added to the feedback from the opponent (mostly content issues) and, the other students (mostly performance issues), thus, the feedback presented to the student was quite massive. The teacher also had to observe and document how the students organizing the conference acted in order to manage their task and so as to later on be prepared to compare those observations with the intentions of the members of the organizing committe.
Since some feedback from the teacher is not easily presented on-line due to its complexity, each paper was critically examined "off-line" and provided with written comments of the teacher (representing one of the more formal aspect of examination of the course).
Some weeks after the conference the organizing committe presented their picture of the conference and how it had been experienced by those who participated. Added to the observations the students of the committe had made during the planning and realizing of the conference, they also made the participants of the conference rate and comment a number of aspects (like information to the students, the arrangements of the conference day, the processing of the student papers during sessions, the feedback sessions "on-line", the summary of the teacher, the written feedback from the teacher an so on). Most students were very positive to the conference concept and valued its impact on their learning the subject issues as very high. The conference seemed to have an integrative function in helping the students to relate the different content areas to each other. It also seemed to strengthen the learning of the subject areas at focus of the conference. Since the last course and the conference had been preceeded by three other courses - on group dynamics, the study of groups, and, how to work with groups - the students were very well prepared to incorporate new material to their already formed knowledge about groups. The students comments after the conference support that interpretation. Since the conference invites the students to take new initiatives to structure the content of the passed courses of group psychology to fit the specific content issues of the conference (and the last course) the student were, probably, optimally motivated for their task to integrate older knowledge with the knowledge to be sought according to the goals of the final course. Also, the freedom of the students to participate in those seminars that were the most tempting to them should support that learning process.
Third example – course evaluations as a prerequisite for student and staff learning. In one of the paragraphs regulating the institute of higher education in Sweden the role of course evaluations is stipulated: "The institute of higher education must give the students who participate in a course or who have finished a course the possibility to express their experiences of that course and to have opinions about it. To accomplish that the course evaluation must be arranged by the institute who also shall bring together the course evaluations and inform about the results and the decision about consequent actions as a result of the evaluations. The results must be accessible to the students." (quote from The Higher Education Ordinance (2000), chapter 1, paragraph 14; free translation)
On a more precise level at the university it is stipulated that "Course evaluations, primarily, means that the students are given feedback on the course they are following or the one they have just finished. This also means that students should reflect over their education (or training) and to be critical to and even question it." (quote from the decision protocol of the university board at Linköping university 2002-06-10).
In other words, course evaluations are stipulated for all higher education in Sweden. The Higher Education Ordinance says that students who have participated on a course or have finished it must have the opportunity to present their experiences of the course and to have opinions about the course. The institute of higher education is also obliged to summarize the course evaluations and to inform the about the results and consequent actions followed by them. The results must be accessible to the students. On a local level course evaluations are seen as feedback on the course that the student is following or just have finished. The students are also asked to actively reflect on their education and to critically scrutinize it and even to question it.
Thus, the course evaluations are motivated from a student perspective. The form of the evaluations are not prescribed and can be flexibly designed and used by the teachers of the course. The main components of the course evaluations normally asks the student to reflect on the achievement of the goals of the course under the circumstances prevailing in terms of input resources (teachers, instructions, literature, schedule, rooms and so on). Questions are also raised about process issues on how the teaching, group work, individual tasks etcetera was managed. Ideally, each teaching event is evaluated by the student via quantitative measures (i e ratings) and qualitative measures (i e descriptions of how a certain teaching episode contributed to the student's learning of the subject at focus). In addition students are asked questions on the outcome of the course activities in relation to the course goals. On an individual level the student also should reflect on whether he has achieved the formally stipulated learning goals, whether the feedback presented to him has helped in that process, and, whether the actions from the teacher and student colleagues has facilitated the achievement of the learning goals of the course. Via the course evaluations the student should have ample opportunities to reflect how the course was organized to support the student's own learning. The course evaluations present a number of causes that could have effect on the student's own learning. By summarizing and sorting the students reactions to those causes it might be possible to infer what has affected the student learning the most.
The use of course evaluations from a teacher perspective is not as articulated as the use of course evaluations from a student perspective. The Higher Education Ordinance just states that course evaluations must be made during or after a specific course by the teachers who also are to inform the students about the results. From a teacher perspective course evaluations might be used as input in a recurring process of changing the teaching conditions in order to become more efficient means for helping the students learn the content of the course. Reading course evaluations from different courses on a study programme, for instance, will probably give a rather disparate picture of significant course events (those selected for the specific course evaluation instrument) and their effects on student learning (as measured by the selected items to assess students reactions). Will the analyzing of the summaries of course evaluations offer the teachers any deeper insight into how the teaching of courses might be improved? The answers ought to be 'yes' although the evidence on that issue might be sparse. Instead the summaries might present general statements on teacher performance, the scheduling of activities, the forms of student work and so on. These kind of data may be important information as signal to change even if they do not tell us why changes should be made.
Alternatively, by using a kind of systems approach to evaluating the teaching activities and surrounding conditions it ought to be possible to infer from the course evaluations what measures to take in the support of student learning. Thus, course evaluations from a teacher perspective should contain information about the goals of the course (on such a level that they can be communicated between the teachers and students; possible to operationalize; preferably in terms of subobjectives), input conditions of the course (in terms of different types of resources), processes (in terms of teaching, selection of tasks, the order of their presentation, the choice of checking points and so on), output measures (in terms of student behavior displayed like writing a paper, doing an exam, presenting an analysis and so on). The course evaluations also should contain comparisons between the output measures defined and the criteria or goals stipulated, thus, forming the feedback information. Depending on how well the output measures coincide with the stipulated goals or criteria of performance (according to the student reports) it can be concluded whether the students are meeting the standards of the course. Depending on the use of the feedback information the course or parts thereof might be changed when it is ongoing or the change may come next time the course is given, thus, measures, actions or activities may by used on a short or long term basis.
The use of a systems approach does present special demands on the design of the evaluations instruments, since all the system components ought to be qualified in important aspects. For instance, using a questionnaire it should be structured according to the systems constructs in order to make it possible to later pinpoint what parts of it were the most important for goal fulfillment or the fulfillment of subobjectives. The teachers should accordingly try to describe, over time, the teaching activities and surrounding conditions of the course. By documenting the course events and observed results in a kind of diary as well as the safeguarding the reactions of the students (via observations, interviews, questionnaires, archives) it should be possible to better infere means-end relations about the course. It would also make it easier to identify causes to why aspects of the course have their impact on the student reports. For instance, "why does not the teaching form X give the expected results Y?". By identifying possible causes to the 'malfunctioning' it might be possible to change the preconditions for using the teaching form X and to observe the results of that change. If the hypothesis about the malfunctioning is correct and the change of preconditions gives the expected results there should be a corresponding learning of the teachers trying to remedy the insufficiences associated with the course.
A final reflection on the use of feedback in academic contexts
Most of the literature cited in the paper on how to present feedback are referring to concrete behaviors of the giver and the receiver of feedback. In order to strengthen or supporting learning in academic contexts the focus of the feedback should supposedly be oriented towards the uncertainty and the dissonances formed when the students enters relatively unprocessed subject areas. Since many of the of the goals of study are not always articulated fully from the beginning, efforts should be made to make them subsequently possible to approach. By using interactive feedback processes where the goals are subsequently becoming more precise, the students learning efforts will probably be adjusted to the different challenges of the subject areas encountered. In each of the examples presented above in the paper this interactivity is shown to be quite complex but also quite possible to master.
References
Anderson, J. R. (1995). Learning and memory. An integrated approach. New York: John Wiley & Sons.
Ilgen, D.I., Fisher, C.D., & Taylor, M.S. (1979). Consequences of individual feedback on behavior in organiszations. Journal of Applied Psychology, 64, 349-371.
Jaques, D. (2000). Learning in groups. A handbook for improving group work. 3rd ed. London: Kogan Page.
Kluger, A.N. & DeNisi A. (1996). The effects of feedback inteventions on performance: A historical review, a meta-analysis and a preliminary feedback intervention theory. Psychological Bulletin, 119 (2), 254-284.
Kreitner, R., Kinicki, A., & Buelens, M. (1999). Organizational behavior. London: McGra-Hill.
London, M. & Smither, J.W. (1995). Can multi-source feedback change perceptions of goal accomplishment, self-evaluations and performance-related outcomes? Theory based applications and directions for research. Personnel Psychology, 48, 803-839.
Publication manual of the American Psychological Association. 5th ed. (2001). Washington, DC: American Psychological Association.
Schein, E.H. (1999). Process consultation revisited. Building the helping relationship. Reading, Mass.: Addison-Wesley.
Note: