'Thinking Aloud' - a technique for uncovering multi-sensory learning in professional education
Professor Valerie Hall DPhil, MSc, PGCEA, RM
Head of Centre for Nursing and Midwifery Research
University of Brighton, Mayfield House
Falmer, Brighton BN1 2HS
Paper presented at the Society for Research into Higher Education Conference, University of Edinburgh, 13-15 December 2005
The ‘Thinking Aloud’ technique was developed as a data generation and gathering tool in a phenomenological study looking at how students learn from experience to become midwives. In this paper I propose that the 'Thinking Aloud' technique is an appropriate tool for illuminating the under researched area of multi-sensory learning in professional education. I will explore the difficulties of collecting data about learning and then I will explain how the 'Thinking Aloud' technique addressed some of these problems. I will describe how the data was collected and then assess the ‘Thinking Aloud’ technique in terms of its value in collecting data on learning. The findings will be presented and discussed in respect of their relevance to learning and teaching in Higher Education.
The ‘Thinking Aloud’ technique was used as one method of data collection in a phenomenological study looking at students’ use of experience in developing midwifery knowledge. The other methods were semi-structured interviews and an analysis of curriculum and course documents. The students were undertaking an undergraduate pre qualification midwifery course which consisted of 50% campus based learning and 50% clinical practice based learning.
The sample was purposive and included two groups of student midwives, all but one of whom were female:
Group 1. 17 students who commenced a three-year course leading to initial qualification as a midwife
Group 2. 23 qualified general nurses who commenced a shortened eighteen-month course leading to initial qualification as a midwife
Because of the differing course lengths, the students undertaking the three year course were interviewed at approximately the six-month and eighteen-month points in the course and the shortened course students were interviewed at approximately the three-month point in the course. The ‘Thinking Aloud’ episode occurred at the end of each course.
Three-year course (N=17)
1st interview at mid point
2nd interview at mid to end of unit
‘Thinking Aloud’ episode
Shortened eighteen-month course (N=23)
1st interview at very end of unit
‘Thinking Aloud’ episode
Table 1 Showing the number of student participants and the interview time scale for each group.
Although the first interview with both sets of students yielded sufficient information to provide a baseline for future interviews, methodological problems in collecting data on learning became apparent.
1. Students may not recognise or be able to articulate tacit aspects of their learning experiences.
2. Students’ reports of learning may not be representative of both the campus based and clinical practice based elements of the course.
3. Interviews conducted during a campus based teaching day may cause the students to limit their response to what they are learning in that environment.
4. Other recall influences may affect the type of learning situations reported.
5. The need to reflect the progressive aspect of knowledge development, in the words of one student ‘how it all fits together’.
The need to address these methodological problems influenced the style and content of the second interview with the three year students and led to the use of the ‘Thinking Aloud’ technique in an opportunistic sample from both groups in the final part of the course. Students in this sample were observed while simultaneously recording their thoughts during the practice activity, and were interviewed immediately after it.
Challenges in Collecting Data on Learning
The five points above indicate that the problems for researchers in investigating learning are not insignificant. Interviews rely on events that happened in the past, and the interviewee’s selective memory will be stimulated by the emotional, important, or even novel event.
Eraut (1999) suggests that because respondents are unaccustomed to talking about learning, they are more likely to refer to a formal learning scenario. If this is so then careful questioning may encourage them to think laterally; however, it may also result in leading questions and answers calculated to please the researcher. The ability to tell may also depend on other aspects such as the level of awareness that learning has occurred and confidence in use of language to express thoughts. In a study on workplace learning, Eraut (2000:120) identified five factors that facilitated respondents’ ability to tell:
1. The use of a mediating object which the subjects were used to discussing.
2. The workplace environment was one of mutual consultation encouraging those consulted to describe what they know.
3. Training or preceptorship relationships existed in which explanations were expected, including cultural and behavioural norms as well as more technical ones.
4. The existence of informal relationships leading to work related discussions out of hours when more provisional or riskier comments might be made.
5. A crisis, review or radical change, which causes people to exchange opinions and experiences, sometimes making values more explicit.
Following these recommendations the shortened course students in the first interview and the three year students in the second interview were asked to describe a learning event facilitated by their preceptor* in the practice setting. It was anticipated that some of the characteristics described above would have been present and would, therefore, better enable the telling of the learning experience. As a result, students were able to describe a learning scenario and in all cases they chose one that had occurred in a practice setting. However, it is recognised that this technique does not address the formerly mentioned effect of selective memory on the choice of experience, which means that the experience reported may not be representative of a normal everyday practice situation.
‘Thinking Aloud’ technique
In order to facilitate the reporting of a learning description representative of everyday clinical practice, the ‘Thinking Aloud’ technique was employed in the last phase of the course. This technique was originally used in the 1920s to describe problem-solving processes, although latterly it has been used in a number of different contexts (Nielsen et al 2002). In health care research it has been used mostly in simulation situations (see Cioffi and Markham, 1997), which denies the richness of the contextual data involved in decision making.
Fisher and Fonteyn (1995) explore the use of an adaptation of the ‘Thinking Aloud’ technique in a non simulated setting by looking at nurses’ use of heuristics (reasoning strategies) for clinical decision making in an Intensive Care Unit. They questioned whether the use of the method might disrupt the ward routine and /or compromise (physiological or psychological) patient safety. The potential sensitivity of relatives hearing the ‘Thinking Aloud’ data was recognised but it was proposed that from anecdotal evidence the relatives of the unconscious patients expressed ‘comfort at hearing the nurse’s thoughts about the care activity and treatment decisions’ (Fisher and Fonteyn 1995:270). Similarly, anecdotal reports from the practitioners indicated that it did not interrupt their care giving or the routine of the unit. In midwifery it is usual for practitioners to describe their decision-making to the mothers and families in order to gain consent, and also to empower them to engage in the decision making process. As family/midwife interaction is unlikely to be disrupted, a modified version of the technique was developed.
Bellezza (1986) believes, despite the critics who suggest that verbal reports have little to do with the cognitive processes actually used, that ‘verbal reports can play an important role in the study of human learning’ (1986:251). Unfortunately, the type of learning mentioned in his work seems limited to mainly recall and verbal rehearsal. Discussion on verbal reports for the affective domain suggests that affective symbols may become attached to cognitive symbols and may enter permanent memory. These are assumed to have inhibitory connotations as ‘people may not be able to verbally report this information’ (1986:267). However, he refers to an interesting experiment by Greenwald (1968) who studied the effects of persuasive messages on attitude change. When asked to write their thoughts whilst the message was being imparted, participants were found to be rehearsing their own attitude to the message rather than rehearsing the message. Bellezza’s (1986) claim for verbal reports as scientific data on which to investigate learning is built on the following set of premises:
1. A person is aware of information that represents some of the structures and processes of the memory system.
2. Information of which one is aware is stored in short term conscious memory.
3. Verbal reports can be given describing the contents of conscious memory.
4. Information generated by the cognitive system and available in conscious memory may become linked to new information presented to the learner.
5. Later, some of the contents of conscious memory can function as recall cues for other information that previously occurred with it.
Although the issue of tacit knowledge is not explicitly addressed, the proposal that what is generated by the cognitive system may become available to the conscious memory hints at the possibility of what is tacit becoming explicit through verbalisation.
Ericsson and Simon (1984, 1993, 1998) spearheaded the recent use of verbal reports as data. They identify three levels of verbalisation in ‘Thinking Aloud’ which are:
1. Direct articulation of the material in conscious memory already in language code.
2. Re-coded non verbal information without additional processing, e.g. describing a visual image.
3. Articulation preceded by some inference or other cognitive process, which is not a description but a transformation.
They state that reports using the first two levels yield appropriate data, but the third does not.
Jeng (1996) provides an overview of the use of verbal reports and he states that the subject’s lack of familiarity with the technique does not affect their ability to participate nor does awareness of the process alter thinking and performance (Norris (1990). However, later authors report that the two cognitive processes that of thinking and verbalising are competing (see Nielson et al 2002).
Despite the different use of the ‘Thinking Aloud’ technique in this study, compared with that proposed by Bellezza (1986) in which the verbal protocol recorded a learning task, and other studies in which the participants report on clinical decision making, the idea seemed sufficiently useful for gaining some insight into the complex issue of learning. The other major reason for the ‘Thinking Aloud’ use was the assumption that it might stimulate the student to verbalise that which might otherwise remain unspoken. Thus the last interview was planned as a ‘Thinking Aloud’ episode to take place in the consolidation part of the course when the student would normally provide care to the women unaccompanied by a midwife preceptor. The element of a real life situation was expected to stimulate the student’s ‘spoken out loud’ recollection of how they had learned a particular aspect of practice. The recorded dialogue happened contemporaneously with the event that stimulated it and the event witnessed by the researcher. After, and to a much lesser extent during the event, the researcher sought clarification of the student’s thoughts and actions, checking interpretation with the respondent. The student was also invited to provide any further comment on her learning. This utilises the third level of verbalisation described by Ericsson and Simon (1984, 1993) although it was included to gain contextual information rather than be a part of the analysis of learning activities reported.
Significant aspects of the ‘Thinking Aloud’ techniques employed in the study were:
To provide a spontaneous report about non-formal learning rather than that remembered for novel or emotional reasons.
The ‘Thinking Aloud’ episode was conducted in a real life situation in a sensory rich environment.
The ‘Thinking Aloud’ episode aimed to stimulate recall rather than record new learning although within the care giving setting the two may be inter-linked.
The recall stimulation was used to potentially access other information held in permanent memory.
Detailed description of ‘Thinking Aloud’ episode
An opportunistic sample from both student groups undertook the ‘Thinking Aloud’ technique. Researcher observation and an interview augmented this technique and immediately followed the care-giving episode during which the students speak and record their thoughts. The use of a real life practice episode was expected to stimulate the students’ spoken out loud recollection of how they had learned a particular aspect of practice. A tape recorded dialogue happened contemporaneously with the event that stimulated it and the researcher witnessed the event. After the event, the researcher sought clarification of the students’ thoughts and actions, checking interpretation with the student. The student was also invited to provide any further comment on her learning. The ‘Thinking Aloud’ episode was not planned as participant observation, nor could the observer’s presence be separated from the activity being observed. A compromise was sought whereby the observer was accepted as being non-judgmental and her presence, apart from observing and occasionally prompting students to speak, was unobtrusive.
NHS Ethics Committee approval was sought and consent obtained from the preceptor, a woman client from her caseload and the supervisor of midwives of the NHS Trusts concerned. The ‘Thinking Aloud’ episode was planned to last approximately half an hour and students chose the particular part of the care-giving episode to be recorded.
The researcher travelled in the student’s car to the woman’s house. Prior to arriving at the house the student was reminded that she could choose the aspect of the care used for the ‘Thinking Aloud’ recording. During the recorded episode the student should give care as normal but, while doing so, she should speak out loud her thoughts on the knowledge she was using and how it had been acquired. The student was also reminded that the researcher would be watching the care giving and would prompt her to speak if necessary but would not be assessing her in any way. Anything the student felt was inappropriate to say in front of the woman could be reported at the follow up interview.
When invited into the house the researcher reminded the woman about the taped portion of the visit. She also reminded the woman that she could ask for the tape recorder to be switched off at any time and that once the recorded care-giving episode was complete there was an opportunity to have further (non-recorded) consultation. The students’ spoken aloud thoughts were recorded on a small voice activated tape recorder contained in a carrying case positioned comfortably around her neck. This strategy was used to ensure that as far as possible only the student’s voice activated the tape recorder and was recorded. The student spoke her thoughts about how she had learned the aspect care she was currently undertaking. During the recorded episode the researcher observed the scenario and in some cases prompted the student to speak her thoughts out loud. When the visit was complete the researcher and the student left together to undertake the follow up interview either back at the community midwives’ office or in the student’s car away from the woman’s house. In the follow up interview the researcher sought clarification of the student’s thoughts and actions, and checked her interpretation with the student. The student could also make further comment about her learning and add anything she was unable to express in the woman’s presence.
The ‘Thinking Aloud’ data was analysed prior to indexing, using the three stages described by Miles and Huberman (1994). In the first level the tapes were transcribed verbatim and analysis aimed to capture the overall impression gained from hearing and reading the student’s report. The second level analysis identified individual sections of text in which the student appeared to be speaking ‘learning’ thoughts out loud. The purpose of this analysis was to extrapolate and comment upon general statements that applied across the nine episodes. The third level analysis of data identified statements relating to learning development, which also contributed to the longitudinal non-cross sectional analysis charting an individual student’s learning progress through the course.
Assessment of the method
Although the ‘Thinking Aloud’ episode was introduced to stimulate students’ reports about their learning sometimes the decision making process was articulated instead, which is not unexpected as the two are inter related. Some students had more insight into the difference between learning and decision making processes and it was possible to obtain learning descriptions from all students.
Methodologically, the ‘Thinking Aloud’ episode was complicated by the artificiality of an observer being present during the interaction, and the effect that this might have on the interaction. However it is suggested that the reality factor of the experience was not diminished by the presence of the observer, but that the care giving may have been enhanced because in some cases the verbalisation of thoughts encouraged the mother to ask further questions. In eight out of nine episodes, care giving was not adversely affected by the presence of the observer but, had she not been present, the articulation of learning might not have occurred. In three cases students found difficulty in articulating learning without substantial prompts. One student later reported that she found the tape intrusive; she said that she had reacted badly to it and this, she reported, affected her verbal interaction with the mother. Although the student reported being less communicative than usual, in the view of the observer who is a practising midwife the delivery of care was not unduly compromised. The mothers visited were given the option of having the tape turned off at any time during the care giving episode, or the opportunity for a further non observed care giving episode, but none chose to do so.
The proportion of students who experienced difficulty in speaking their thoughts out loud (without prompts) constitutes a third of the total group undertaking the ‘Thinking Aloud’ episode. Therefore the effectiveness of the ‘Thinking Aloud’ technique as the sole method for enabling students to articulate learning requires further investigation. However, in this study the learning descriptions were obtained from both the ‘Thinking Aloud’ episode and semi-structured interviews, enabling cross checking of this aspect of the findings.
Discussion of the findings
Non-verbal learning was articulated more in the ‘Thinking Aloud’ episode than in earlier interviews. This may indicate that at this point in the course the students had a greater appreciation of its contribution to learning.
Visualising, usually relating to anatomy and physiology
External tactile memories
Internal bodily sensations
Memories with an emotional base
External auditory sources such as listening to another’s voice or other sound
Heard from views and behaviours of other (non-professional) women
These aspects of learning were not reported in early interviews as ways in which the student would learn to become a midwife, but students did feel that their ability to emphasise was something that existed prior to entry into the course.
It may be that the above aspects of learning did not spring readily to mind because the initial interview was held in an office, in contrast to the stimulation of being in the practice setting. Alternatively, the difference could represent a change in the learning process.
In view of the absence of much reported propositional learning in the ‘Thinking Aloud’ episode, it is tempting to suggest that the learning reported was only the tip of the iceberg, and therefore more occurrences of learning may have been implicit in the care giving. However, there are other possible explanations for the limited amount of propositional knowledge reported:
1. Care giving decisions are based on explicit propositional knowledge that has become tacit.
2. The scenario is complex and of necessity memory is selective.
3. Memory of learned propositional knowledge is better stimulated in classroom based settings.
4. Learning from preceptors using propositional knowledge is not interpreted as such if taught by a preceptor.
The nature of professional practice makes learning in the practice setting unpredictable and complex. This combination of unpredictability, coupled with an inherent inability to define specific and all encompassing outcomes, means that some aspects of the learning are likely to be tacit. Fox (1997) asserts that implicit learning occurs in the nursing practice environment where ‘the stimuli are rich and complex and the acquisition of knowledge is characterised by the absence of both conscious awareness and explicit strategies for learning’ (1997:460). Even if some of the expected learning is pre-specified, multi-sensory non-verbal aspects are not usually overtly recognised or commented upon either by the student or the preceptor; nor does preceptorship preparation address this type of learning.
The most common form of non-verbal learning was visualisation, reported by the students as ‘seeing pictures in one’s head’, or ‘in the mind’s eye’’. This is contrary to the findings of Belenky’s (1986) study of female American undergraduates where the main findings are described through the women's use of metaphors relating to voice, which are then interpreted as a crucial difference between male and female knowledge construction. The equivalent male metaphor is that of the ‘mind’s eye’ which is assumed to indicate a distance between themselves and the situation whereas voice and hearing indicate closeness to the experience. Whilst Belenky’s work can be criticised for taking a stereotypical approach to gender attributes, two major sensory modes associated with higher education are highlighted. The use of sight is common in higher education and much propositional knowledge in the form of written texts is accessed through this mode. It is usual for visual material to be accompanied by verbal/auditory commentary during lectures. Interestingly, in this study, during interviews and the ‘Thinking Aloud’ episode, students did not make much reference to the auditory sense input. Given the amount of auditory information in clinical practice, it is tempting to speculate on the possible reasons for its absence from the reports.
Possible reasons for minimal use of auditory information:
It is more taken for granted than visual tacit sources, and therefore underreported.
Preceptors, who expect to use propositional knowledge in the form of pictures and text-based information, cite it less.
Because visual and auditory sources often go together, the primary visual mode was reported, leaving the secondary tacit.
The stimulation provided through care planned in the ‘Thinking Aloud’ episodes did not require students to listen to fetal heart, or breathing patterns etc., therefore memories were not stimulated.
A great deal of evidence existed for the use of feeling, which included kinaesthetic (internal bodily functions), touch from being touched as well as from touching other people, and through emotional feelings.
There were many examples of students’ learning that was facilitated by touching and being touched by others. Students gave many examples of learning that had either an emotional component, or were stimulated by a recent emotional event or memory. Sadly they did not often recognises emotional feeling as a part of the learning itself and in some instances they saw it as detracting from the learning. This finding does question the importance that teachers and preceptors placed on the affective domain within the experiential learning cycle.
Although scarce the evidence of kinaesthetic imagery accessed through internal bodily sensation is particularly significant in the light of arguments regarding the inaccessibility of ‘reflex’ actions.
There was only one reference to sense of smell, possibly due to the lack of stimulus to memory in the ‘Thinking Aloud’ episode. Because of the nature of the work it is not surprising that there is not any reference to taste.
It is clear from the data that images are used for more than just recall. Images appear to constitute an important part of learning by representing what has been perceived in order to understand, reason and provide a basis from which to hypothesise. There is also evidence that images are manipulated to imagine scenarios and theorise about these scenarios as in abstract thought. For example, one student described an image of a woman scrubbing the front step, through which she worked out that the fetus was more likely to adopt an anterior position in relation to the pelvis. Kosslyn (1985, cited Daniels-McGhee and Davies 1994) demonstrated that the internal manipulation of images operates similarly to that of perception and examination of normal objects, thus allowing the person to imagine from different perspectives.
According to Bruner (1966), unlike Piaget’s (1971) theories of imagery in children, iconic representation is part of learning at any age. Despite Bruner’s view that modes of representation continue through life almost intact, and that they are partially translatable into one another, the emphasis on symbolic representation as the sole means of abstract thought seems to relegate the iconic representational forms to a subsidiary position in a hierarchy of thought processes. The findings from this study suggest that both representational modes of cognition occur together.
This may be because:
In Bruner’s terms the student, being a novice in the area, is still organising the perceptual field and eventually will decrease the use of imagery.
The action orientated nature of midwifery practice, which draws heavily on biological science, encourages images of objects and people whilst at the same time requiring students to empathise and theorise.
Imagery provides the basis of creative thinking (Daniels-McGhee and Davies 1994, Buzan 1993). However Kosslyn’s (1990) work contests this view and shows that imagery mostly occurs merely as a spontaneous response to associated information and is not often used in problem solving or complex thought processes.
The work on imagery can be linked to that on metaphor (symbolic representation through language). Morgan (1993) in his book on ‘Imaginization’ talks about the way in which perspectives, assumptions, mind set, life view and frames of reference lead people into seeing things in a particular way and repeating past behaviours. He advocates using images and metaphors as a way of getting people to think differently.
Even if imagery is the key only to unlocking tacit past perceptions, using a visual image can provoke associative thinking revealing values and beliefs, which can be open to challenge in the classroom (Hall and Hart 2004). Used in clinical settings it can aid recall of otherwise tacit, multi-sensory knowledge starting the process of articulation and making the tacit explicit and potentially enhancing learning.
Students appear to build up mental knowledge representations that not infrequently take the form of visual and other types of sensory images, through past and present learning experiences, some aspects of which may be implicit. The way the experience is perceived depends on values, assumptions and beliefs already held. This analysis reinforces the complex nature of learning. No one process can be separated from any other. When the resulting (explicit or tacit) knowledge is used it leads to further learning. Students’ reports indicate that imagination (using images) is one way of using knowledge from which further learning occurs. The boundaries between learning and other forms of knowledge use are unclear, for instance one could be seen to learn from decision making (see Benner 1984, Elliott 1993, Fox 1997).
Identification of the use of non verbal forms of learning by the students suggest that current models of knowledge representation which do not fully acknowledge images in adult learning, should be challenged. However, some authors do make reference to visual forms of knowledge representation (see Eraut 1994:25-58, Schön in Ortony 1993:140). Visual images can be used in two ways, either as a representation of what has been perceived or as the beginning of the imagination of something else, where the original image changes to something new. The use of a visual image can be the key to unlocking tacit past perceptions, and can provoke associative thinking revealing values and beliefs, which can be discussed in the classroom. Used in clinical settings it can aid recall of otherwise tacit, multi-sensory knowledge starting the process of articulation and making the tacit explicit and potentially enhancing learning.
In this study the use of the ‘Thinking Aloud’ technique as a data generation and collection method enabled the recognition of all aspects of learning contributing to professional education thereby challenging current notions of knowledge representation and learning in Higher Education.
* NHS Trusts provide a practice preceptor who is a qualified midwife and whose role is a practice based assessor and/or mentor.
Belenky, M.F. et al. (1986) Women's Ways of Knowing: The development of self, voice and mind. New York: Basic Books.
Bellezza, F.S. (1986) ‘Mental cues and verbal reports in learning’, The Psychology of Motivation and Learning, 20:237-273.
Benner, P. (1984) From novice to expert: excellence and power in clinical nursing practice. Menlo Park, California: Addison-Wesley Publishing Company.
Bruner, J. (1966) Toward a theory of instruction. Cambridge, Mass: Harvard University Press.
Buzan, T. and Buzan, B. (1993) The mind map book. London: BBC Books.
Cioffi, J. and Markham, R. (1997) ‘Clinical decision making by midwives: managing case complexity’, Journal of Advanced Nursing, 25: 265-272.
Daniels-McGhee, S, and Davis, G.A. (1994) ‘The imagery-creativity connection’, The Journal of Creative Behavior, 28(3): 151-176.
Elliott, J. (1993) ‘Professional education and the idea of a practical education science’, in: Reconstructing Teacher Education, edited by J. Elliott, pp.65-85. London: The Falmer Press.
Eraut, M. (1994) Developing professional knowledge and competence. London: Falmer Press.
Eraut, M. (1999) Non-Formal Learning in the Workplace – the hidden dimension of lifelong learning: a framework for the analysis and the problems it poses for researchers. Conference Paper on Work and Learning given at the Department of Continuing Education, University of Leeds.
Eraut, M. (2000) ‘Non-formal learning and tacit knowledge in professional work’, British Journal of Educational Psychology, 70: 113-136.
Ericsson, K.A. and Simon, H.A. (1984) Protocol Analysis. London: MIT Press.
Ericsson, K.A. and Simon, H.A. (1993) Protocol Analysis: Verbal Reports as Data. London: MIT Press.
Ericsson, K. A. & Simon, H. A. (1998), How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking., Mind, Culture, & Activity, vol. 5, no. 3, pp. 178-186.
Fisher, A. and Fonteyn, M. (1995) ‘An exploration of an innovative methodological approach for examining nurses' heuristic use in clinical practice’, Scholarly Inquiry for Nursing Practice, 9(3): 263-276.
Fox, C. (1997) ‘A confirmatory factor analysis of the structure of tacit knowledge in nursing’, Journal of Nursing Education, 36(10): 459-466.
Greenwald, A.G. (1968) ‘Cognitive learning, cognitive response to persuasion and attitude change’, in: Psychological Foundation of Attitudes, edited by A.G. Greenwald, T.C. Brock and T.M. Ostom, pp.147-170. New York: Academic Press.
Hall, V. and Hart, A. (2004) ‘The use of imagination in Higher Education to enable learning about disadvantaged clients’, Learning in Health and Social Care 3(4): 190-202
Kosslyn S.M. et al. (1990) ‘When is imagery used in every day life? A diary study’, Journal of Mental Imagery, 14(3, 4): 131-152.
Jeng, L.H. (1996) ‘Using verbal reports to understand cataloguing expertise: two cases’, Learning Resources and Technical Services, 40(4): 343-358.
Miles M.B. and Huberman, A.M. (1994) Qualitative Data Analysis: an expanded source book, 2nd ed. London: Sage.
Morgan, G. (1993) Imaginization: the art of creative management. London: Sage Pubs. Inc.
Nielsen, J., Clemmensen, T., Yssing, C. (2002) Getting access to what goes on in people's heads?: reflections on the think-aloud technique, ACM International Conference Proceeding Series; Vol. 31 Proceedings of the second Nordic conference on Human-computer interaction. Aarhus, Denmark. New York: ACM Press
Norris, S.P. (1990) ‘Effect of eliciting verbal reports of thinking on critical thinking test performance’, Journal of Educational Measurement, 27: 41-58.
Piaget, J. and Inhelder, B. (1971) Mental Imagery in the Child: a study of the development of imaginal representation. USA: Routledge & Kegan Paul.
Schael, J. and Dionne, J. (1991) Judges’ Agreement and Disagreement Patterns When Encoding Verbal Protocols. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, April 1991.
Schön, D. 1993. Generative metaphor: A perspective on problem-setting in social policy. In: Metaphor and thought. edited by A Ortony. 2ndedn, pp137-163. Cambridge: Cambridge University Press
This document was added to the Education-Line database on 17 March 2006