Education-line Home Page

Design and evaluation of an action research toolkit for teacher professional development

Steven Coombs
School of Education, Bath Spa University College

Rosie Penny
School of Education, St. Mary's College, University of Surrey

Ian Smith
Faculty of Education, University of Sydney, Australia

Paper presented at the British Educational Research Association Annual Conference, Heriot-Watt University, Edinburgh,
11-13 September 2003


There is no doubt that the action research mode of enquiry-based learning is one of the most important approaches that validates classroom research and on-the-job professional enquiry of teachers' in schools and colleges. Indeed, action enquiry is supported by the UK DfES through applied research schemes such as the Best Practice Research Scholarship. However, the action research qualitative process is generally considered not to be as systematic compared to more traditional positivist experimental methodologies. This paper explores the need for a more systematic and transparent experimental approach to support teachers' carrying out classroom-based enquiries and proposes the rationale of a professional development toolkit to help scaffold and enrich systematic enquiry. The systems thinking conversational science paradigm of Self-organised-Learning (S-o-L) has been applied to an action research paradigm from which the toolkit has been designed. Case study evidences drawn from teacher professional development prototypes will be illustrated from which a generic pedagogical design protocol has emerged.


This paper considers the case for introducing systems thinking tools to support action research fieldwork. The systems thinking pedagogical framework is derived from the conversational science paradigm of Self-organised Learning (S-o-L) proposed by Harri-Augstein and Thomas (1985, 1991). This pedagogical framework was developed into a project management action research toolkit by Coombs (1995) and further applied by Perry (1998), Lee (2001) and Ravindran (Coombs & Ravindran, 2003) for their educational action research projects. Since then, the tools have been further refined for critical thinking use by students at Bath Spa University College, towards applied course work that requires educational research investigation and enquiry within the workplaces of schools and colleges. Diverse examples of these critical thinking scaffolds (Coombs, 2000) were developed so as to assist action researchers involved in work-based project management, and these will be shared as toolkit exemplars within this paper. The deeper philosophical basis and pedagogical rationale of critical thinking scaffolds, and the linked concept of a knowledge elicitation system (KES) (Coombs 1995, 2000 & 2001) are explored in the associated BERA2003 paper "Improving personal learning through critical thinking scaffolds". This paper, however, overviews the systems thinking conversational paradigm of self-organised learning and then proposes an S-o-L pedagogical framework for action research. This framework is then related to the pedagogy of critical thinking scaffolds that is then used as a systems thinking S-o-L design template from which to construct an action research toolkit that supports the qualitative project management tasks of action researchers within their real-life social learning environment.

Understanding Self-organised Learning and the conversational science paradigm

This paper considers the benefits of reflective practice for the professional development action researcher from the conversational learning paradigm perspective of Laurie Thomas and Sheila Harri-Augstein (1985). They define human learning as "...the construction and reconstruction, exchange and negotiation of significant, relevant and viable meanings" (p.2). Their theory of the self-organised learner (S-o-L) is a personal constructivist theory (Kelly, 1955) of human learning that considers the design and use of reflective tools and processes, which lead to an improved repertoire of inner-reflexive skills and builds upon Kelly's (1955 and Bannister, 1981) Personal Construct Theory. Steven Coombs and Ian Smith (1998) explored the person-based relationships between reflection and reflexivity. In their article "Designing a Conversational Learning Environment" they identified a learning theory based on "conversational constructivism" that provided a new insight into understanding the relationship between thinking and learning. They summarised Harri-Augstein & Thomas' S-o-L conversational paradigm in terms of three core principles:

1. real personal learning depends on self-assessment and reflective evaluation through the construction of internal referents;

2. the S-o-L practice depends on the ability of the learner to self-monitor and control the learning process whilst developing appropriate models of understanding; and,

3. shared meaning is negotiated conversationally from social networks. Such social networks can be understood as conversational learning environments that construct their own viability and validity, resulting in a capacity for creative and flexible thinking. (p. 7)

A systems thinking S-o-L framework for Action Research

Action research is underpinned by a systems-thinking process, which Elliot (1991) describes as an action-reflection professional development cycle that determines the qualitative change-management process of how the teacher might experiment with their curriculum. Thus, action research is deemed to have a critical thinking process that puts the action researcher into a reflection upon practice cycle that is supported as a form of systematic self-enquiry by McMahon (1999) from which various support resources are suggested by McKernon (1996). A reflective praxis represents the philosophical assumption that forms the foundation methodology of action research for professional learning, and is supported by leading critical thinkers such as Wildman (1995) and Schön (1987). This action research methodology provides a work-based experimental rationale that generally seeks qualitative evidences that demonstrate an improvement in one's own professional development and working situation, i.e. a social manifesto (Coombs, 1995) objective as opposed to the more traditional positivist experimental paradigm that seeks generalisable laws via hypothesis testing (Coombs & Smith, 2003). Indeed, Coombs and Smith (2003) recently underlined the social learning benefits of participatory action research by teachers' operating within their own classrooms as a new paradigm interpretation and validation of the Hawthorne Effect, that has long been used as a criticism of a 'researcher' operating within their own social domain. However, given the rationale that action research seeks only localised (not generalised) improvements with supporting evidences supporting such change-management (Lomax, 1989) that also result in change-culture within a learning organisational setting, then it appears to be relevant that a consistent set of generic experientially content-free thinking tools could be recruited to assist the action researcher to collect meaningful qualitative data as part of the project management field enquiry process.

This idea that the action researcher is a self-enquiring experiential learner using critical thinking tools to aid personal reflection is further underpinned by Harri-Augstein's and Thomas' (1991) notion of the self-organised learner, whereby an individual's learning capabilities can be enhanced through technology assistants that they refer to as an Intelligent Learning System. Coombs (1995), in his PhD thesis; "Design and Conversational Evaluation of an IT Learning Environment based on Self-Organised Learning", further elaborates this notion of an Intelligent Learning System in terms of a Knowledge Elicitation System. Whereby, information technology-assisted learning is considered in terms of its interactive reflective learning capability with an individual learner that is socially extended to both the action researcher and all the engaged field participants (Coombs, 2000a). It is therefore understood that the quality of learner-learning that employs a critical thinking scaffold process can be explained in terms of the learner being able to systematically manage their own elicitation in the form of self-organised reflective construing experiences. From this pedagogical perspective, knowledge is considered as being relative to the user¾ as learner¾ via focused technology-assisted reflections, construed and elicited by the person in the form of meaningful inner learning conversations. This form of internal knowledge construction from self-managed reflective experiences represents a new learning theory that Coombs and Smith (1998) refer to as "conversational constructivism". Conversational learning methods by the action researcher to support action research is also illuminated by Jones (1989), but is only explained as systematic thinking enquiry by Harri-Augstein & Thomas (1991) and Coombs and Smith (1998) who maintain that:

"...[critical thinking] tools used for activities which encourage, stimulate and focus meaningful reflection can be viewed as knowledge modeling devices that facilitate learning in a social context. This particular paradigm empowers learner control of the learning process using appropriate conversational tools to achieve one's learning goals and provides a valid learning theory that explains the motivational role and educational value of a conversational learning environment." (p.27).

Given this understanding of how reflective technology learning systems (Coombs 2000a) may impact upon the personal learning capability of the learner¾ coupled with the demands of higher education (EU DGXII, 1995 & Dearing, 1997), such as developing the professional learners higher-order reflective skills¾ it can easily be seen how appropriately designed learning technologies could bring considerable benefits to action research professionals located within normal community working environments (Rodd & Coombs, 1997 and Coombs & Rodd, 2001a). Indeed, the personal involvement of the teacher/trainer in developing his/her curriculum through an action research project has long been supported by educational critical thinkers like Stenhouse (1975) and Elliot (1991) who support the notion of teacher as experimenter of his or her curriculum. Action research evaluation techniques often involve the keeping of a reflective learning biography of main project events as a means of project management review and self-evaluation of the important lessons learnt as they were experienced on-the-job. This paper considers both the philosophical framework and conversational tools to successfully engage in such action research activities.

Information and Communication Technology (ICT) software systems also offer a powerful range of reflective learning tools to support the action researcher and can be both understood and evaluated for effectiveness against the pedagogical design criteria of a knowledge elicitation system that Harri-Augstein & Thomas (1991) have proposed as the following self-organised thinking steps in order to achieve higher-order critical reflection and knowledge elicitation:

  1. elicitation of items of meaning;
  2. sorting of their relationships, and
  3. display of the final pattern.

Coombs (1995) maintains that all well-designed and effective Knowledge Elicitation Systems follow the above 3-step reflective learning design criteria and thus operate as a critical thinking scaffold (Coombs, 2000).

Thus, ICT reflective-tools operate meaningfully as KES critical thinking scaffolds and can therefore be designed to assist action research participants carry out a small-scale action research project from within their own social and working environment (Coombs, 1997 & 1997a). ICT action research project management techniques included the generic use of: -

  1. the Internet to research contemporary background information and professional literature of a participant's subject/professional development field;
  2. email as an asynchronous critical thinking medium to share research questions and concerns with project supervisors and other team members;
  3. spreadsheets for quantitative data analysis and graphical presentation;
  4. wordprocessing facilities to keep a computerized reflective log/account of key project events and submit the final assessment dissertation; and,
  5. bespoke conversational critical thinking scaffolds (adopting the three-step reflective learning design criteria) using MS Word© and other standardised ICT software interfaces - see the later exhibits.

Looking at the real-life social context of action research, and its relationship to the social dimensions of a self-organised conversational learning environment, one can see that action research operates within a team-based social learning environment that integrates the individual task-based activity with group learning collaboration and enquiry - as illustrated in the action research S-o-L pedagogical framework given in Figure 1 below.

Figure 1: A systems-thinking based action research S-o-L Environment.

Figure 1 shows how action researchers operate as learners within a self-organised conversational learning environment that provide systems/resources/technologies with the flexibility and choice of enabling the action researcher to migrate from a situated open learning environment, as an autonomous learner, to that of a collaborative learning team-based environment and vice-versa. The same pedagogical flexibility applies to opportunities to transfer between a virtual and an on-site physical learning environment, or vice-versa, and between a flexible learning environment to a more structured one. This systems-based flexibility underpins the pedagogical design framework of an S-o-L action research environment, thereby creating enriched knowledge-building opportunities via multiple forms of personal and social interactions within a situated open learning environment, i.e. the action research project field.

This S-O-L action research paradigm and systems thinking framework provides for a combined collaborative and individualised learning environment. Collaborative learning vis-à-vis self-organised learning may seem paradoxical, but is in line with the S-O-L interpretation of social constructivism via conversational social networks. This is because learning conversations must operate in meaningful real-life contexts, such as social networks, and that these real-life social situations generate valid and motivational group learning opportunities. In a sense, the collaborative S-o-L environment appears to be a pedagogical paradox in that collaborative learning is actually about individual learning that operates within and across a defined social domain context, hence the notion of an S-o-L conversational group network.

Examples of a Project Management Toolkit for Action Research

The following exhibits illustrate some of the S-o-L critical thinking scaffolds that operate as project management tools/templates for teachers as action researchers within their schools as learning organisations.

Exhibit 1: P-S-O-R Conversational Template for Action Research Project Management

Exhibit 2: P-S-O-R Conversational Template for Action Research Project Management

Exhibit 3: P-S-O-R analysis of how CILL's S-O-L curriculum is delivered

Exhibit 4: Example of a Project Management Spidergram Template to support educational action research

Exhibit 5: Appendix 1 The Personal Learning Contract Conversational Template, CSHL©

Exhibit 6: Interview Design Template

The S-o-L templates illustrated in exhibits 1 to 3 show the action research application of the Purpose-Strategy-Outcome-Review (P-S-O-R) procedure as described by Harri-Augstein and Thomas (1991) and designed as a generic systems thinking project management template by Coombs (1995). Exhibits 1 and 2 illustrate how the P-S-O-R tool elaborated Lee's (2001) educational action research project " Evaluating critical thinking pedagogy to support primary school project work through an action research approach" for her main professional development Masters degree. Exhibit 3, however, shows how the same generic tool was used to elaborate Ravindran's (Coombs & Ravindran, 2003) action research project for developing an S-o-L paradigm within her Centre for Independent Language Learning (CILL) unit within Temasek Polytechnic in Singapore. A more elaborate systems thinking template is the S-o-L Personal Learning Contract (PLC) (Harri-Augstein & Thomas, 2001), which includes the P-S-O-R routine within a three-phase action learning event-time pedagogy that Coombs (1995) designed as an action research project management evaluation tool:

Phase 1: reflective planning - the use of P-S-O-R to design the action research project, i.e. a pre-emptive reflective learning project management phase.

Phase 2: on-the-job reflection - the use of P-S-O-R to monitor and evaluate the action research project within the field and during the project cycle, i.e. reflective action learning project management phase.

Phase 3: reflective closure - the use of P-S-O-R as a post-qualitative action research tool to analyse and evaluate the completed action research project tasks, i.e. final reflective evaluation of the action research project.

Exhibit 5 illustrates a completed PLC template that provided an overall evaluation of Lee's (2001) action research professional development Masters project.

Other action research project management tools include Coombs' (1995) Spidergram, illustrated in exhibit 4, showing how a critical thinking scaffold can be developed into a "loose thinking" but focussed tool for brainstorming and construing related events. In the case of exhibit 4 the Spidergram has been adapted to provide the fixed focussed question "What are the evidences that suggest the emergent themes of the findings?" This question was posed for students working in small teams conducting an educational action enquiry project at Bath Spa University College as part of their Education Studies undergraduate degree programme. The Spidergram was used as a project management tool at various stages throughout the student project cycle. Exhibit 4 was used as a critical thinking tool to help each student team to elicit and evaluate their research evidence findings. Exhibit 6 shows another project management tool used by the same student teams earlier in the educational project cycle to help them elicit and design their field interview questions.

Conclusions and future projects

This paper has briefly overviewed the pedagogical concepts of self-organised learning as a pedagogical framework to support action research via critical thinking tools operating as project management scaffolds. The pedagogical design criteria for these action research 'tools' has been simply explained and applied to the generic S-o-L procedures of P-S-O-R and PLC from which working exhibits have been trialled in a range of diverse educational professional development projects.

The systems thinking approach towards project management is widely supported by critical thinkers such as Checkland (1993) as well as Harri-Augstein and Thomas (1985, 1991) and Coombs (1995). Integrating systems thinking project management with action research and the S-o-L paradigm was experimented with by Coombs (1995) in his PhD at the Centre for Study into Human Learning (CSHL) at Brunel University, from which the S-o-L action research framework was first identified. The same work identified the concept of a knowledge elicitation system (KES) as a design template for critical thinking, which was further explained (Coombs, 2000), more fully, as a critical thinking scaffold to enable learner-learning. The idea that technology can support action research reflection and evaluation was also discussed by Coombs (1997, 1997a) and linked to European developments in adopting telematics as a reflective learning environment. The six exhibits produced for this paper show a small range of the conversational templates designed to support action research project management, but all operate in common as critical thinking scaffolds within the S-o-L paradigm. Future continuing professional development research across several university centres will develop and trial more content-free generic templates with the aim of producing a public domain comprehensive toolkit for en masse application and validation of any work-based action research project.


Bannister, D. (1981). Personal construct theory and research method in Reason, P. and Rowan, J. (Eds.), in Human inquiry: A sourcebook of new paradigm research pp. 191-200. Chichester, UK: John Wiley and Sons.

Checkland, P. (1993). Systems thinking, systems practice, New York: John Wiley & Sons.

Coombs, S. (1995). Design and Conversational Evaluation of an IT learning environment based on Self-Organised Learning. PhD thesis. London: Brunel University.

Coombs, S. (1997). Applied Telematics for Interdisciplinary Action Research, in Research across the Disciplines, conference paper published in the proceedings of the Singapore Educational Research Association 602-609 - Singapore: National Institute of Education.

Coombs, S. (1997a). Towards a Telematic-Assisted Learning Environment. Singapore: Paper published in the proceedings of the On-Line Educa Asia: International Conference on Technology Supported Learning held in Singapore, September 1997.

Coombs, S. & Smith, I. (1998). Designing a self-organized conversational learning environment, in Educational Technology, 38(3), 17-28.

Coombs, S. (2000) The psychology of user-friendliness: The use of Information Technology as a reflective learning medium. Korean Journal of Thinking and Problem Solving. 10(2), 19-31. Korea: Keimyung University.

Coombs, S. J. (2000a). IT tools for reflective learning. In Williams, M. D. (Ed.), Integrating technology into teaching and learning - Concepts and applications. (2nd ed.). Singapore: Prentice Hall.

Coombs S. (2001). Knowledge Elicitation Systems. Paper presented on behalf of the Association for the Advancement of Computers in Education at the EdMedia 2001 conference in Tampere, Finland, June 25-30, 2001.

Coombs, S. and Rodd, J. (2001a) Using the Internet to deliver higher education: A cautionary tale about achieving good practice. Computers in the Schools (invited article for special edition), 17(3/4), 67-90.

Coombs, S. J. & Smith, I. D. (2003). The Hawthorne effect: Is it a help or hindrance in social science research? Change: Transformations in Education, 6(1), 97-111.

Coombs, S. J., and Ravindran, R. (2003). Techno-pedagogy and the Conversational Learning Paradigm: Delivering the curriculum at the Centre for Individual Language Learning. Asia Pacific Journal of Education, 23(2). Singapore: National Institute of Education (2003, in press).

Dearing, R. (1997). National Committee of Inquiry into Higher Education - The Dearing Report. London: HMSO. (

Elliot, J. (1991). Action research for educational change. Buckingham, UK: Open University Press.

European Union DGXII (1995) Teaching and Learning - Towards the Learning

Society. European Union White Paper on Education and Training: (

Harri-Augstein, E. & Thomas, L. (1985). Self-organized learning: Foundations of a conversational science for psychology, London: Routledge & Kegan Paul.

Harri-Augstein, E. & Thomas, L. (1991). Learning conversations: The self-organized learning way to personal and organizational growth, London: Routledge & Kegan Paul.

Jones, B. (1989). In conversation with myself; becoming an action researcher in Lomax, P. ed. (1989).The management of change: Increasing school effectiveness and facilitating staff development through action research. Clevedon, Avon, England: Multilingual Matters.

Kelly, G. (1955). The psychology of personal constructs, Volumes 1 & 2. New York: Norton.

Lee, V. (2001). Evaluating critical thinking pedagogy to support primary school project work using an action research approach. Unpublished MEd. thesis, Nanyang Technological University, National Institute of Education, Singapore.

Lomax, P. ed. (1989).The management of change: Increasing school effectiveness and facilitating staff development through action research. Clevedon, Avon, England: Multilingual Matters.

McKernan, J. (1996). Curriculum action research: A handbook of methods and resources for the reflective practitioner. (2nd ed.) London: Kogan Page.

McMahon, T. (1999). Is reflective practice synonymous with action research? Educational action research, 7(1), 163-168.

Perry, J. (1998). An evaluation of the practitioner-team ethic towards developing the concept of the learning organization. Unpublished master's thesis, University of Plymouth, U.K.

Rodd, J. and Coombs, S. (1997) Development and evaluation of an Experimental IMP web site as part of a flexible and distance education learning policy for the University of Plymouth, paper no. 182 - CAL'97, University of Exeter, March 1997 - CD-ROM proceedings (

Schön, D. (1987). Educating the reflective practitioner: Towards a new design for teaching and learning in the professions, San Francisco: Jossey-Bass.

Stenhouse, L. (1975). An Introduction to Curriculum Research & Development. London: Heinemann.

Wildman, P. (1995). Research by looking backwards: Reflective praxis as an action research methodology. Pinchen, S. and Passfield, R. eds. (1995). Moving on: Creative applications of action learning and action research. (pp.172-191). Queensland: Action Learning, Action Research and Process Management (ALARPM).


This document was added to the Education-line database on 24 September 2003