Education-line Home Page

Exploring new methodologies for researching multimodal technologically-mediated literacy practices

 Geraldine Castleton and Claire Wyatt-Smith

Paper presented at the British Educational Research Association Annual Conference, University of Glamorgan, 14-17 September 2005

Abstract: This paper reports work-in-progress on a longitudinal study (2003-2007), funded by the Australian Research Council. Conducted in 15 schools and involving approximately 750 students, the study explores the nature of high school students’ digital curricular literacies to identify the features of classrooms and schools that are associated with enhancing students’ literacy and learning capabilities as they work in online environments. The focus of the paper is on some of the conceptual and methodological issues that we have encountered in our efforts to record, code and analyse students’ multimodal, web-based product and process data (recorded using screen-capture software) as they work online. It will discuss how we are resolving these issues as well as explore the new research opportunities afforded by the technology applied in the project. By bringing together a concern for online literacies and their role in mediating the acquisition, evaluation, organisation and utilisation of knowledge, this study takes up the argument for multiple perspectives that capture sociolinguistic concerns as well as cognitive and ontological aspects of new literacies. In this way, it seeks to theorise new literacies by drawing on innovative research applications of the technologies in classroom practice.

 

This paper reports work-in-progress on a longitudinal study (2003- 2007), funded by the Australian Research Council. Conducted in a range of high schools in Queensland, Australia, the study explores the nature of students’ digital curricular literacies to identify the features of classrooms and schools that are associated with enhancing students’ literacy and learning capabilities as they work in online environments. In investigating the products and processes associated with how and how well students engage with existing knowledge and create new knowledge, the authors argue that the acts of investigating "the new literacies" (Leu et al., 2004) of the classroom call for new research literacies where ICTs afford research insights not previously available.

The focus of the paper is on some of the conceptual and methodological issues that we have encountered in our efforts to record, code and analyse students’ product and process data that includes the video record of their screen activity as they work online. It will discuss how we are resolving these issues, drawing both on our analysis of product and process data as well as explore the new research opportunities afforded by the technology applied in the project.

The large-scale study which this discussion draws on, includes 16 schools, involving one class of Year 8 students (aged 13) and one class (each class numbering approximately 25 students) of year 10 students (aged 15) in each school. The student population in Australia is characterized by diversity and, accordingly, the school sites have been selected to ensure coverage of a range of demographic variables, including cultural and linguistic backgrounds and ethnicity.

The study involves an investigation of how and how well students in the first and third year of high school use the Internet to research information, including locating, reading, comprehending and evaluating information about a given environmental problem. A central concern was how and how well the students composed an online text to convey relevant information about a problem and to present a feasible solution to a nominated audience. Accordingly, in the first year of the study, the research team of the university researchers and teachers developed a cross-curricular web-based research activity. Its focus - the impact of plastic bags on the environment and strategies/solutions to reduce that impact - was chosen because of its links to contemporary topics of study in a range of curricular domains in high schools. The activity involved students in 1) researching – searching, locating and evaluating ‘suitability for purpose’ of specific information from web-based resources identified in the task; 2) designing a multimodal digital text (in most cases a website or powerpoint presentation) for a specific audience to convey knowledge about the problem and the proposed solution/s; and 3) reflecting, that is thinking about how the processes involved in completing the task were managed. Screen tracking software (in this case Camtasia Recorder available from www.techsmith.com) was used to capture real-time digital video recordings of students’ screen moves as they undertook the research and created texts in an online environment. Camtasia Recorder also has an audio taping capacity that was used to capture talk at the computer of a sample of students who worked in pairs on their online task; this data forming a data-pool for another aspect of the study not reported on here. The screen tracking capacity provides a move-by-move, real time trail of individual student’s decisions as they read, search, locate and evaluate information, while sometimes concurrently organizing and representing ideas, moving to acts of transforming and designing. It is the software, installed on the school network and then on each student’s computer, that generates and makes available for analysis the authentic records of individual student’s digital curricular literacies. Following a pilot in a selected school the task was refined before being used for wide-scale data collection with Year 8 and Year 10 students in the second year of the study.

Limitations of existing literacy and multimodal frameworks

The project began with existing frameworks relating to literacy, multimodal environments, and to curriculum, with just the first two of these concerns taken up in this paper. One of the first challenges facing the project was the limited empirically based understandings of what new literacies look like. The published research shows that there is considerable interest in identifying and mapping different aspects of literate practice, with a number of writers pointing out the limited research about new literacies of the Internet and other ICTs. While the call for research on new literacies is pressing, what is also clear is how, at present "we lack a precise definition of what new literacies are’ (Leu et al., 2004, p. 1571).

In the main, literacy frameworks have tended to extrapolate from a conventional print base. For example, the four resources model devised by Freebody and Luke (1990, 1999) that currently informs literacy pedagogy in a number of Australian states, was originally designed to apply to the reading of print text. This conceptual framework identifies how the desired practices and capabilities of reading can be thought of as four interrelated "roles": 1) Code Breaker: the practices required to ‘crack’ the codes and systems of written and spoken language and visual images; 2) Meaning Maker: the practices required to build and construct cultural meanings from texts; 3) Text User: the practices required to use texts effectively in everyday situations, and 4) Text Analyst: the practices required to analyse, critique and evaluate texts, through to the point of values promoted by the text. Subsequently expanded to incorporate literate capabilities, the roles serve to identify the repertoire of capabilities or resources that a reader needs to access and use to make meaning of print material, extending to critical engagement. To date, however there have been no large-scale, longitudinal studies showing the usefulness of this framework for analysing the processes students engage in as they interact with online resources, including those of the Internet.

Similarly, Green’s 3D model (1988, 1999), that shares with the four resources model the notion of literacy as an assembly of social practices, and including three dimensions: operational, cultural, and critical, taken to overlap, intersect and be interdependent, was also informing insofar as it had been influenced in its development by Green’s (1988) research into the relationships between literacy and subject or content area learning and was subsequently developed in response to the increasing "technologisation" of literacy (See Bigum & Green, 1993).

In the case of multimodal environments, the projects started with the work of Corbel (1999) who similarly extrapolated from conventional print-based literacies, to identify four categories that reflect the distinctive online environment: Orientation, Interaction, Modification and Integration. Insofar as existing theoretical frameworks have been originally conceptualized in relation to reading and print, they do not automatically have relevance to new and emerging literacies on the Internet and other ICTs.

Conceptualizing new literacies in online environments

With its systematic investigation of actual practice, the interest of this study is in establishing an empirically-based understanding of what new literacies of the Internet look like, made available through the products of students’ online work. In addition it is interested in identifying and mapping different aspects of literate practice, provided through the capturing and recording of students’ working processes thereby addressing the need for theory development of multimodal technologically-mediated literacy practices.

As already discussed, the four resources model and the 3D model provided working concepts for researching the capabilities that young people need in reading, comprehending, and writing online. However, we identified that they were not, of themselves, sufficient for analysing the literate capabilities students relied on to progress through an online task, negotiating and creating multimodal forms. At this point we turned to three multiliteracies concepts of the New London Group (NLG) (1996), and the developing conception of new literacies that Leu and colleagues (2004) have begun to frame. From the work of the New London Group our study drew on the concepts of 1) design and its related concern with multimodality, 2) hybridity and 3) intertextuality, with the first of these emphasising how any semiotic activity uses "Available Designs" including linguistic, visual, audio, gestural, spatial, and multimodal resources for meaning making, to create "The Redesigned", that is a new set of meanings. The NLG members suggest that this Designing process

transforms knowledge by producing new constructions and representations of reality. Through their coengagement in Designing, people transform their relations with each other, and so transform themselves … Transformation is always a new use of old materials, a rearticulation and recombination of the given resources of Available Designs (NLG, 1996, p. 76).

Thus text construction or composing of any kind can be seen to be what Chandler-Olcott and Mahar (2003, p. 360) refer to as a "blend" of an individual creative process and a social process.

Hybridity and intertextuality are related to these processes, with hybridity defined as the articulation in new ways of "established practices and conventions" and intertextuality being "the potentially complex ways in which meanings … are constructed through relationships to other texts, discourses, genres, and modes of meaning" (NLG, 1996, p. 82). In writing about the Internet Zembykas and Vsasidas (2005) argue that its hypertext/hypermedia nature "allows users to move with unprecedented ease from document to document, accessing images, text, and sound, and to form new paths as they explore connections and co-construct knowledge" (p. 70), a point borne out by out data analysis to date. These three concepts, originally developed within the Multiliteracies pedagogical framework, have been directly applied to our investigation of student transactions with digital resources.

The work of Leu and colleagues (2004, p. 1571) has also informed our developing conception of new literacies. As they proposed:

The new literacies of the Internet and other information communication technologies include the skills, strategies, and dispositions necessary to successfully use and adapt to the rapidly changing information and communication technologies and contexts that continuously emerge in our world and influence all areas of our personal and professional lives. These new literacies allow us to use the Internet and other information communication technologies to identify important questions, locate information, critically evaluate the usefulness of that information, synthesize information to answer those questions, and then communicate the answers to others.

Our own expectations that new theoretical perspectives could emerge from the literacies we observed in-situ, as students engaged with the visual, linguistic, spatial and audio resources afforded in online environments, are aligned with this conception. This work points to how the defining and researching of new literate practices calls for a rethinking of literacy as multimodal design where, what is valued, is students’ demonstrated capabilities to combine different meaning systems and channels of communication. From this perspective, there is a need to rethink the complex of (re)presentation, to engage with the notion of transformation and transmodal design and the implications of this for learning and assessment.

Methodological challenges facing the project

The most significant challenge we have faced in the project is how to develop a principled and manageable methodology for coding students’ actions captured in the process data. Previous studies have involved coding of data relating to digital technologies. For example, working from a situated cognitive perspective, Barab, Hay and Yamagata-Lynch (2001) have proposed a coding methodology for studying the uses to which new technologies can apply to support learners in developing transferable knowledge in teaching and learning contexts. Chandler-Olcott and Mahar (2003) have also developed a coding framework, using a multiliteracies perspective to capture an adolescent girl’s use of modes in technology-mediated literacy practices. While these studies illustrate approaches to sectioning data and parsing it down to codes, a unique feature of the current study that distinguishes it from previous work is how it relies on computer-technology as a data-collection tool. With this point in mind, the main challenge was to capture and generate rich accounts of the patterns of interconnection inherent in the data, thereby engaging with the multimodality of student working, and how they move between and orient to different media such as graphic/video image, text, music and other sound effects. This involves, in part at least, what Zembylas and Vrasidas (2005, p. 73) describe as coming up with ways to analyse the transformations information communication technologies are causing in education.

According to Zembylas and Vrasidas (2005, p. 70), "the hypertext/hypermedia nature of the Internet allows users to move with unprecedented ease from document to document, accessing images, text and sound and to form new paths as they explore connections and co-construct knowledge". Fundamentally, then, our aim was to stabilize, though temporarily, actions that are inherently dynamic, tracing for example, how students engage with digital resources, from searching the Internet for information through to designing their webpages or other web-based products.

Devising a coding methodology and stages of product and process analysis

The coding methodology developed for the project is, of necessity a construct on our part, beginning with inductive analysis, and then shifting to theoretically driven moves through the process data. With our dual focus on new literacies and multimodality and following our immersion in the literature and initial analysis of the data collected during the pilot stage, the research team initially developed a provisional coding framework for the product data. Our preliminary analysis of student-produced multimodal texts showed various features of performance and wide variation in quality so we decided to put this concept at the center of our analytic concern, and set out to develop a framework for judging quality. The research team initially drafted a framework for assessing students’ performance before sharing this with our group of advisory teachers (15) who provided direct feedback and input as we used this framework to make judgments of a random sample of students’ product data.

This process of negotiation resulted in a set of criteria and standards linked to a four point scale that could be used to score students’ product data in preparation for quantitative analytic procedures. The framework incorporated three separate categories with each category further divided into subcategories: Cohesion (unifying the structure, representation, organisation of ideas, links - the internal logic); Content (working with existing knowledge to create new knowledge); and Design (creating an aesthetic, artful design) – the compositional relationship of text elements), with these operating as fully interrelated elements working together to convey meaning [Appendix 1)]. An overall judgment was made of each text, applying the agreed standards, before scores on a four-point scale were awarded for each of the sub-categories. The process of scoring the entire data set by the research team and research assistants included appropriate reliability and consistency checks and resulted in high levels of agreement, demonstrating that a consistent notion of quality had been applied. The research team was able to code and score the product data of 643 students (representing complete data sets). Outcomes from this stage of the data analysis included a suite of identifiable literate capabilities emerging from the empirical data; a tentative vocabulary (meta language) that enabled thinking about and the exploration of process data; and the emergence of a new theoretical composite that started with our initial interest in reading as enquiry but moved the team to consider issues of e-proficiency and e-credibility that will be returned to later in the paper.

Moving from product to process

Building on our analysis of the product data, and remembering our concern with literacy we turned our focus to what literate capabilities or repertoires of practice were evidenced in the students’ process data. Our unit of analysis was the students’ observable actions as they worked with curricular knowledges online and captured by Camtasia Recorder. We were interested in determining how, for example, do you ‘see’ reading online? An interpretive framework was devised to act as a shared lens so that we could keep a running record of the sequence of actions, stabilizing them for the purpose of analysis, and was developed around the notions of 1) reading as inquiry, 2) representing and designing in networked environments, and 3) managing operational aspects of networked environments.

In relation to reading as inquiry, of interest were the strategies students use to locate online information related to the problem set in the task, evaluate it for both credibility (Haas & Wearden, 2003) and usefulness, and then synthesize, drawing from various resources. Four features of students’ online reading processes became the focus of our attention:

1. retrieval of relevant information from online resources provided as part of the task;

2. initiation and use of search strategies to locate additional relevant web-based information;

3. ways of recording findings of searches;

4. ways of recording how students evaluate information, its ‘believability’ and fitness of purpose.

In general terms, the representing and designing codes focus on the processes students use as they make decisions about what, and how to represent and communicate their new knowledge. Theoretically, the codes draw on the multiliteracies notions of the NLG outlined above, namely design (including multimodality), hybridity and intertextuality. The codes therefore address 1) the typography used in designing multimodal digital texts, including the articulation in new ways of "established practices and conventions" (NLG, 1996, p. 82, 2) evidence of working with several modes, including the actions students take in operating in a single mode and in bringing together written and visual language, and other audio and visual effects; and 3) evidence of intertextuality operating both as linking within the site and to external sources (NLG, 1996, p. 82).

The third element of the framework for coding process data, managing networked environments, seeks to address how students control the procedural aspects of working online relevant to the task. For example, what attention did students give to the task requirements provided through a website checklist and to the assessment criteria, also provided through a website link. Of further interest was students’ capacity to work on different online activities concurrently.

The concept map and evaluation matrix, included in the task to assist students to make their thinking and decision-making visible, afforded the researchers some insights into their higher-level information processing strategies and provided a framework for the mapping of the process data. The framework that evolved incorporates the three key categories described above, with each one broken down into sub-categories of observable acts, and the time of text entry, scored and captured in a matrix for quantitative analysis [Appendix 2].

The three elements of the framework are taken to be interdependent, all being of equal importance in exploring and coming to understand the transactional and transformational relationships between literacy education, technology and curricular learning in schooling. The point is that currently research has little to say about how and how well students access and use knowledge on the Internet – reading to determine not only relevance but also credibility. Further, little is known about how students carry forward what they have learnt, working multimodally and by combining different semiotic systems to construct new texts.

Some insights from analysis of product and process data

At this stage of the project the coding and scoring of product data is complete and similar work on the process data will be completed by the end of this month (September 2005). Analysis conducted to date already provides a clear picture of the widely divergent repertoires of practice students display when working online, lending support to Chen and McGrath’s (2003) observation that "students were found to have diverse knowledge construction processes and considerable variation in their ability to use hypermedia for knowledge representation" (p. 33). Specifically, the data have brought to light a wide range of capabilities in designing, representing and managing information. Evident from the 2004 data is how some students choose to invest considerable time and effort in what are, in essence the aesthetic aspects of designing, paying considerably less attention to the curricular knowledge they convey. Also worth noting is how the technical aspects of importing and transforming visual materials are not uniformly well controlled, with few student texts showing a capacity to bring a holistic, critical perspective to how various modes converge for effect. There is a significantly small number of students (four) who attained the standard of ‘Outstanding’ for their product, with their work demonstrating their repertoire of skills across the three elements of Cohesion, Content and Design.

The knowledge and skills of students cover a considerable range, both in terms of how they access and use knowledge online and also how they assume the role of designer to convey their new knowledge multimodally. There are, for example, those students who have considerably well developed technical skills that enable them to import, modify or create images with ease, and yet the context of their texts – the curricular information that they convey – is limited. On the other hand, there are students who have demonstrated little, if any, capacity to work with visual materials, but have demonstrated abilities to locate, retrieve and use relevant content information. The format of choice for these students is not a multimodal document but a unimodal text privileging written language.

Essential literacies for the knowledge age

Emerging for this work and that of others (e.g. Kalantzis et al, 2003; Leu et al, 2004) is confirmation that the most crucial skills required for the knowledge age are those of location, interpretation, synthesis and transformation of information, often gained from complex sources, into meaningful knowledge. Essential literacies for this knowledge age include multimodal authoring skills involving a range of semiotic systems and incorporating multi-mediated critical analysis. Crucial literacy capabilities include navigation skills and investigation and exploration strategies as well as the ability to (de)construct visual and verbal images, and the capacity to evaluate how they weave together for maximum effect in multimodal environments. Our observation and analysis of students at work in online environments highlights the importance of expanding our conceptions of literacy and literacy learning to accommodate recognition of literacy as multimodal design. Technologically-mediated literacies are widely recognised to be critical in contemporary education, therefore failure to recognise these appropriately in pedagogical and assessment terms means that we are not giving due acknowledgement to the varied and ever-expanding knowledge and skills base that students in contemporary classrooms need to possess.

New ways of researching new literacies

A further outcome of the present study is confirmation of the new research opportunities afforded by the technology applied in the project. Through the use of computers and the screen-capture software Camtasia Recorder we are given a window into the integration of action with output as students move backwards and forwards online, engaging in transmodal operations involving reading as inquiry, representing, designing and managing information, leading to knowledge creation (Castleton & Wyatt-Smith, 2005 forthcoming). The technology we have employed enables us to maintain the integrity of students’ work. The software not only allows the capturing of students’ products, enabling thick descriptions of student outcomes but it also provides us with the ability to ‘backtrack’ again and again, the process students have undertaken to arrive at that product, resulting in a possible set of considerations that bring product and process together. We have established a tripartite lens for looking at quality in students’ transmodal work while making process data manageable for analysis of their online literate capabilities. We welcome the opportunity this project has provided us to contribute to new methodologies for researching new literacies.

Lessons emerging from the study to date and ways forward

Equally important for us has been the identification of what we see to be limitations of our work to date, namely that a significant dimension of new literacies is currently not captured within the frameworks, that will become focal to our work as the project advances. Our analysis of both the product and process data, highlighting the wide variation in students’ capabilities in multimodal environments has brought the issue of human agency into sharp focus and consequently made us very aware of what may be called e-proficiency (Wyatt-Smith & Kimber, 2005). This term is meant to encompass more than just the possession of a set of technical multi-modal skills, but also to incorporated "nuanced" (Wyatt-Smith & Kimber, 2005, p. 41) ways of working with "design elements, knowledges, and software, as well as understandings of purposes and audiences, that taken collectively, open rich possibilities for self expression". In arguing for the centrality of human agency in multimodal text production, Wyatt-Smith and Kimber (2005, p. 41) note that "in a culture of innovation, it is these ways of working that allow us to revisit more traditional notions of imagination and creativity, thereby extending the domain of what counts as knowledge."

In progressing the work of the project and collaborating with one of the teachers from our advisory group, Wyatt-Smith has explored the definition and application of this concept in some depth, using it as an over-arching criterion in assessing students’ work. Wyatt-Smith and Kimber (2005, p. 26) take the term e-proficiency to "apply to the capabilities and the repertoire of practices that students need to participate in Internet discourse (email, chats, newsgroups etc) as well as in online environments" and encompasses knowledge of available software options and the capacity to match intended purposes and audiences. In addition, they argue that the term encapsulates what Bruce (2000) and other have identified as the credibility challenges inherent in working in online – a form of net-savviness (Wyatt-Smith & Kimber, 2005) that extends to assessing the credibility, relevance and currency of sources and located information, as well as understanding the nature of accessed sites. Martin (2004) draws on the uses of search engines as mediators in the process of locating knowledge to argue the point that the Internet and other forms of information communication technologies have changed conceptions of knowledge as well as the ways in which we acquire knowledge.

Our work to date, and particularly our reflections on the outcomes of that work lead us to want to separate out the capabilities of e-proficiency from another term now found in that literature, namely e-credibility. According to Haas and Werden (2003, p. 170) "e-credibility entails deliberately and critically discerning the qualities of trustworthiness, accuracy, completeness, and timeliness that enter a sense of "believability’." They further argue that judgment are not judgments about these concepts but that they are primarily judgments made in the context of relationship, as much about building relationships among people as it is about information (Haas & Werden, 2003, p. 181). This notion can be linked to Barbules’ (2001) contention that what he calls ‘web paradoxes’ are not about information but about ethics, that judgments in networked environments constitute communities of obligation and commitment. He argues that the social dimension of this work always entails elements of judgment and value.

Bearne (2003) also takes up the issue of criticality in what she describes as ‘shifts’ in how we use the word ‘literacy’ and the implications of these shifts for classroom work with texts. She argues for a stronger shift towards a more critical praxis and aligns this with an "approach which takes account of the rhetoric or representation – designed, visual, verbal or multimodal, - [as]offer[ing] a means of sharing common terminology about text structure and cohesion whilst recognizing the different affordances of modes and media [that can result] in … the kind of language about text making which can inform – and subvert or at least amend" Bearne (2003, p. 102), existing pedagogic and assessment practices that she believes do not do justice to children’s situated text experiences gained from their everyday interactions with the world of communications. It is to these key concepts that we are currently directing our attention as we work with our advisory group of teachers in planning the task to be employed in our data collection during 2006 where we return to the same schools, this time focusing on the students who will be in Years 10 and 12.

Further, as yet unresolved decisions for the team also include how we qualitatively interrogate our extensive data pool. We are currently working with questions such as "What are the distinctive features of the students’ work judged as ‘Outstanding’"; "Do these students all work in the same way?" "What common processes are evident?" Our direction in this regard will be guided, at least in part, by the outcomes of the structural equation modelling (SEM) currently being undertaken on both the product and process data. These results will no doubt provide us with an empirically-warranted, contextually relevant and educationally informative basis for further investigation that we anticipate will have messages for policy as well as for classroom practice.

Concluding remarks

One of the original purposes of this project was to address the argument of Nixon (2003, p. 409) and others of the need for longitudinal studies and rich school-based case studies that provide opportunities for the "systematic observation" of new practices associated with computer use. As we approach the end of the third year of the project, we believe that this project will make a significant contribution to the educational research landscape in this regard. It will also contribute to another of Nixon’s pleas, namely for studies that exemplify the affordance technology provides for new ways to represent and to problematise our research (Nixon, 2003 p.411).

Our ultimate goal is that this work contributes to a better understanding of how systems and schools prepare students to work within "a complex and diverse society in which knowledge has become the engine of national development and self-fulfilment, [requiring] a much more multifaceted approach to tracking and reporting the educational achievements of individuals and educational institutions" (Kalantzis, Cope and Harvey, 2003, p. 16).

Resources

Barab, S., Hay, K. and Yamagata-Lynch, L. (2001). Constructing networks of action-relevant episodes: An in situ research methodology. The Journal of the Learning Sciences, 10 (1&2), 63–11.

Bearne, E. (2003). Rethinking literacy: communication, representation and text. Reading Literacy and Language. November.

Bigum, C. and Green, B. (1993). Technologizing literacy: or interrupting the dream of reason. In A. Luke and P. Gilbert, (Eds.), Literacy in contexts: Australian perspectives and issues. St. Leonards, NSW: Allen & Unwin.

Bruce, B. (2000) Credibility of the web: Why we need dialectical reading, Journal of Philosophy of Education, 34(1), 97-109.

Burbules, N. (2001) Paradoxes of the Web: The Ethical Dimensions of Credibility, Library Trends, 49(30, 441-453.

Castleton, G. & Wyatt-Smith, C. (2005) Investigating digital literacies: resolving dilemmas of researching multimodal technologically-mediated literacy practices. In B. Maloch, J. Hoffman, D. Schallert, C. Fairbanks & J. Worthy, (Eds.), Fifty-fourth Yearbook of the National Reading Conference. Chicago: National Reading Conference.

Chandler-Olcott, K. and Mahar, D. (2003). "Tech-savviness" meets multiliteracies: Exploring adolescent girls’ technology-mediated literacy practices. Reading Research Quarterly, 38 ( 3), 356–385.

Chen, P. and McGrath, D. (2003). Knowledge Construction and Knowledge Representation in High School Students’ Design of Hypermedia Documents. Journal of Educational Multimedia and Hypermedia, 12(1), 36–61.

Corbel, C. (1999). Computer literacies: Working effectively with electronic texts [Office 97 version]. Sydney: National Centre for English Language Teaching and Research, Macquarie University.

Freebody. P. and Luke, A. (1990). "Literacies" programs: Debates and demands in cultural context. Prospect, 5(3), 7–16.

Green, B. (1988). Subject-specific literacy and school learning: A focus on writing. Australian Journal of Education, 32(2), 156–179.

Green, B. (1999). The new literacy challenge? Literacy learning: Secondary thoughts, 7(1), 36–46.

Haas, C. and Wearden, S. (2003). E-Credibility: building common ground in web environments. L1 - Educational Studies in Language and Literature 3, 169-184.

Kalantzis, M. Cope, B. & Harvey, A. (2003) Assessing Multiliteracies and the New Basics, Assesment in Education: principles policy and practice 10(1), 15-26.

Leu, D., Jr., Kinzer, C., Coiro, J. and Cammack, C. (2004). Towards a theory of new literacies emerging from the Internet and other information and communication technologies. In R.B. Ruddell and N. Unrau (Eds.), Theoretical models and processes of reading (5th ed.). Newark, DE: International Reading Association, 1570–1603.

Luke, A. and Freebody, P. (1999). Further notes on the four resources model. Practically Primary, 4(2), 5-8.

Luke, A., Freebody, P. and Land, R. (2000). Literate futures: Review of literacy education. Brisbane: Education Queensland.

Martin, A. (2004). New Literacies: Changing Knowledge and Classroom Learning, Book Review. Journal of eLiteracy, 1, 61-65.

New London Group (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66, 60–92.

Nixon, H. (2003) New research literacies for contemporary research into literacy and new media? Reading Research Quarterly, 38(3), 407-413.

Wyatt-Smith, C. & Kimber, K. (2005) Valuing and evaluating student-generated online multimodal texts: rethinking what counts, English in Education, 39(2), pp 22 – 43.

Zembylas, M. and Vrasidas, C. (2005). Globalization, information and communication technologies and the prospect of a global village: promises of inclusion or electronic colonization. Journal of Curriculum Studies, 37(1), 68-83.

APPENDIX 2

Coding Framework for Process Data

Initial orientation to process data (scrubbing through each file quickly)

A. Students’ relative emphases on completion of interim and final products as determined by the time of their initial text entry:

Concept map

Decision making matrix

Multimodal text

1 in the first 15 minutes of recording

1 in the first 15 minutes of recording

1 in the first 15 minutes of recording

2 in the first 30 minutes of recording

2 in the first 30 minutes of recording

2 in the first 30 minutes of recording

3 between 30 minutes and 2 hours of recording time

3 between 30 minutes and 2 hours of recording time

3 between 30 minutes and 2 hours of recording time

4 after 2 hours of recording time

4 after 2 hours of recording time

4 after 2 hours of recording time

9 initial text entry not available

9 initial text entry not available

9 initial text entry not available

Focusing on reading/viewing for completing the interim products (concept map and decision making matrix)

B. Searching for information as students complete the interim products:

1 student relies on prior knowledge (no evidence of looking at resources)

2 student reads/views 1 given resource

3 student reads/views more than 1 given resource

4 student initiates a search for information in addition to that provided with the task

5 student ignores given resources, instead initiates a search for information

9 unable to determine

C. Cognitive decision-making – how does student record the information they find in completing the concept map? (Student should enter text in at least 4 nodes)

1 student makes limited use of sources of information (follows on from 1 in B)

2 student mainly ‘copies and pastes’

3 student mainly paraphrases information from given and/or found sources into their own words (may be written in note form or in complete sentences)

9 insufficient evidence to code

D Cognitive decision-making – how does student record/transform the strategies they find in completing the matrix? (Student should enter at least two strategies)

1 student makes limited use of sources of information (follows on from 1 in B)

2 student mainly ‘copies and pastes’

3 student mainly paraphrases information from given and/or found sources into their own words (may be written in note form or in complete sentences)

4 student mainly synthesises information from given or found sources

9 insufficient evidence to code

E Cognitive decision-making – how does student evaluate the strategies they find in completing the matrix? (Student should enter text in at least two rows)

1 student makes limited use of sources of information (follows on from 1 in B)

2  student mainly ‘copies and pastes’

3  student mainly paraphrases information from given and/or found sources into their own words (may be written in note form or in complete sentences)

4 student mainly synthesises information from given or found sources

9 insufficient evidence to code

Focusing on writing/designing the multimodal text:

F Selection of design template/theme

1 student does not draw on given template/theme

2 student selects from a range of available templates/themes and uses without any modifications

3 student selects from a range of available templates/themes and incorporates modifications

9 insufficient evidence to code

G Looking for additional material to build on interim products once multimodal text has begun

1 no additional searching

2 student searches for written information

3 student searches for visual information (images, graphic elements)

4 student searches for both written and visual information

9 insufficient evidence to code

H Use of concept map in completing the final product

1 student makes limited use of previously recorded information

2 student mainly reproduces text by copying and pasting elements into the final product

3 student mainly transforms previously recorded information

9 insufficient evidence to code

I Use of matrix in completing the final product

1 student makes limited use of previously recorded information

2 student mainly reproduces text by copying and pasting elements into the final product

3 student mainly transforms previously recorded information

9 insufficient evidence to code

J Combining modes

1 student works monomodally - constructs text in a single mode

2 student works multimodally – works in modes sequentially – inclusion of additional modes does not lead to revision of work in earlier modes

3 student works transmodally – oscillates between modes to enhance effect throughout construction of text

9 insufficient evidence to code

K Modification during text production

1 student does not make changes to written or visual elements

2 student attempts changes to written language (grammar, word choice, spelling etc.)

3 student attempts changes to visual elements (size/position of graphics, colour, texture …)

4 student attempts changes to both written language and visual elements

9 insufficient evidence to code

Managing the task

L Students attention to task requirements as determined by whether they view Website Checklist

1 student does not view website checklist

2 student checks the website checklist

9 unable to determine

M Students attention to task requirements as determined by whether they view Assessment Criteria

1 student does not view assessment criteria

2 student checks the assessment criteria

9 unable to determine

N Attention to online task

1 students do not engage with task – worksheets and multimodal text not attempted

2 students work on the given task online concurrently with other online activities

3 student’s online work remains within the given task

9 unable to determine

This document was added to the Education-Line database on 09 December 2005