Education-line Home Page

"SO TELL ME ABOUT IT!": A QUALITATIVE INVESTIGATION OF INTERNET SEARCH STRATEGIES.

Greg Hale and Nicola Moss

Department of Information Studies, University of Sheffield, Western Bank, Sheffield, S10 2TN, UK.

E-mail: g.hale@sheffield.ac.uk, n.c.moss@sheffield.ac.uk

Paper Presented at the European Conference on Educational Research, Lahti, Finland 22 - 25 September 1999

ABSTRACT

This paper presents a model illustrated by the findings from the first stage of the qualitative strand of a project investigating Internet searching behaviour [1]. The other strand is a quantitative strand (see Moss and Hale, 1999) investigating links between cognitive style and choice of Internet strategies. The project is funded by the Arts and Humanities Research Board, United Kingdom and runs from May 1999 to the end of April 2000.

The qualitative strand uses a grounded theory methodology to investigate emergent issues related to Internet search strategies, with iterative interviewing (where participants views are deepened and reflected back to them by the interviewer) used to explore internet search strategies. Five participants undertook informal interviews (while not undertaking Internet searching, i.e. out-of-system interviews) and while undertaking Internet searchers (i.e. in-system interviews) using real search problems that the participants were facing. The use of real search problems facilitated an understanding of the individual meanings and strategies that searchers bring to search episodes.

This paper describes a model for understanding Internet searching that can inform practice in teaching by showing the importance of the strategy-tactics continuum in Internet searching and by explicitly linking Internet searching to the wider contexts in which it takes place.

INTRODUCTION

The Internet Searching Project is based at the Department of Information Studies, Sheffield University. The project has both quantitative and qualitative strands. The quantitative strand is investigating links between cognitive style and choice of Internet strategy. The qualitative strand is using a grounded theory methodology to investigate emergent issues related to Internet search strategies. The project is funded by the Arts and Humanities Research Board of the United Kingdom and runs from May 1999 to the end of April 2000. Further details can be found at the project website, (open 1st October 1999, http://dis.shef.ac.uk/ahrb/default.htm).

Internet Searching and the Context-Action-Event model

E-MAIL GREG HALE FOR COPY OF THIS DIAGRAM, ENCLOSING POSTAL ADDRESS, BY CLICKING HERE

Information searching is a key area in information science research, with different models being developed to describe search patterns (see for instance the 'berry picking' model of Bates, 1989 and the behavioural 'chaining' model of Ellis et. al., 1993). In addition, investigations of motivational states that affect Internet searching has been undertaken (e.g. uncertainty, Kuhlthau, 1993) and the development of search patterns over time have been investigated ('successive search phenomena', Spink et. al., 1998[2]).

The Context-Action-Event model was developed as part of a research project to explore Internet searching in entrepreneurs (Hale, 1999 [3]). It has grown out of work on human-computer interaction, human concepts, software assessment and systems analysis [4] and the work reported here, as well as linking with earlier theoretical conceptions (see Hale and Moss, 1999). The model will be further developed in the main phase of this research (October 1999 to the end of April 2000) as well as through ongoing work on Internet searching in entrepreneurs. The aim is to use repeated investigations of groups of very different searchers to build a generic model of Internet searching.

The model in its current stage of development is presented diagrammatically in Figure One. Only the Context and Action levels are shown, describing at 'top' level (i.e. not a detailed granularity of analysis) the hazyspace of internet searching behaviour. Hazyspace is a 'real-life' explanation of human concepts, "...focused on the messiness of real-life and the indeterminancy of concepts as key features in understanding the conceptual domain of human beings" (Hale, 1998 p.185 [5]). Hazyspace highlights the fact that concepts are 'messy', often inaccurate and usually idiosyncratic. The diagram of the model shows Internet searching in the context of these assumptions. It is not therefore intended to be an exhaustive description of the phenomena, nor are the elements definable in the way that they would be using a systems analysis approach (which would arguably be incapable of handling the flexibility and haziness of human searching behaviour).

The model employs a top-down description from the necessary and preceding temporal and logical structures of searching, joined with a bottom-up description starting from the data that is collected and analysed using a grounded theory approach (Glaser and Strauss, 1967). In addition, accepted theoretical and known system elements are also included, thus making effective use of theoretical sensitivity (ibid.) and the realities of searching. Upper levels of the model can be decomposed down to lower levels or lower levels built up to upper levels. It is hoped that the lowest level, the Event level, will provide the possibility of computer modelling of search behaviours of users, using techniques from the field of expert systems.

The Context level comprises the five major contexts of searching (Social-Structural, Problem Space, Searcher, System and Search Sequence). The decomposition from the contexts of Searcher, System and Search Sequence provides the 'frame' for the next decomposition at the Action level (see Figure One). Each element in the Action level refers to an observable state, consisting of clusters of activities at the Event level (not shown in the diagram). These will be further developed as work on the Event level proceeds. At the moment, the input and output handling can be seen as operating at a lower level of interaction than the higher cognitive functions taking place in the Processing part of the model.

Internet Searching: Strategy, tactics and the Context Level

The word 'strategy' comes from the Greek strategos, meaning the art of a general (Jablonsky, 1992 p. 1). Since strategy relates to certain types of problems (in contrast to the types of problems that tactics deals with, see below) the concept is heuristically fruitful for the study of Internet searching, despite the fact that in normal conversation the two terms are often used (inaccurately) as if interchangable. Strategy is what generals do, and is composed of ends, ways and means (ibid.) at the large scale. The classic military definition of strategy as 'the calculated relationship of means to ends' (Jablonsky, 1991 p. 5) is incomplete because strategy cannot adequately be understood as a purely rational process but needs to be studied by taking into account psychological processes related to irrationality and emotion (Jablonsky, 1991). This emphasis is very salient for the Sheffield project on Internet searching.

Strategy in Internet searching therefore involves high level problem formulation and solving at the macro level. It involves metacognitive skills, the use of insight, critical thinking and creativity. There are elements of the subjective as the searcher decides what 'ends' are appropriate and should be worked towards. A typical strategic decision might therefore be what search engine to use to achieve ends that have specifically been formulated in advance. Strategy is particularly vulnerable to poor identification of aims, the effects of problem domain ignorance, and the effects of irrationality.

Tactics in Internet searching is lower level problem formulation and solving at the micro level. Tactics therefore concentrates on the application of techniques, procedures and rules. A typical tactical decision for instance, will relate to correct query formulation. Tactics are particularly vulnerable to effects of errors, slips and system ignorance.

Because of its global nature strategy operates at the Context-Action end of the Context-Action-Event model. Because of its local nature tactics operates at the Event-Action end of the model.

Following the title, the material that follows concentrates on the strategic level of Internet searching. However, both tactics and strategy are important. Strategy ensures an effective search which is strongly linked to formulated ends. But tactics is the building block for successful strategy.

METHOD

Qualitative methods are increasingly being used to investigate information seeking (e.g. Spink et. al.,1998; Yang, 1997). This project uses qualitative measures not on the basis of generalisability to a wider population but rather on the basis that individual meanings (the 'why' issues) are crucially important. It is explicitly recognised that meanings and concepts vary widely within individuals. The research does not suppress this fact but chooses rather to celebrate it, as well as seek to understand it. It is these personal meanings that this papers focuses on, illustrating themes related to strategies of Internet searching as they have emerged from the data.

Data collection

This exploratory stage of the investigation explored individual meanings related to the Internet search strategies of five participants from the Department of Information Studies. Four of the participants were female, one male. Unsuccessful efforts were made to redress this balance. Data was collected to determine the participants cognitive styles, profile Internet use and experience, and measure linguistic flexibility. Two settings were used to gather data. The first was out-of-system informal interviews, with participants views deepened and reflected back to them by the interviewer. This setting avoided system constraints or cues and provided a wide and coarse grained investigation of Internet searching as it related to participants' individual lives, information needs and information searching behaviour. The other setting was in-system interviews, where participants undertook searches using information problems they were facing, using a search engine of their choice. Iterative interviewing was used in this setting too, with 'think aloud' verbal protocols gathered from participants and reflected back and deepened by the interviewer in an iterative cycle within each search episode. Both sets of interviews were recorded by tape recorder, supplemented as necessary by field notes.

Data analysis

A grounded theory approach was used (Glaser and Strauss 1967), "...a qualitative research method that uses a systematic set of procedures to develop an inductively derived grounded theory about a phenomenon." (Strauss and Corbin, 1990 p. 24). The choice of this methodology ties in with the project commitment to the process of developing emergent theory (see Hale and Moss, 1999; Moss and Hale, 1999).

Data was transcribed and theorised, leading to the development of more specific issues to be researched, together with the formulation of 'proto-theories' to be progressively developed as the project unfolded. When no new theory emerges from the findings, theoretical saturation has been reached and the theory can be considered 'dense'. Such theory can then be explored in other contexts, as well as inform quantitative data analysis. It is hoped that the main phase of the project will provide the opportunity to reach this stage with the further investigation of key issues.

As the research process unfolded both strands of the project (the quantitative and the qualitative) have moved independently nearer each other, taking in aspects more normally associated with the 'opposite' paradigm, a bi-modal approach.

The results from this stage of the research are presented within the (top) Context level (i.e. the strategic level) of the model mentioned above. This level consists of the five major contexts of searching (Social-Structural, Problem Space, Searcher, System and Search Sequence). Within the developmental flow of the research, the Context-Action-Event model was developed after the first stage of the data collection was complete so the Social-Structural component is under theorised because participants were not asked specific questions about this area (though the transcripts do contain some relevant material). This area is therefore not used in the illustrations of the model which follow. The Social-Structural context is specifically covered in other work (Hale, 1999).

THE MODEL ILLUSTRATED

The elements of the Context level are now defined more fully, illustrated from the qualitative data Note that some minor aspects of the transcript have been changed in order to preserve participant confidentiality.

Elements of the Context Level

System knowledge and subject domain knowledge

These are conceived in the model as data stores. The overlap with the searcher is intended to illustrate the fact that the knowledge of the searcher is partial. The corners of the data stores that overlap the circle which represents the searcher could be shown in ragged lines to illustrate the fact that not only is the knowledge of the searcher incomplete, it is often inaccurate and idiosyncratic, because it is a hazyspace. These are not illustrated in the current paper.

System

'System' comprises the physical and software elements of the system. Clearly these can have a potent effect on searching behaviour. The description of 'system' as used in this paper is search engine and web page.

Most of my education is based bookwise, I don’t look at journals, the net, or anything like that, I do it from books… I just went to Altavista...Actually I think I ended up, now thinking about it, on Netscape

This searcher did not make much use of the Web generally and just went to the default settings. There was therefore (due to lack of knowledge and confidence) no significant choice of options at the strategic level.

The following searcher has an established rationale for the choice of search engines, a strategic choice.

[influences on choice of search engine] It depends what sort of search I'm doing. Definitely. If I'm looking for a very specific page that I don't whether it exists or not, say a specific organisation that I know about, then I'll use a big search engine that's got a big directory because if there is that page it's more likely to be in there because the automatic robot is indexing everything. If I know it’s a subject that there'll a lot about and therefore I want some high quality information about it because there's so much to choose from then I'll go to one of the directories so in that case I might use Yaho…And also it depends on where I want the other information from, so if it's UK based information I try and go for the ones that are more European like Euroferret whereas if its US information then I don't mind using AltaVista and HotBot.

Searcher

It is crucial to profile the user (Hale, 1997), paying particular attention to the participants' knowledge of Internet searching and knowledge of the particular search engine used. The profiling phase would normally be done using a questionnaire. In the main phase of the project specific questioning will be used that references the Context-Action-Event model. It is intended to develop more quantitative measures in the profiling.

Most of my education is based bookwise, I don’t look at journals, the net, or anything like that, I do it from books.

This searcher is, as noted above, inexperienced with the Internet and lacks confidence in its use. The searches of this searcher were simple and relatively unembedded in either the complexities of the search or the potential of the technology.

Problem Space

The concept of problem space comes from computer-simulation approaches to problem solving. Problem space has been defined as,

...the problem solver's internal representation of Initial state...Goal state...Intermediate problem states...and...Operators- the moves that are made from one state to the next (Mayer 1992, pp. 180-181, italics in the original).

As defined here this is a purely internal model. For the purposes of studying technologically embedded problem solving (as Internet searching is), it is better to work with a wider notion of problem space which includes elements outside the direct conceptual world of the user i.e. both the subjective elements related to the conceptual world of the users but also objective elements related to system. Such elements impinge on this conceptual world however, shaping and reshaping it in the active nature of the problem solving and human-computer interactions of the user, a fluid and ongoing process. That is why in this model the problem space encompasses Searcher, System and Search Sequence. The problem space also includes the type and shape of problem that the searcher faces. Such information will need to be elicited from each participant and forms a problem space unique to that individual. This can only be touched upon in this paper, because this is where tactics starts to play a large role.

Both the following extracts centre around a specific information need, related to academic work. This forms the problem space within which moves are made to try and satisfy the information need.

I needed a load of facts to produce an introduction to the dissertation, I want to do about 2-3000 words on just the basic facts on 'Aids', from history to how many deaths and so on and so on. And I didn’t actually have enough pure facts…

I'm doing an MSc in information management and my dissertation is on how the leisure sector are exploiting the web, whether they're exploiting the web and internet facilities and technologies

Search Sequence

Search Sequence is the intersection of the searcher with the system, set within the Problem Space, see diagram). Clearly search sequence is the major element at the Context level of the model, with links to an extensive literature and body of work [2].

…I started with the words...I think 'Aids' and health- I usually try put an AND in, to try, you know, narrow it down a bit...I think that's all I tried, I think I tried a few but I can’t remember what they were…What I tended to find was that 'Aids' would lead me to a 'Aids' site and I would go that- and they are quite extensive, and I would go through each of their points…I knew when they were no good and maybe go back to the main site, point and pick a few more different

This is the searcher (above) who lacks confidence and knowledge. The searching was characterised by repetitive use of the Boolean operator AND with a limited variety of keywords being reported. The keywords were not always carefully chosen with regard to the end in mind, which seemed in this case to be a break down in subject knowledge, illustrated by the fact that the searcher did a search on the word 'judo' but caused problems by not recognising judo as a martial art. The next searcher had a strategic approach to searching:

I probably would have started it off doing something like leisure sector and Internet…Just to get an idea of what's there out there…When I start off I normally do quite a broad search just to give me an idea of what's out there is out there and to tell me whether I need to narrow the search and how much by and in what way.

This searcher displays a problem solving technique at the strategic level, what might be called intelligence gathering, or surveying. A number of searchers used this technique, with a quite explicitly developed set of reasons. It may also be of course that this is merely a subtle use of cognitive-behavioural actions that delay the onset of hard work in dealing with the search. A 'try-it and-see' approach may also be linked to this.

CONCLUSION AND FURTHER RESEARCH

The strategy-tactics continuum is clearly a heuristically fruitful concept. It will be necessary however to operationalise what elements within each continuum are regarded as salient, for instance informed choice of search engine (strategy) or use of Boolean operators (tactics). It might be that work at the profiling stage could lead to the development of a series of questions that resulted in scores on the strategy-tactics continuum. Later work could refine these questions into validated experimental instruments, with the linkage of existing work on Internet searching with work on the strategy-tactics continuum proving particularly fruitful, the data mapped as below (Figure Two).

 E-MAIL GREG HALE FOR COPY OF THIS DIAGRAM, ENCLOSING POSTAL ADDRESS, BY CLICKING HERE

A key issue is the relative difference in assessing strategy and tactics. With tactics, the results happen in a short time span and are relatively easy to assess, they are therefore to some degree concrete. Strategy is more difficult to assess, it involves a longer time span and may be more diffuse than tactics, so is less easy to assess. Strategy is concerned about 'ends' and metacogitive understandings of the elements at the Context level. It may be difficult to develop a standardised questionnaire to assess it, because of the danger in changing what is observed. If this problem can be addressed then a series of theoretical understandings can be developed that are tightly integrated, emerging from and rooted in the data but also strongly linked in with existing theoretical understandings.

Three issues in particular will benefit from further research. The first is the emotional factors in searching. It is becoming increasingly evident that emotional factors play a strong part in shaping search sequences. Words and phrases such as 'bored' and 'fed up' as reasons for stopping a search, rather than more rational reasons, are very evident, even in groups such as would-be information professionals.

The second area for further research (already an area of study in its own right) is at the lowest Event level, where simple scanning errors, slips or query errors can set a searcher on a course of action that is guaranteed not to produce results. Because the user lacks awareness of the errors the search becomes a sequence of failures. It can be speculated that such a level of continuous negative reinforcement will have a profound effect on users who remain unaware of the nature of the problem with their searches.

The third area relates to 'schemas' and 'scripts' (see for instance, Marshall, 1995). These could be described as familiar cognitive-behavioural patterns which are cognitively economical as they allow for actions to be taken which contain the possibility of success yet require minimum effort (for instance, the type of effort in cognitive processing related to deciding search strategy). The fact that searchers have relatively accessible cognitive-behavioural structures for tactical approaches to searching (which they use very readily) often leads to a longer searches undertaken with no thought given to strategy, particularly relating to the mental investigation of the problem space at a more global level. Hence Internet searching may become overly reactive, shaped merely by the results of the search.

In the main phase of the project it is intended to explore, amongst other things, the articulation of searchers' end-ways-means within the context of tactics and strategy. It is also intended to develop a greater and more explicit linkage between the qualitative and quantitative strands of the project, with each strand informing the other, leading to new theoretical understandings. Two papers are currently under preparation. The first will examine the operationalisation of the strategy-tactics continuum. The second will report the findings from the qualitative strand of the project in the light of the Context-Action-Event model, with a more developed and explicit linkage to key theoretical work and issues.

NOTES

[1] Thanks to colleagues Nigel Ford and David Miller for their comments on this manuscript.

[2] The main phase will deepen these theoretical links ready for publication in a paper for information science specialists.

[3] Hale (1999) was accepted for publication in Learning Resources Journal before ECER99 but is due for publication in October 1999. ECER99 is therefore the first occasion that this material and the underlying theoretical model have been publicly presented, although the model was developed before the current paper and is detailed in the Learning Resources Journal paper just mentioned.

[4] See for instance, Hale 1997, Hale 1998.

[5] In the article an extended figure of speech was developed to describe how new users interact with computer based systems. The underlying issues relate to the fact that human users of computer systems are highly active in their use (see for example, Caroll and Mack 1995), though the new user is often in danger of losing his or her way or initiating some potentially harmful set of actions. It is a formulation of concepts that stands opposed to notions of mental maps and mental models, with all their suggested predictability and completeness. Hazyspace was couched in figurative language, involving deep canyons, tall rock pillars forming stepping stones and shifting mist, with the new user sometimes hesitantly, sometimes too boldly, moving across the stepping stones (using familiar system features) whilst in constant danger of getting lost or falling into serious problems due to the highly active nature of computer use. As Caroll and Mack comment, "[Users] do act. Indeed, perhaps the most pervasive tendency we have observed is that people simple strike out into the unknown…If something can be interpreted (no matter how specious the basis for this interpretation), then it will be interpreted. Ad hoc. theories are hastily assembled out [of] these odds and ends of partially relevant and partially extraneous generalisation." (Caroll and Mack, 1995 : 699).

REFERENCES

Bates, M. (1989) The design of browsing and berrypicking techniques for the online search interface. Online Review. Vol. 13, pp. 407-424.

Caroll, J. and Mack, R. (1995) Learning to use a word processor: By doing, thinking and by knowing. In, Baecker, R.M., Grudin, J., Buxton, W.A.S. and Greenberg, S. (1995) Human Computer Interaction: Towards the year 2000, Morgan Stanley, pp. 678-717.

Ellis, D., Cox, D. and Hall, K. (1993) A comparison of the information seeking patterns of researchers in the physical and social sciences. Journal of Documentation. Vol 49, No. 4, pp.356-369.

Glaser, B.G. and Strauss, A.L. (1967) The Discovery of Grounded Theory. Aldine.

Hale, G.G. (1998) Hazyspace: Towards re-understanding the user. British Journal of Educational Technology, Vol. 29, No. 2, pp.185-186.

Hale, G.G. (1999) Developing a model of the Internet searching of entrepreneurs. Accepted for publication in Learning Resources Journal, Vol. 15, No. 2.

Hale, G.G. (1997) Putting the user first: Developing a user-centred model of software assessment in an educational context, Learning Resources Journal. Vol.13, No. 2, pp.8-13.

Hale, G.G. and Moss, N.C. (1999) Methodological Issues in using grounded theory to investigate Internet searching, Presented at the European Conference on Educational Research, Lahti, Finland, 22nd-26th September.

Jablonsky, D. (1991) Strategic Rationality is not enough: Hitler and the concept of crazy states. Professional Readings in Military Strategy, No. 3, Strategic Studies Institute, U.S. Army War College.

Jablonsky, D. (1992) Why is strategy difficult? Professional Readings in Military Strategy, No. 4, Strategic Studies Institute, U.S. Army War College.

Kulthau, C.C. A Principle of Uncertainty for Information Seeking. Journal of Documentation. Vol. 49, No. 4, pp.339-355.

Marshall, S.P. (1995) Schemas in Problem Solving. Cambridge University Press.

Mayer, R.E. (1992) Thinking, Problem Solving, Cognition. Freeman.

Moss, N. and Hale, G.G. (1999) Cognitive style and its effect on Internet search strategies: A quantitative investigation. Presented at the European Conference on Educational Research 22-26th September 1999, Lahti, Finland.

Rowley, J. and Slack, F. (1998) Designing Public Access Systems. Gower.

This document was added to the Education-line database 05 October 1999