Education-line Home Page

On Quality in Qualitative Studies

Staffan Larsson

University of Linköping, Sweden, E-mail: stala@ipp.liu.se

Contribution to the 1st International Workshop: Biographical research in Social and Educational Sciences, Institute for Applied Biographical and Lifeworld Research, University of Bremen, 5 - 6 October 1998
Revised version of a paper presented at the ECER-conference in Ljubljana, 1998.

Notions about quality are fundamental in academic work. Without such notions researchers can not perform good work. Thus, they are essential to those who produce scientific knowledge. Research is evaluated and criticized in various situations; it may concern scrutiny of doctoral dissertations or other kinds of graduate and undergraduate work. It may also concern decisions about publication of articles in scientific or professional journals. Criticism in seminars, conferences etc. rests on assumptions of what is good scientific work and what is not. A large portion of university teachers' activities are about supervising students, who are involved in different kinds of scientific work. Without assumptions about quality meaningful supervision cannot be carried out; without such criteria supervision has no direction and, thus, no value. One can draw the conclusion that quality judgements are a central part of academic work-life. Even if there are no guarantees that those judgements are very well-founded or that they are discussed in depth - they are a matter of fact and part and parcel of the academic activities.

Yet, there are not many texts about such criterias. Those existing are often quite brief; e.g., Howe & Eisenhart (1990) and in the Swedish context: Härnqvist (1978) and Ekholm, Eklund, Erasmie, Eriksson, and Werdelin (1983). There exists a good deal of literature on validity criteria (see e.g., Kvale, 1989, 1997) but quality covers (includes) considerably more than validity. One can notice that the subject index in the "Handbook of qualitative research" (Denzin & Lincoln, 1994) contains a great number of references to "validity", but none (0) to "quality".

For and against a text on quality criteria

I want to argue that there is a mismatch between the lack of texts and the importance of this aspect in everyday academic life. When a proper discussion is missing, the criteria that are actually used, run the risk of being private; everyone has his or her own more or less well-founded ideas about what constitutes good quality. Rolf (1993) has pointed out that it is, ideally, up to a professional community to preserve and develop, autonomously, criteria of quality in their profession. The autonomy should concern the group, not the individual. Thus, it is about inter-subjectivity; i.e., agreements which can be influenced by individual contributions, if they are able to convince the academic society. This article can be seen as one contribution to a discussion where the inter-subjective understanding could be highlighted. It is a way to invite to an investigation of the intersubjectivity - by presenting my version.

In my view, the open discussion should lead to more rigorous thinking; one has to consider many lines of thought and also have to take a standpoint. The open discussion may also result in questioning of criteria and thus modify the private and therefore often authoritarian feature of evaluation. If there exist anything like a communicative rationality (Habermas, 1992) in practice public deliberations on quality criteria in research is maybe one of the main ways to contribute to such a rationality in academic life, even though it is a topic that is impregnated with interests, power, micropolitics etc, i e where strategic action is often the case. Anyway, in this text I will act as if a rational discourse was possible.

On the other hand, there are certain risks about presenting a set of explicit criteria. One risk is that they may be assigned too much importance, so that the sum of criteria makes the total quality. Then the criteria become a straight-jacket. Towards the end of the 60's Cronbach and Suppes realized the risk of too rigid norms and maintained that there could be high quality in studies which deviated from standardized norms. This quality, according to them, lay in the fact that, after all, a "disciplined inquiry" had been carried through:

"Disciplined inquiry does not necessarily follow well established, formal procedures. Some of the most excellent inquiry is freeranging and speculative in its initial stages, trying what might seem to be bizarre combinations of ideas and procedures, or restlessly casting for ideas." (Cronbach & Suppes, 1969, p. 16)

We should thus take this piece of advice and not forget that criteria are blunt. We must be open to the original, provided that the reasoning is convincing.

My conclusion will be that it should be texts on this topic, but also a stress on the need for many versions and debates on the criteria that are used and proposed. Ongstad (1997) has presented an interesting critique of an earlier Swedish version of this text (Larsson, 1993) from a postmodernist perspective. This is what I think is needed. Thus I intend to suggest some criteria for evaluating qualitative studies with the intention to contribute to reflection and an open discussion. Oualitative studies inspired by hermeneutics, phenomenology and ethnography have been a part of normality during 20 years in the Swedish context and is now dominating the scene in educational research. In many other parts of the world it is the same situation. Qualitative research is thus main-stream and we should be aware of this. Part of that is to discuss quality criteria - the time for a concentration on legitimating has passed.

CRITERIA OF QUALITY AND QUALITATIVE METHOD

Qualitative method is about ways to characterize something, to give it a shape or "gestalt". The word originates from latin, "qualitas", which means character, feature, kind. Qualitative method, thus, is systematized knowledge about how to describe something in a way that presents its character.

Qualitative approaches can often be related to one of three traditions. First, the oldest tradition, hermeneutics, which once started in a theological or humanistic discourse. Second, the one here referred to as ethnography, emanating from social anthropology and sociology. Finally, phenomenology with its roots in philosophy. However, sometimes these traditions has legitimated already existing approaches (1) that was developed in practical contexts for practical purposes, sometimes they have been inspiration for developing empirical approaches. Some approaches seem to lack an explicit relation to these traditions as in the case of grounded theory. Even though those traditions have a decent in different faculties we meet them today in educational research as well as other disciplines.

Before the second world war qualitative approaches were strong in social science, for instance in the U.S. The Chicago-school in sociology was such an exemple (Faris,1970). In education there were also phenomenological perspectives represented for instance in Sweden, during this time. However, it seems as if there has been a positivistic and quantative/experimental hegemony from the end of the 40ies until the early 70ies in educational research, with some exceptions. In the end of the 60ies some qualitative studies in education that are now famous were published (Hargreaves, 1967, Jackson, 1968). During the last decade qualitative approaches has been the dominating "paradigm" in many parts of the world. Thus qualitative research has reappeared in a grand scale. In some places the lack of skills in advanced quantitative analysis is in fact a problem, as a consequence of the attractiveness of qualitative approaches.

One obvious question in this context is whether there are specific criteria for qualitative research or the criteria are general regardless of what kind of research it is. It is the old question that was debated in the 19th century if there was a single method or different in natural science from the human sciences. There are still different opinions about this. On a very practical level, I have found that some specialists in quantitative methods assume that most of the criteria presented in this article can be used also for quantitative studies. I believe, however, that it holds no longer as one becomes more concrete - in actual fact, differences occur even between different qualitative approaches. As Smith (1997) puts it: there are several vocabularies, that tend to create a balkanization in educational research. However, around some criteria there is more of a harmony across the borders.

There are two aspects necessary in evaluating scientific work which I will not touch upon in this text. One is how to deal with formalities in a proper way; in this area there exists an abundance of literature. The other aspect is the quantity of work, which is of interest in cases of meritation for professional standards; for this purpose it is not possible to formulate any general viewpoints.

It should be added, that my point of departure is literature about quality in qualitative research and research methodology as well as my own experiences in the field, not least from participating in contexts where academic work is evaluated. Consequently, my basis is not a philosophical discourse but rather a discussion that can be directly related to practical, empirical research. From this knowledge base I have tried to elaborate the views that I have met into a structure within which certain lines of thought are presented.

For the sake of clarity I would like to present the overall structure in the following headings:

Qualities in the presentation as a whole:
Awareness of perspectives,
Internal consistency,
Ethical values;

Qualities in the results:
Richness of meanings,
Structure,
Theoretical contributions;

Criteria of validity:
The discourse criterion,
Heuristic value,
Empirical anchorage,
Consistency,
The pragmatic criterion.

It may be necesssary to point out that all criteria are not always applicable. For instance, the empirical anchorage and the pragmatic criterion may, to a certain extent, be alternative to each other. Some have possible connections as for instance the heuristic vale and the pragmatic. Here one can imaging that the conception of a new pattern is followed by actions that will evaluate the use of this new understanding of a phenomenon.

QUALITIES IN THE PRESENTATION AS A WHOLE

Perspective awareness

A dominating thought in our time is that thruth is not absolute - the world is interpreted. In the broad tradition of thought named hermeneutics this idea has received a theoretical foundation.

According to hermeneutics, understanding always falls back on the idea that meaning is constituted in an interplay between parts and the whole. The parts are given meaning in relation to the whole and the whole is given meaning by the parts. There is, so to say, always an interpretation inherent in all meaningful thinking; "facts" are always perspective dependent. Already at the first encounter with what is to be interpreted we have a conception of what it means; we have a pre-understanding. This pre-understanding is changed, developed in the interpretation process into a new understanding. By making this pre-understanding explicit the basis for the interpretation becomes clear, since this preunderstanding is the foundation for the kind of interpretation that will be developed. Thus the researcher should not withhold his perspective from the reader. Ultimately, this is built upon the idea that research shall be available for critical inspection.

An account of pre-understandings then becomes a quality. To account for one's pre-understanding is not a simple task. The difficulty in the explicit description of all pre-understandings is apparent in the fact that all previous thinking in prinicple might influence interpretation.

In a practical sense, then, it is a question of limiting oneself to what is reasonably relevant. It may be to describe the actual "state of the art": different hypotheses or interpretations which have been formulated previously. These become the starting-point for approaching what is to be interpreted. This is what Ödman (1979) did when he went through the different hypotheses that were presented concerning why the men in the André-expedition (an expedition to the North Pole at the turn of the century) died.

Another way to describe one's pre-understanding is to choose a theory of interpretation, for instance, psycho-analytical or marxist, and declare the choice openly. Provided that one makes the interpretations based on the chosen theory, the reader can then judge the position of the interpretation, i.e., she can observe how the interpretation is marked by the specific standpoint that was outlined.

A third possibility is to account for one's pre-understandings by describing personal experiences, which have been influential and relevant and led to the researcher's pre-understanding of what is to be interpreted.

Within the hermeneutic tradition it is thus a sign of high quality to make explicit the perspective that is an important part of the interpretation. Inferior quality is assigned to interpretations where the perspective is not openly declared but remain concealed. The reader is then put into a situation where she has to disclose the perspective by her own analysis or else be "affected" by it in an unconscious manner.

You can find at least three reasons why perspectives are not accounted for. One reason may be that the researcher is working consciously with a perspective but wants to give the interpretation an absolute status by pretending that no perspective exists in his mind. A second reason may be that the researcher is unclear about there being a perspective inherent in the interpretation he has made. This is a case of human imperfection and incapacity to know everything, which probably influences more or less all interpretations. There are normally layers in interpretations of taken-for-granted assumptions. A third reason could be that the researcher has a view of research which differs from the hermeneutic position on pre-understanding as a a fundament for interpretation. This is true about phenomenology as well as the so called naturalistic ethnography (e.g., Blumer, 1969). What we have here is an example of disagreement among qualitative approaches as regards this particular criterion.

According to phenomenological tradition pre-understanding, i.e., the complete set of assumptions or "prejudices" we have about the phenonemon we want to describe, can be placed within brackets and thus give way to a pure description of the phenomenon. Pre-understanding - common sense as well as scientific theories - can thus be transcended according to this tradition.

Take for instance the phenomenon "anxiety". Some tend to explain it: it is due to bodily tensions or inner conflict. By such an explanation we reduce what the phenomenologist is interested in "the thing itself": What is the content of the lived experience of anxiety? In other words, the main thing is to bracket all possible "explanations" (Giorgi, 1988).

In the ethnographic tradition there is a wavering relationship to the declaration of perspectives. Many authors contend that at least some forms of theory can stand in the way of good analysis (Glaser & Strauss, 1967). One wants to get close to reality as openly as possible in order not to shut reality out by using pre-conceived categories in the analysis.

Other writers give more weight to the perspectives. Hammersley and Atkinson (1987) criticise, for instance, Blumer's "naturalistic" view of reality and point out that perspective awareness is of great value in an ethnographic study. The researcher is a part of the social context that she describes and therefore cannot produce a neutral, independent description. In order to make up for the lack of consistency among ethnographic researchers they propose one version of perspective awareness "reflexivity". They mean that one should reflect on one's own results, on how they depend on the researcher's perspective and role, and on the progression of the research process. It is a matter of reflecting, post factum, on how the final interpretation was produced.

In harmony with the thought that a world view is a social construction, the reflexivity is not only focused on theoretical preferences of the researcher but also on more trivial parts of the researcher's life. This could be e.g., in what way the gender of the researcher has influenced him/her into excluding information about certain phenomena, or how the researcher's background has influenced his/her view of teachers od students (e.g., who is regarded as (conceived of as) "hero" and who is "villain") and what impact this has had on the results of the analyses.

One concrete expression of reflexivity as regards the "social construction" of an analysis is "research biographies". These are the researcher's informal stories about how the research was carried out. In a research biography reflection is about relating (connecting) the results to the social context in which the research has taken place. Within the ethnographic tradition you often find reference to an early example of such a story, appended to Whyte's (1981) "Street Corner Society". Burgess (1984) gives several other examples from educational research in Britain.

Awareness and consistency in treatment of background assumptions are key values that Howe and Eisenhart (1990) mention in their suggested general criteria for quality of research. The consistency aspect is an important point here. A prerequisite for the meaningfulness of perspective awareness is, by and large, that there exists some consistency in application of the perspective. Lincoln and Guba (1990) have a similar point in their argument that the paradigm applied should resonate in the empirical material.

As a conclusion I want to argue in the following way: Assumptions together with the empirical data constitute the central parts when trying to understand how a result is "construed". A clear account of the assumptions as well as the empirical basis then becomes valuable, as the limits of the interpretation grow more visible. The reader can for instance judge if she shares the assumptions or not. Consequently, the reader gets a more precise understanding. By describing the assumptions, the researcher clarifies the conditions under which the results are valid. (See also Svensson, (1978). Assumptions, in this sense, may be complete theories of interpretation, or they may be more restricted assumptions pertaining to limitations of the phenomenon, methodological rules, and one's own taken-for-grantedness discovered during empirical work.

Taking into consideration the lack of communality in this respect, there is, however, a risk that a totalisation of my own view will be made at the expense of those traditions that have a different view on the perspectives, i e phenomenology and naturalistic ethnography. The notion of perspective awareness thus becomes perspective dependent in itself. It would be possible, though, to make a distinction between perspective in terms of research approach and perspective in terms of theories, or pre-conceptions, of the specific phenomenon under study. The controversy is likely to pertain to the latter case. Account of one's research approach and consistent application thereof ought to be ascribed value generally.

Internal consistency in the study as a whole

Internal consistency is, as far as I can see, one of the most commonly used criteria. This becomes more obvious in formal situations, like assessment of doctoral dissertations or review of articles submitted for publication. One reason for its frequent use may be that, as a reviewer or examiner one does not need any deep knowledge of the specific topic to be able to apply the criterion.

Howe and Eisenhart (1990) express this criterion as a harmony between research questions, data collection, and techniques of /data/ analysis. They hold that there is a subsequent line under this general norm of harmony, and that is that "research questions should drive data collection techniques and analyses rather than vice versa" (p 6).

This last recommendation is not often followed in practice, not even in cases where it is clearly stated. Often the reason is what is commonly known as "method fixation", i e methods are choosen before any other deliberations on the task.

I want to argue that the norm "research questions should drive data collection techniques and analyses" is oversimplifying the logic. Choice of methods, approaches, and the like, also include a perspective which will influence the kind of results produced. Habermas (1972) has demonstrated how experimental design deep down/at the heart of it/ contains the idea of creating a tool with the intention to rule something, e.g., nature or the human being. Methods are not neutral. Often we talk about "approaches" in order to express this: it is all about methods connected with some perspective. Therefore, it is reasonable not to use the problem exclusively as the point of departure. It is equally reasonable to assume that preferences concerning methods come into play, and this should not be denied. Peshkin (1993) argues against that restrained view of research which is often used in reviewing research plans and which leads to these plans being discarded because they are not "theory driven, hypothesis testing, or producing generalisations". He demonstrates how proposterous this view is by scrutinizing research products, among which he finds many important studies lacking these characteristics.

Another difficulty that challenges the idea that a distinct problem motivates choice of method is that qualitative approaches often discourage one from starting the research process by formulating a problem which then becomes fixed. Ethnographers would advise the researcher to walk rather openly into a "field" with an interest or "focus" formulated in general terms. As the researcher gets familiar with the field, formulation of the problem gradually becomes more distinct. It is generally argued, that this kind of procedure leads to more adequate research problems.

In phenomenographic studies the idea of the problem as the driving force also becomes difficult to apply. Here the problem is defined within the framework of the approach; conceptions are studied, because it is meaningful according to this approach to study conceptions. The vantage point is that learning, or research, is studied in a way ascribed by the approach. Therefore, in this case, the problem is more to do with choice of method for data collection or other specific methodological considerations.

In phenomenological as well as phenomenographical traditions there is also the "truthfulness to the true nature of the phenomenon" as a central value when one is planning a study. Shortcomings in this respect can, for instance, affect some quantitative methods, which require the phenomenon to be described in units that do not answer to/respect/ the nature of the phenomenon. One example would be observational studies of teaching, where teaching is described in terms of how much different people are talking in the classroom. From a phenomenological point of view it would be argued that amount of speech does not say much about teaching, since teaching is exchange of meanings, whether these meanings be about knowledge or about upbringing. A study of teaching, therefore, should have what is essential in teaching as its point of departure. Blumer (1969) who has had great impact/influence on several ethnographic studies argues very strongly for the whole research process being subordinated to "the nature of the empirical world". I think that it is generally reasonable to assume that researchers must reflect and take a standpoint about the "nature of the phenomenon" that they want to focus on.

We can, thus, state that the notion of the problem as driving the choice of method cannot without difficulty be united with the methodological thinking that belongs to qualitative approaches. Thus I would like to elaborate the original phrase that "harmony ought to exist between research question, data collection and technique of analysis". As can be observed in the reasoning above, it should be broadened to include assumptions about the nature of the phenomenon under study as well as assumptions about research. With this addition we could formulate the thesis as follows: "Harmony should exist between the research question, assumptions about the research and the nature of the phenomenon to be studied, data collection, and methods of analysis".

The idea of this harmony is built on the aesthetic principle that scientific work should be a well integrated construction - a complete system. Ths idea has a stronghold in many research environments, not least outside the qualitative domains. An excellent representative for quantitative analysis, Härnqvist (1978, p 6), writes about criteria for doctoral dissertations:"that the discussion of the results is so structured that it gives feedback on the problem and the research view before accomplishment of the study, and, at the same time, allows the author's own contributions to be sharply outlined against this background.

According to this criterion different kinds of critique can be brought forward. Some research questions may have been dislocated during the presentation, so that they are missing in the conclusions, or some conclusions have no connection with the original questions. Results from other researchers mentioned in the background chapter remain without comments, although the results of the study have clear connections with them. The background chapter may contain descriptions of theories and research with little or no relevance to the actual study. Behind this last fallacy often lies the ambition to demonstrate everything one knows about the subject, or the inability to distinguish a research report from a textbook.

Under the surface of the notion of internal logic there is also the idea of a construction, where all single parts are integrated to a whole. Parts that do not fulfil a function within this whole are regarded as disfigurements. This is a hermeneutic thought: the criterion of quality is the degree of harmony between the part and the whole.

Ethical value

An important aspect of a study is whether it reflects sound ethics. According to the Swedish HSFR* (1990), in research the urge to gain new knowledge shall be weighted against the requirement of protecting individuals that have taken part in the study. The Council, then, does not operate with an absolute protection of the individuals who have been "researched", but, rather, regard it as a matter of balance. From this, one should conclude that a reasonable balance in this respect would be a sign of good quality.

Howe and Eisenhart (1990) emphasize that a high ethical value often reduces other qualities in a study. Maybe we have to abstain from the best arrangements, or we may not be able to confirm our conclusions with full clarity if we are to protect the individuals. In other words, a conflict emerges between validity and ethics.

In case-studies this problem becomes particularly obvious, because, in such studies people tend to be identifiable if not a lot of measures are taken. Another side of the etnographic case-study tradition is that thick descriptions impedes upon the personal integrity of the persons in the field. Here, then, appears the question of how participants in the study should be protected. Standard procedure is to make anonymous individuals, places, and institutions, so that they cannot be identified. Thus, the conflict between ethical value and validity becomes apparent. You "muddle the cards" to make the presented evidence partly false and obscure.

Another aspect of ethical value has to do with protecting groups from being affected by interpretations and conclusions made from the study (see e.g., Finch 1985). The researcher has here a political problem which some may not think belong in a discussion of qualitative criteria: to what extent the researcher has been conscientious in his/her conclusions about possible consequences of the study. In other words, one should not invite hasty conclusions that may affect innocent people. Even if the researchers has little control of the interpretations made once the study is published, they will never escape responsibility for consequences. The discussion in the 50ies around researchers' involvement in the production of nuclear weapons taught a lesson for researchers about responsibility. Kvale (1997) expresses a more general ethical virtue that researchers should follow: to promote good conseqences.

A researcher's greatest sin, however, is to tell lies. As researchers we legitimize our existence as professionals by trying to be truthful. this is the virtue that I think the general public expect to be linked to researchers. Research results can be very powerful; they can also be used by those who have power and thus be part of different power games. This imposes a heavy responsibility upon researchers not to be corrupted by serving some powerful interests with legitimation of their actions. This is very obvious, when research is sponsored by organisations that almost always have such interests - like different branches of the state or the corporate sector.

Hartman (1991) points at the problem that research students can have to maintain what one holds as true, if this happens to collide with the supervisor's own interests.

Qualities in scientific work presupposes that one does not tamper with what one holds to be true. Trankell, who was behind the introduction of qualitative studies and hermeneutics in Swedish educational research in the end of the 60ies, has made a very ambitious check-list of "scientific honesty" (Eriksson reports this in Ekholm et al 1983 p 59):

  1. the author has not concealed anything he knew;

  2. he has anchored his conclusions to the material he has brought forward;

  3. he has not named things differently at different points;

  4. he has not given matters different weights according to his own motives/ objectives;

  5. he has not claimed to have aims that he has later abandoned in favour of other aims without explaining why/stating his reasons.

Scientific honesty can certainly have several aspects; they become obvious to the researcher in concrete working situation. It is difficult to cheat without being aware of it - the awareness is part of the definition of cheating. The reader may here raise the objection that researchers are not driven by such idealisms - virtues. If such disbelief is the case, it may be feasible to regard the researcher's care about scientific honesty in relation to the risk of being exposed. This is not only an individual problem - it is a problem for the profession, since researchers' position in the society partly is based on the expectation that they do not tell lies. Thus one can notice collective actions against those who are exposed as cheaters, which can be interpreted as a selfdefence in the interest of the academic profession.

The fact that the researcher has shown consideration for the people who have been studied, for those who have been affected by the research results, has to be included in the qualities of the scientific work. On the other hand, this cannot compensate for an untruthful account of the results, because thruth is a superordinate value in all research. If ethical considerations lead to the results being false in some important aspects, the researcher should withdraw from the study.

QUALITIES OF THE RESULTS

Richness of meanings

Qualitative method is about ways to characterize something, to give it a shape or "gestalt". Therefore, attention has to be concentrated on how meanings are depicted. One determining quality is richness of meanings. It is in the heart of the matter - often qualitative approaches are legitimated on the ground that it provides the richness in meanings that is lacking in other ways of doing research (Mischler 1986).

Malinowski (1922), often regarded as the one anthropologist who in the 20's gave field-work a modern methodological paradigm (Clifford & Marcus, 1986), actually built his arguments for participant observations on richness of meanings. It should provide the "flesh and blood" to the other part of the investigations results - the structure. In a similar manner Geertz (1991) argues that descriptions should be "thick". If you do not give a thick description of a culture's concrete reality, the analysis loses in richness of meanings; in other words, you do not quite know what the description means/signifies.

Firestone (1993) emphasizes the importance of the reader in case-studies. The usefulness is often that someone takes part of the case-study and uses it in her own context. The pre-supposition is that the reader has access to a rich description of the case-study, so that she can judge whether her own situation resembles that of the case.

Within the phenomenographic approach you find similar thoughts. Categories that describe conceptions ought to be rich in meanings; they should capture the essentials and at the same time maintain the nuances (Larsson 1986). Under such circumstances it is important to highlight what is unique to the specific phenomenon. General ideas such as, that one conception of a phenomenon is "abstract" and another is "concrete" can be regarded/conceived of/ as poor meanings. Glaser & Strauss present a similar thought in their advocacy for substantive theory and that formal theory should be grounded in the unique characteristics for a specific activity.

In parallel, Husserl, the father of phenomenology, claimed that you have to turn to "the things themselves", and by that he meant that we must describe phenomena as they really appear to us and in the process disregard all general and abstract theories and prejudices we happen to have about them:

We should reflect our direct experiences of the thing itself, i.e., we should turn to those acts in which we make our experiences, and with this reflective attitude we will give a neutral description of the thing itself, just as we experience it directly. We must stop inserting a theory between ourselves and the thing. (Lubcke 1987 p 47.)

Here richness of meanings becomes a taken-for-granted quality.

The same scepticism to simplified schemata can also be traced in Blumer (1969). It is, for instance, apparent when he claims that familiarity with reality ought to be the basis for studies and not a superordinate method that creates artificial results.

In hermeneutics the interpretation is described as an act in which different parts of a "text" can be integrated into a whole. It then becomes self-evident that interpretations are of higher quality if they can capture more nuances.Richness of meanings in the presentation of results is basic for the value of the interpretations as more nuances are emphasized. In this way, the interpretation becomes more precise.

In qualitative methods richness of meanings, thus, is a central value. Lack of this quality is often the starting point of critique of qualitative methods, survey studies, etc, when proponents of qualitative research attempt to legitimize their approaches.

Structure

As a counterbalance to richness of meanings it is possible to place the requirement that interpretations should have a good structure. This includes requirements of lucidity and reduction of complexity. It is feasible to imagine an implied rule of economy of thought: "as simple as possible" as a norm. This principle of economy was presented around the turn of the last century by the positivist Mach: one should not use superfluous concepts (Lubcke 1987). This seems like a sound principle even for research outside the positivist paradigm.

There is tension between richness of meanings and demand of structure. This tension must be dealt with in the qualitative analysis in an way, where both sides is not losing too much. In my view, in good cases, the tension dissolves into an exactness of the interpretation where the parts and nuances is highly structured in the presented interpretation. An interpretation, therefore, must not be the result of compromises that reduces richness and blurs the gestalt. Instead it should capture, exactingly, something fundamental that prevails in the raw data (Larsson 1986). The idea is that there exists an exact configuration/depiction of something, i.e., a considerable amount of raw data rests on exactly the same fouondation. If this foundation is made explicit, an exact description of some specific characteristic in the data emerges - the interpretation becomes precise/pronounced. A compromise, on the other hand, means that the category, or description, perhaps is not valid for any single case in the raw material.

For example, a sample of school tests is drawn, and it is found that the bulk of the sample has one characteristic in common, i.e., that knowledge is treated as pieces that can be joined together so that the sum total expresses the students' achievements. This description, then, has to fit exactly to a great many cases. Cases that do not fit have to be presented differently. The idea is that phenomena are distinguished by qualitative differences and not by gradual changes/displacements. This, of course, is a choice of perspective, but it is also the core of what we mean by qualitative methods.

According to an orthodox phenomenological tradition a good structure becomes totally decisive, as one is supposed to arrive at an essence. Not only does this mean paying attention to all aspects and nuances but also that the analysis is carried very far, i.e., to a total description of the phenomenon under study. An extensive description of experiences is not enough; it has to be made clear what is essential and what is not.

Further, the structure of the results can be diverse in complexity, e.g., in phenomenography different conceptions can be described as a number of parallel categories. But they can also appear as a hierarchical system where some conceptions are parallel and some are subordinated others. In this sense, good structure is of central value. Another fundamental requirement is for conceptions to be related to the same phenomenon, in other words., that the phenomenon does not vary. Otherwise, there is a risk that one lands in a content analysis, i.e., an account of a number of different circumstances that have been mentioned in the interviews, but the analysis is in want of an axis around which to circulate the data (Uljens 1989).

Yet another requirement is it should be possible to follow the reasoning throughout; important parts should not be omitted. In other words, there has to be a main line of thought to follow. Cronbach and Suppes (1969 p 18) writes about good studies, that are not dependent on being carried out according to standard procedures:

The detail of the argument, whether it is describing methods of data collection or the derivation of practical recommendations, is lucid, specific, and pertinent. With such a presentation there is something to learn from explication de texte, whereas in an undisciplined discussion the summary message is all that can be taken seriously.

The issue here is to develop a rationale for the reader to be convinced, as the arguments are carefully sequenced so as to create a basis for a conclusion or an interpretation. Texts often suffer from unclarity and it then becomes difficult to decide what is the main thing and what is less important. Essential points are served in one sentence, while less important matters are given much space. Good structure in a text means that it is obvious what is essential and what is less so. In other words, one important feature of the structure criterion is good rhetoric. The art of writing is thus vital for the quality of a qualitative study.

To sum up: The results should have a structure as simple and clear as possible. The structure has to be derived from the data, but the results must be presented in such a way so as to emphasize and explicate the structure. Ideally, the tension between structure and richness of meanings dissolves into an exactness of the interpretation where the parts and nuances is highly structured in the presented interpretation.

Theoretical contributions

Collective theory building is of central value in research. Here, quality is about how well one has been able to relate to earlier theory and whether one's results can change it. The concept theory is used in many ways and can be divided into a lot of variants as Glaser & Strauss have demonstrated. However, I will use it in a very modest sense; i.e., to find patterns or distinct features in the data. Reports in which general patterns the empirical material are not pointed out, do not have this quality and can hardly be called research. In other words, an interpretation of data is called for which highlights what is general. This could be fundamental features in people's experiences and conceptions of something, as in phenomenology and phenomenography. It could be rules, norms or material circumstances that give the key to how some groups act, as in ethnographic studies. It could be descriptions of processes that have a common pattern. It could also be interpretations that make use of a certain theory of interpretation aimed at creating new meanings of texts or of reality.

If you have succeeded in expressing your results in such a way, one may say that a theoretical contribution, in my modest use of the word, has been made. Since research ultimately is about producing new knowledge you have then taken a step in the right direction. To go further you must be able to relate the interpretation to the already existing relevant collective knowledge. An important aspect is the quality of this "interpretation of the interpretations", i e how the significance of the study made to the already existing discourse is pointed out.

The first step in a process aiming at a theoretical contribution is to choose a problem or focus that contains a potential for further theory building in the current knowledge base within the field of research. The formulation of a problem, therefore, can be judged on the basis of what potential of theory development it contains. This requires wide reading but also courage to do your utmost to generate new knowledge.

True theory development comes about in various ways. One widely spread idea, presented by Glaser and Strauss (1976), is to generate theories from empirical data without regard for formal theory. Thereby we get a variation of theory building, i.e., development of a theory-like knowledge within a special activity, a so called substantive theory. You work with empirical data and search for general features in them. The variant of theory contribution thus emerging simply describes different features of a specific activity. You describe mechanisms, coin concepts that capture special processes, formulate typologies, or describe how human beings conceive or experience specific phenomena.

Another type of theory contribution is to build further on earlier "substantive" theories and develop these - a cumulative development of theories about a certain activity or problem, e.g., what constitutes teachers' knowledge. The lack of such development in more naturalistic approaches has been pointed out, lately by Hammersley (1992).

Yet another variant of theory contribution is the empirical study which corrects earlier theories by demonstrating that they are wrong in some respect (Woods 1985). This is a case of falsifying an established theory.

Phenomenological and phenomenographical studies are, in principle, descriptive. Theory development in these cases becomes a matter of developing the description. This can be done either by moderating earlier descriptions or by synthesizing them into more general descriptions.

A special case is one where the researcher is pursuing/looking for/ general features in studies of activities in order to develop theories that are cutting across different areas of activity, i.e., formal theory, according to Glaser and Strauss. The task here is to scrutinize studies already carried out, in order to develop a formal theory or connect a "substantive theory" to an existing formal theory.

Finally, there is the case of theory contribution in relation to general theories of interpretation (phenomenology, marxism, etc). Here the task may be to describe an activity in some unexpected way using concepts from the chosen theory of interpretation. In this case it is hardly appropriate to talk about a contribution to theory, instead it is a matter of building a bridge between the studied case and more general knowledge.

It may also be a matter of accumulating "theory" within the framework of a larger theory. The task then is, within the frame of a certain theoretical perspective, to create a contribution according to rules laid down earlier. We get a construction of empirically grounded theory, for instance about school, based on, say, a symbolic interactionist perspective, to which we can contribute (Woods 1983). This is the way theory building is commonly carried out within the framework of general theories.

We can, thus, state that theory contributions may appear in many various forms, and that different approaches have different views on theory and theoretical contributions. Judgement must, however, be passed on the grounds of the premises for each tradition and everything evaluated according to its kind (Thorild 1790).

When assessing the quality of theoretical contributions it is first a matter of judging whether or not there is a theoretical feature in the study. Second, connections to earlier relevant theory must be assessed - does the study contribute to it? Especially, one has to consider whether one's results are discussed in relation to other relevant studies or theories. It is not uncommon that those studies that are closest are missing from the list of references; one wants to be alone in the terrain/ domain. If one refers to other relevant work, is it then possible to place one's data into the theoretical landscape in a modulated way? An important value, then, is to relate to other researchers' concepts, whenever they may function in one's own material, and only contribute with new concepts, where adequate expressions are missing. The same applies to rules, conceptions, mechanisms, etc. To describe the same thing that somebody else has done by creating differences on surface level or by giving known entities new names, is not a contribution.

Finally, this assessment is about the importance of the contribution. Have the study succeeded in contributing something decisive or is it just a note in the margin?

CRITERIA OF VALIDITY

So far, I have dealt with qualities of reports and of the character of results. We shall now focus upon the validity of results. It is not easy to draw a line between quality of reporting and validity of results. This line is somehow wavering, because the manner in which results are presented is such a central part of qualitative results. One can, after all, describe the researcher's texts as anecdotes, just like any other stories (Mischler 1986, Atkinson 1990, Gudmundsdottir 1992), in which case the line between quality of presentation and validity of results disppears. Nevertheless, I have made an attempt to draw this line.

The discourse criterion

Especially in the case of validity there is a tendency in teh discussion to treat the question of criteria in terms of one single study. This is natural, as single studies are assessed in the concrete case. As a consequence, however, the disucssion is not complete. You can, as it were, address the input from a single study to the already existing relevant research with the question of validity - has something been added to the discussion? Here validity is viewed as a conversation about the world or reality. Argumentation becomes essential and the meeting between meanings plays a key-role in the process of establishing what is valid knowledge about something.

We are now approaching a criterion which I have chosen to call the discourse criterion. It is dependent on how statements or arguments stand against other statements or arguments that can be made about the same phenomenoa or aspect of the world. Kvale (1986) points out such a criterion in stating that the test of validity is inherent in the "discourse" about the actual phenomenon. Here we are close to what was the superordinate validity criterion for knowledge about reality during medieval times, namely the dialectics in disputations (Durkheim 1977).

We have a similar setting in the courts with prosecutors and defence lawyers, where the implicit logic is that validity of claims can be judged better in the dialectic that is in the conversation or dispute. In the dialogue between opposition and defense the good arguments becomes visible.

One problem with this criterion, however, is that it may not have an independent position, as it may fall back on other criteria, e.g., empirical ground and consistency. What speaks for its independence is that it raises what could be a strong argument in these relativistic times: the comparison of alternative reasonings or arguments rather than claims about absolute truth. This "disputational" validity is not captured by the other criteria, since they are so much linked to the single study and its validity in relation to the investigated reality. In their views of knowledge about the empirical world as unclear (not logic or mathematics), our time and medieval time seem to have something in common, a fact that also seems to lead to the same suggestions as to how validity of knowledge should be tested.

The problem in having "the discourse" as a criterion may be illustrated by the fact that this must have been what Bacon in the 16th century turned against in his arguments for empirical observations. The institutionalized discourse perhaps became too complex, for the simple observation to stand its ground. This may be an example of how "complex stupidity" can develop in institutionalized scientific groups (cf. Sandgren 1974).

Heuristic value

Advocates of qualitative method often argue that it is a method of systematizing the discovery of something new. Research as discovery of new knowledge is, of course, not confined to qualitative research - it is the overall requirement for all research. The specific trait is that discovery is seen as something that can be systematized and a part of the data analysis in stead of something that is before or after the analysis.

Furthermore, in qualitative studies the gestalt, the pattern described or the line of argumentation for an interpretation is fundamentally the main thing. If you make a qualitative study you become, in a way, dependent on the fact that you can create something new; in an experimental study it is enough to falsify or accept the hypothesis or in a survey it is "new knowledge" to present figures that are affirming earlier studies. A qualitative study that replicates an already existing seems to be pointless. The quality of qualitative studies is much more based on presenting an interpretation that is not already the established one. There is a stress on a different and new interpretation or a discovery of something that was not highlighted. here the reader becomes central - it is the reader that have the old or established views and it is the readers interpretation of the world that should be affected by the qualitative study. Thus the extent to which the study can communicate a new way of conceiving something becomes vital. I will call this quality a heuristic value. Heuriskein is a greek word meaning "to discover" and in this context I use heuristic for the quality that some aspect of reality can be seen in a different way that was not obvious before one had taken part of the study. A successful interpretation thus results in a possibility to regard reality differently or that new categories of thought are created. These may be concepts, mechanisms, or the fact that a phenomenon may be connected to a context which makes something until then incomprehensible more reasonable - you can understand it. If you do not succeed to do this in some way the qualitative analysis to the ground. Jackson (1992) in fact seems to consider qualitative studies' heuristic power as a main or the main quality of qualitative studies.

One could say that a successful configuration has rhetorical qualities: Atkinson (1990) writes, referring to Edmondson, that sociological texts "persuade" the reader by means of rhetorical elements. This is not mere ornamentation, it is the substance of the results - the persuasive power. Atkinson further writes, with reference to Brown, about the centrality of certain metaphors in sociology. Metaphors such as describing society as an "organism" or reality as a "drama" or the organisation of society as a "language". Metaphors use pictures, they demonstrate what phenomena look like - they are iconic. If we pursue this line of reasoning we may also note that a successful analysis often is dependent on its results being used by people in their thinking - we come close to a pragmatic critera.

The importance of the ability to mediate interpretations effectively through metaphores and images etc. becomes not least obvious from a generalisation perspective. You cannot easily generalise from a case study to other corresponding cases. A case does not represents other cases in a simple way. Furthermore, interpretations cannot be forced upon the incredulous that reality is not how she pictured it; she may just dismiss the results as specific to that particular case. The generalisation from the case happens when those who have been persuaded by the interpretation keep it in mind as they reflect on other cases and thus may discover the relevance of the description that the qualitative analysis resulted in, if they find it useful in the particular case. Stake (1986) calls this "naturalistic generalisation". The accurate picture, metaphor, or description then becomes the way for the case study to be useful outside its own context.

Mediation to an audience, in other words, becomes especially important. Ödman (1979, p. 98) points out that this at the heart of "the interest of knowledge in hermeneutics to increase the intersubjective understanding among human beings of our time".

In sum: An essential criterion when judging the quality of qualitative studies is what I would call its heuristic quality. By heuristic I mean to what extent the reader can be convinced by the presentation of the study to see some aspect of reality in a new way.

Empirical anchorage

This criterion is about the relation between reality and interpretation. It has relevance to most paradigms in some way or other. A correspondence criterion is classic in thinking on validity. However, it is connected with the idea of a direct correspondence between interpretation and reality in a one-to-one connection, i e that there is only one possible true interpretation. Part of the critique against positivism has been about this unproblematic view of reality. The possibility of alternative interpretations of the same facts becomes excluded, which seem to be problematic. Therefore, I avoid to use the concept correspondence and instead choose to talk about empirical anchorage. In that way it is not a one-to-one relation but about a relation between interpretation and its claim to be grounded in the empirical world.

A relativistic view is comparatively common among those who work with qualitative anaysis. The reality or the text can be interpreted in more than one way, depending on the perspective that is part of the interpretation. One can thus talk about "multiple realities". On the other hand, it is difficult to point at concrete examples where resarchers in their empirical practice do not operate with the idea of the interpretation as being somehow related to the reality or the text that is being analyzed. Normally, the relativism is thus limited. All interpretations are therefore not equally reasonable - they can be more or less well anchored in the reality they claim to portrait..

The idea of an anchorage in an empirical material is a logical pre-requisite for our being at all able to talk about empirical research, it is an essential part of the concept empirical research. Blumer (1969) claims, as a take-off to the formulation of methodological principles for an empirical science: "I shall begin with the redundant assertion than an empirical science presupposes the existence of an empirical world." (p 21). Even in cases where one will not accept Blumer's realism and naturalism, it seems to be reasonable.

In ethnographic tradition one emphasizes presence in the context that is the object of study. Familiarity with the reality must be at hand and, thus, makes up the basis for claims of validity. There is almost a claim that presence - in the form of participatory observations - overcomes the distance that is implied in the idea of validity. It is especially apparent when one argues for the superiority of one's own research in relation to "variable research", such as surveys or experiments, that the ethnographer rests on this type of anchorage in the reality. Hammersley (1992) writes:

"So ethnographers long have sought to justify their approach on the grounds that it enables them to get closer to social phenomena and thereby facilitates a superior understanding to that provided by other methods." However, ethnographers' claim to represent reality has been criticised as being naive in several ways (Clifford & Marcus, 1986). The history of social anthropology show how cultures have been judged in an etnocentric way, not least being judged as primitive etc (Vindich, 1994). One side of this is the way texts makes the authors invisible - as if there was no subject behind the text. Thus the fact that it is a person representing one culture writing about her meeting with persons representing an other culture is made invisible. It is as if the researcher had no culture - no perspectives. A more profound reflexion on what claims that can be made from the presence in the context described is evident - and should be a part of the evaluation of the quality - the awareness of the problems with interpreting realities.

A practical technique which has been developed in order to validate analyses within the frame of ehtnographic research is the so called triangulation. Triangulation is a maritime term which means that it is possible to estimate one's position by noting the grades on the compass to several points in the "terrain". In a figurative sense, this means that, in an empirical study, you use several sources as evidence of your description (Hammersley & Atkinson 1987). The vantage point is that consonance between different sources is a sign of validity. There are, however, different kinds of triangulation. One variant is that a few observers independently have taken part in the same events. Another variant of triangulation is when data about the same phenomenon exist in various forms, i.e., technique triangulation. In other words, you have interview data as well as observational data or register data about the same phenomenon. Ball (undated) mentions still another variant, namely theoretical triangulation, where different theories are used in the interpretation of the same data.

A systematic use of triangulation is very effective in order to give credibility to interpretations (for an extraordinary example, see Ball 1981). Two problems may occur, though, which limits the claims of this technique. One is the meaning of the fact that different sources give different answers. That different sources give different versions - it need not mean that one is untrue, only that the same thing appears differently from different points of view. All statements are interpretations which leads to the possibility of giving different versions of the same thing. The problem is that the idea of a simple relation between interpretation and reality in implied, which is in discordance with the idea of reality as a social construction among most ethnographers

The second problem is that all reporting of raw data in the literature must be based on a selection of reality. So, when one reports different evidence it all falls back on the researcher's choice of events to be included in the report.

A special case of triangulation is "respondent validation". This is when an interpretation is brought back to the persons who have been studied in order to get their judgements of the fairness of the interpretations. The problems of such a procedure have been noticed (e.g., Ball 1984). Subjects have have many other reasons for falsifying interpretations than just being void of connections with reality. You may, for instance, get emotionally disturbed by the analysis. It would be proper to compare this state with the concept of "resistance" in freudian theory of interpretation. Psychoanalysts work with a similar process; first an analysis of the client later to get this analysis accepted eller falsified by the client. Here the same difficulty appears to clarify the meaning of, for instance, rejection of an interpretation.

Also from a hermeneutical point of view is it important that the interpretations have an anchorage in what is to be interpreted - a text or soem aspect of reality. One must have proof for what one holds as being a reasonable interpretation. Ödman (1979) calls this external control of the interpretation; do indeed the data I work with have anchorage in the reality?

Within the phenomenological tradition the question of empirical anchorage is posed in a special way. It is a matter of the description expressing "the true nature" of the studied phenomenon. In this there is a critique of descriptions of phenomena that is reducing the phenomenon to something else, in stead of going to "the things themselves". Thus, something - a theory or a taken-for-granted conception of the world is between us and the phenomenon we want to understand the essence of. A related example is that of describing learning á la Ebbinghaus, in which case learning is seen as memorizing meaningless syllables (Marton, Dahlgren, Svensson & Säljö 1977). This view of learning, according to these authors, fails to hit "the true essence" of learning, namely that learning is about meaning and change of meaning. In phenomenology it is a matter of accordance between description of the phenomenon and the phenomenon as it appears to the intrepreter-cum-author.

Kvale (1993 p 7) writes about this: "Objective may also mean reflecting the nature of the object investigated, letting the object speak, being adequate to the object investigated, expressing the real nature of the object studied", something that can also be more technically expressed in phenomenological language. Giorgi (1988 p 173) writes about validity in the phenomenological tradition:

"If the essential description truly captures the intuited essence, one has validity in a phenomenological sense. This means that one adequately describes the general essence that is given to the consciousness of the researcher."

In simple terms, the description of the phenomena must coincide with all possible ways of experiencing the phenomenon in question. The essence must be present in all conceivable experiences of the phenomenon.

In the eyes of a phenomenologist reductionism becomes an especially important form of validity failure. Reductionism in this case means that you "explain" the phenomenon by claiming that it is something other than what is present in the experience. Let us come back to our previous example, anxiety. The essence of the lived experience of anxiety is not to do with corporal tensions, which are quite common even when we do not feel anxiety. Tensions as explanation thus becomes a theory that stands in the way of a description of the phenomenon. This example may perhaps illustrate what is peculiar about the concept of validity in phenomenology. From a natural science perspective tensions as explanation can be tested for validity, and there is much in favour of it as being a valid explanation. Yet, from a phenomenological perspective this explanation is not valid as an description of the phenomenon.

Within the framework of a phenomenographical approach, one has very clearly operated with the idea that the interpretation should be anchored in the transcriptions of the interviews (Larsson 1986). Often this has been controlled in an independent judgement. What is special about such a judgement is that there are not several judges making the same analysis independent of each other. Instead, a completed interpretation is presented to an independent judge, who is sopposed to identify those interviewees that have expressed the different conceptions. One can say, that the correspondence between raw data and interpretations is established when the judge is able to identify the individuals who represent the different conceptions. The co-judge thereby is not given any opportunity to make his own interpretations, he must follow the categories created by the researcher. Therefore, it is a test of the possibility to regard the data/raw material/ from that perspective which is expressed in the interpretations.

Anchorage of the interpretation in an empirical foundation is, thus, an essential vriterion of quality in all traditions, regardless of what view of the existence of reality one holds.

Consistency

This criterion is in the heart of what we mean by hermeneutics: An interpretation is built up by the interplay between part and whole. From the parts the meaning of the whole is created meanwhile the meaning of each part depends on the meaning of the whole. Let us take as an example a film , a detective story. The first scenes are understood against the background of our previous understanding of whodunits. In the beginning we witness a murder; we know something is behind it. Our understanding of the part, the murder, is based on the general picture we have created by means of what we know so far. The more we see of the film, the more details we get; these we understand in the light of the interpretation we have so far created. But we also change our understanding of the story because of new details that are revealed to us. In this way an interpretation is built up.

The use of the hermenutical circle is not confined to the understanding of what happens in the process of interpretation; it can also be used as a model for what an interpretation should be like. We now move close to a criterion of consistency. A high quality interpretation is one where the parts of a text or available data are dealt with in such a way so as to yield as few contradictions as possible between the interpretation (the whole) and the single pieces of data that is building stones in the interpretation(the parts). Ödman (1979) mentions this in terms of internal logic. Should there exist data which do not agree with the interpretation, doubts may be raised about the interpretation. There may, then, exist an alternative interpretation that can produce connectedness with more of the data. This line of reasoning could be illustrated by using the metaphor of a jig-saw puzzle. Each piece of the puzzle has to fit in order for the real picture to appear.

We can also take judgements in legal cases as examples. In the decision about who is guilty of a crime all facts must be integrated with the conclusion (interpretation) about who is to be convicted. If one fact contradicts the interpretation (someone has an alibi) it becomes unreasonable. If more, and more significant, facts point to x as being guilty rather than z, the interpretation that x is guilty is preferred. The consistency criterion is present in all types of qualitative analysis, even when a hermeneutical approach is not used.

The criterion of consistency may seem unproblematic when one discusses examples as above. Applied on other phenomena one will find the criterion to contain a view of the world that can be questioned, namely that parts are always integrated into wholes. In other words, here is an embryo to a totalitarian feature. If reality is disintegrated, dispersed, atomistic or consists of local cultures, it is violated when attempts are made to fit the parts into the whole (Kvale 1990). It all ends up in "either - or" instead of "both - and". If you use the concistency criterion when assessing quality this means that a certain type of pre-understanding is awarded.

The idea of wholeness in the consistency criterion leads to a possible tension as regards the requirement of empirical anchorage. We may think of situations, where the wish to arrange a large amount of different circumstances in the same whole leads to disregard of certain circumstances because they do not fit into the whole. In fact, one could say that interpretations always include this problem. In my view, this problem is in want of a solution; after all, interpretations presuppose an interplay between part and whole. This in turn leads to a tension between requirements of consistency and empirical anchorage. Awareness of the imperfection of interpretations, and sound judgement are therefore required in assessment of scientific work. Ultimately, it is a question of optimization, or to reach as far as possible taking both criteria into consideration.

The pragmatic criterion

As regards the pragmatic criterion the question of validity to a large extent becomes a question of consequences of the results brought about in a qualitative study. Howe and Eisenhart (1990) suggests as one of their criteria "external value". By external value they mean the value that the resarch may have to practice: "the 'so what' question". One aspect of this is how the researcher has succeeded in communicating his results to the practitioners and in telling them what practical significance the results have; "educational researchers themselves should be able to, and be prepared to, communicate what value their research has (if only potentially) for educational practice". Choice of media for presenting the results, and use of language are here equally important.

Habermas (1972) distinguishes between different kinds of studies according to different knowledge interests. The concept "knowledge interest" contains in itself a pragmatic criterion, i.e., that use of research results are in the forefront. In line with this idea the natural sciences have brought forward a kind of research, with experimental design as a model, the use of which is to rule over nature. He calls it a technical interest of knowledge. As a consequence validity is contained in the fact that the research generates results which are useful/applicable when you wish to manipulate nature. Having said this, Habermas criticizes the use of the natural science model in the field of social science, because the idea of science as a tool to govern/ rule over/ something becomes doubtful within the social sciences. However, against this Habermas places two types of hermeneutical interest of knowledge for the human and social science fields; a practical-hermeneutic or an emancipatory. This is directly applicable to qualitative studies.

The practical-hermeneutical interest aims at improving communication by contributing interpretations, world views that can create platforms for understanding between people. Here the question of validity will be whether or not the interpretation through its qualities is capable of creating such a ground for the "dialogue". The emancipatory interest has its application in the critique of myths about how society is or ought to be. The idea is to prove that some conceptions held by people which allow them to accept oppression, are not in accord with reality. By making empirically grounded interpretations of reality one can create a dynamic situation where previous interpretations can be questioned. The use of scientific analyses, then, is a way for people to free themselves of their mythical ideas and act rationally. Habermas uses as a metaphor psychoanalysis, where the individual's understanding of how she irrationally oppresses herself leads to liberation, the effect of which is that she can act more rationally. Habermas tranfers this line of thought to a societal level, where the liberation is political.

From an emancipatory interest of knowledge you may derive a pragmatic criterion of validity, namely, to what extent research can contribute to the disclosure of oppressive myths. Another side of it is whether the analysis is correct to the extent that it can be used as a ground for functional actions. In feminist and marxist traditions such a pragmatic criterion stands firm.

Among different ethnographic types of validity there is one with an evidently pragmatic feature. Here the ability to translate results from the analysis of a culture to an ability to perform competently in the described culture is emphasized: "The extent to which the cultural description produced provides a basis for competent performance in the culture studied" (Hammersley 1991). Hammersley (1992) is also pointing at an instrumental position in a discussion about different foundations for validity. He notices that the symbolic interaction-tradition (Blumer, 1969) originally was related to pragmatism, but that this fact has not meant so much in ethnography. Hammersley argues for a position where he relates a "subtle realism" with a "relevance"-criterion. "Relevance" must be something that is close to a pragmatic criterion

In the phenomenographic tradition you will also find a pragmatic criterion. It is presumed that the need for a pragmatic criterion has grown against the background of maintaining that people's conceptions of a phenomenon are applicable in a teaching context. It is actually claimed that such a thesis/hypothesis can create the basis for didactics (inte bra). The pragmatic criterion has been formulated thus: "The quality of the analysis is dependent on to what extent teaching based on the described conceptions can result in increased understanding of a certain content." (Marton 1991).

IN CONCLUSION

When using the criteria discussed above, I imagine that they can serve as tools for finding out the strong and the weak points in a scientific piece of work. After all, most work has weak points for different reasons. For some types of studies certain criteria are important. In a description richness of meaning becomes especially important. If the work is a stage of the development of schooling maybe pragmatic validity is of particular importance. The criteria must not be used mechanically; they are subordinate to a general assessment of the whole.

Some criteria are interchangeable, such as empirical anchorage and pragmatic validity. Others may be contradictory. This is the case with, for instance, richness of meaning in relation to structure. It also applies to ethical value that has to be weighed against the requirement of validity. Common sense has to decide whether a piece of work has found a proper balance.

Finally, I hope that this work can be of some use. At the same time, I am afraid it may not be taken for what it is - a suggestion, a personal construction of my experiences and some literature I have read on the subject. There are other perspectives. Do not forget that there must be room for studies that take their own course; studies that are capable of convincing despite the fact that they do not follow the rules laid down/ established rules/.

Acknowledgement

I would like to thank Peter Alheit, University of Göttingen, Ivor Goodson, University of East Anglia Philip, W. Jackson, University of Chicago for the suggestions and critique of earlier versions of this text

Referenser

Asplund, J., (1971) Om undran inför samhället. Kalmar:Argos Förlag AB

Atkinson, P.,E (1990) Ethnographic Imagination. Textual constructions of reality. London: Routledge, 1990.

Ball, S., J., (1981) Beachside Comprehensive: A Case Study of Secondary Schooling. Cambridge: Cambridge University Press.

Ball, S., J., (1984) Beachside Reconsidered: Reflections on a methodological apprenticeship. I: Burgess, R., G., (ed.) The Research Process in Educational Settings: Ten Case Studies, London: The Falmer Press.

Ball, år okänt, The verification and application of participant observation case study.

Blumer, H., (1969) Symbolic Interactionism, Perspective and Method. Berkeley: University of California Press.

Burgess, R., G., (ed.) (1984) The Research Process in Educational Settings: Ten Case Studies, London: The Falmer Press.

Clifford & Marcus (1986)

Cronbach, L.,J., Suppes, P.,(Eds.) Research for Tomorrow´s Schools. Disciplined Inquiry for Education. Toronto: The Macmillan Company

Denzin, N.,K., Lincoln, Y.,S., (Eds.) (1994) Handbook of Qualitative Research. Thousand Oaks: Sage.

Durkheim, E., (1977) The Evolution of Educational Thought. Lectures on the Formation and Devlopment of Secondary Education in France. London: Routledge & Kegan Paul.

Ekholm, M., Eklund, H., Erasmie, T., Eriksson, K., H., Werdelin, I. (1983) Fem uppsatser om handledning och avhandlingsarbete. Rapport LiU-PEKR-82. Universitetet i Linköping, Pedagogiska institutionen

Elster, J., (1988) Vetenskapliga förklaringar. Göteborg: Korpen.

Faris, R.,E.,L., (1970) Chicago Sociology: 1920 - 1932. Chicago: University of Chicago Press.

Finch, J., (1985) Social Policy and Education: Problems and Possibilities of Using Qualitative Research. I: Burgess R., G., Issues in Educational Research: Qualitative Methods. London: The Falmer Press.

Firestone, W.,A. (1993) Alternative Arguments for Generalizing From Data as Applied to Qualitative Research. Educational Researcher, 1993, Vol 22, No 4, pp 16-23.

Geertz, (1991) Tjock beskrivning. För en tolkande kulturteori. Häften för kritiska studier, 3/91.

Glaser B., G., Strauss, A., L., (1967) The Discovery of Grounded Theory: Strategies for Qualitative Research. New York:Aldine.

Georgi, A.,(1988), Validity and realiability from a phenomenological perspective. In: Baker, W., J., Moz, L.,P.,Rappard, H., V.,Stam, H., J.,(eds.) Recent Trends in Theoretical Psychology. N.Y: Springer Verlag pp 167 - 176.

Gudmundsdottir, S., (1992) A critical look at the research interveiw. Reviderad version av ett paper framlagt på AERA, 1992. Stencil.

Habermas, J., (1972) Knowledge and Human Interest. London

Habermas, J., (1992) The Theory of Communicative Action, Volume One. Oxford: Polity Press

Hammersley, M., (1989) The Dilemma of Qualitative Qualitative Method: Herbert Blumer and the Chicago Tradition. London: Routledge.

Hammersley, M., (1990) Classroom ethnography,

Hammersley, (1991) Reading ethnography, manuskript.

Hammersley, M.,(1992) Some reflections on ethnography and validity. Qualitative Studies in Education, 1992, Vol. 5, No. 3, pp 195-203.

Hargreaves, D., H. (1967) Social Relations in a Secondary School. London: Routlegde & Kegan Paul.

Hempel, C., (1969) Vetenskapsteori. Lund

Howe, K., Eisenhart, M., (1990) Standards for Qualitative (and Quantitative) Research: A Prolegomenon. Educational Researcher, Vol. 19, No. 4, pp 2-9.

HSFR etik. (1990) Forskningsetiska principer för humaniora och samhällsvetenskap. Stencil

Härnqvist, K. (1978) Om krav på doktorsavhandlingar. Några handledares synpunkter sammanställda av Kjell Härnqvist. Rapport från Pedagogiska institutionen, Göteborgs universitet. Nr 170. Oktober 1978.

Jackson, P. (1968) Life in Classrooms. New York: Holt, Rinehart & Winston

Jackson, P. (1993) Qualitative research and its public. Qualitative Studies in Education. Vol 6, No 3, 227-231.

Jaeger, R.,M.,(ed.) (1988) Complementary Methods for Research in Education, Washington, DC: AERA.

Kvale S., (1989) To Validate is to Question. I: Kvale (ed) Issues of Validity in Qualitative Research. Lund: Studentlitteratur.

Kvale, S., (1990) Postmodern Psychology: a Contradictio in Adjecto? The Humanistic Psychologist Vol. 18, No 1, Spring 1990.

Kvale, S., Ten Standard Objections to Qualitative Research Interveiws. Journal of Phenomenological Psychology (in press)

Kvale, S., (1997) InterViews. Thousand Oaks: Sage.

Larsson, S., (1986) Kvalitativ analys - exemplet fenomenografi, Lund: Studentlitteratur.

Larsson, S., (1988) Response to Clandinin and Conelly. Journal of Curriculum Studies, 20(2), pp 171-172

Lowie, R.,H.,(1938) The History of Ethnological Theory. New York: Holt, Rinehart and Winston.

Lubke, P. (red.) (1987) Vår tids filosofi. Stockholm:Forum AB

Malinowski, B., (1961) Argonauts of the Western Pacific. Dutton Press.

Marton, F., (1981) Phenomenography - describing conceptions of the world around us. Instructional Science, 1981, 10, pp 177-200.

Marton, F., (1991) muntlig kommunikation

Marton, F., Dahlgren, L., O., Svensson, L., Säljö, R. (1977) Inlärning och omvärldsuppfattning. Stockholm: AWE/Gebers.

Mishler, E., G.,(1986) Research interviewing. Context and Narrative. Cambridge: Harvard University Press.

Ongstad, S. (1997) Sjanger, posisjonering og oppgaveideologier. Et teoretisk-empirisk bidrag til et tverrfaglig, semiotisk og didaktisk sjangerbegrep. Trondheim: NTNU.

Peshkin, A., (1993) The Goodness of Qualitative Research, Educational Researcher, Vol. 22, No 2, pp 23 - 29.

Polkinghorne, D., (1983) Methodology for the Human Sciences. Systems of inquiry. Albany: State University of New York Press.

Smith, J. (1997) The Stories Educational Researchers Tell About Themselves. Educational Researcher, Vol. 26, No 5, pp 4 - 11.

Snyder, B., R., (1973) The Hidden Curriculum, Cambridge, Ma: The MIT Press.

Stake, R., E. (1986) An Evolutionary View of Educational Improvement. In: House, E., R. (Ed.) New Directions in Educational Evaluation. London: The Falmer Press. pp 89 - 102.

Starrin, B., Larsson, G., Dahlgren, L., Styrborn S. (1991) Från upptäckt till presentation. Om kvalitativ metod och teorigenerering på empirisk grund. Lund: Studentlitteratur.

Thorild, T., (1791) En critik öfver critiker. Med utkast til en lagstiftning i snillets värld.

Whythe F., W., (1981) Street Corner Society. The Social Structure of an Italian Slum. Third edition. Chicago: The University of Chicago Press.

von Wright, G., H., (1971) Understanding and explanation.Ithaka, NY.

Ödman, P-J, (1979) Tolkning, förståelse, vetande. Hermeneutik i teori och praktik. Halmstad: AWE/Gebers.

Österberg, D., (1991) Sociologins nyckelbegrepp och deras ursprung. Göteborg: Korpen

Notes

1. This is the case with the relation between phenomenography and phenomenology.

This document was added to the Education-line database 19 October 1998