trustworthiness and integrity in qualitative research

Controversies about quality

A major dispute has involved whether “validity” and “rigor” are appropriate.

Some reject these terms and concepts totally, some think they are appropriate, and others have searched for parallel goals.

no common vocabulary exists

• - goodness
• - truth value
• - integrity 
• - trustworthiness 
• - validity and rigor

• - some frameworks and criteria aspire to being generic - to be applicable across qualitative traditions
• - other frameworks are specific to a tradition or even to a specific analytic approach within a tradition

the criteria that were outlined by Lincoln and Guba

trustworthiness

• concerns the "truth value" of a qualitative data, analysis, and interpretation
• a parallel perspective, with analogs to quantitative criteria

• - credibility 
• - dependability 
• - confirmability
• - transferability 
• - authenticity

• refers to confidence in the truth of the data and interpretations of them 
• the analog of internal validity in quantitative research 
• arguably the most important criterion for assessing the quality and integrity of a qualitative inquiry

• refers to stability of data overtime and over conditions
• the analog of reliability in quantitative research

• refers to neutrality - the potential for congruence between two or more people about data accuracy, relevance, or meaning
• the analog of objectivity in quantitative research 

• the extent to which findings can be transferred to other settings or groups
• the analog of generalizability or external validity in quantitative research

• the extent to which the researchers fairly and faithfully show a range of different realities and convey the feeling/tone of participants' lives as they are lived
• no analog in quantitative research 
• added to the Lincoln-Guba framework at a later date

• researchers can take many steps to enhance the quality of their inquiries 
• consumers can assess quality-enhancement efforts by looking for these steps and assessing their success in strengthening integrity/validity/trustworthiness

• prolonged engagement 
• persistent observation 
• reflexivity strategies
• data triangulation
• method triangulation
• comprehensive and vivid recording of information 
• member checking

the investment of sufficient time collecting data to have in-depth understanding of the culture, language, or views of the people or group under study, to test for misinformation and distortions, and to ensure saturation of important categories

concerns the salience of the data being gathered. it refers to the researchers focus on the characteristics or aspects of a situation that are relevant to the phenomena being studied

involves awareness that the researchers as an individual brings to the inquiry a unique background, set of values, and a social and professional identity that can affect the research process

the use of multiple data sources to validate conclusions [time, space, and person triangulation]

• involves collecting data on the same phenomenon or about the same people at different points in time
• similar to test-retest reliability assessment 

involves collecting data on the same phenomenon in multiple sites, to test for cross-site consistency

involves collecting data from different types or levels of people [e.g., individuals, family, communities], with the aim of validating data through multiple perspectives of the phenomenon

the use of multiple methods of data collection to study the same phenomenon (e.g., self-report, observation)

Maintenance of an audit trail, a systematic collection of documentation and materials, and a decision trail that specifies decision rules

• the raw data [interview transcripts]
• methodologic, theoretic, and reflexive notes
• instrument development information [i.e., pilot topic guides]
• data reconstruction products [i.e. drafts of the final report]

providing feedback to participants about emerging interpretations; obtaining their reactions

A controversial procedure—considered essential by some but inappropriate by others

• investigator triangulation 
• stepwise replication 
• theory triangulation
• searching for discomfirming evidence and competing explanations
• negative case analysis
• peer review and debriefing 
• inquiry audit

• use of two or more researchers to make data coding, analytic, and interpretive decision
• the thought is that through collaboration, investigators can reduce the possibility of biased decisions and idiosyncratic interpretations

• dividing the research team into two groups to undertake parallel analyses and interpretations that are then compared
• associated with Lincoln-Guba dependability criterion

use of competing theories, hypothesis, or conceptualizations in the analysis and interpretation of data

• Search for disconfirming evidence as the analysis proceeds, through purposive/theoretical sampling of cases that can challenge interpretations
• this strategy depends on concurrent data collection and data analysis; researchers cannot look for discomfirming data unless they have a sense of what they need to know

• a specific search for cases that appear to discredit earlier critical feedback
• this strategy is a process by which researchers review their interpretations by including cases that appear to disconfirm earlier hypotheses

sessions with peers specifically designed to elict critical feedback

• a formal scrutiny of the data and relevant supporting documents and decisions by an external reviewer 
• can be a good tool for persuading other that qualitative data are worthy of confidence

Thick and contextualized description: vivid portrayal of study participants, their context, and the phenomenon under study

Researcher credibility: enhancing confidence by sharing relevant aspects of the researcher’s experience, credentials, and motivation

Interpretation in qualitative inquiry—making meaning from the data—relies on adequate incubation.

Similar interpretive issues as in quantitative research: credibility, meaning, importance, transferability, and implications

is the process of living the data, a process in which researchers must try to understand their meanings, find their essential pattern, and draw well-grounded, insightful conclusions.