Research Methods I- Qualitative Research- SWRK 653

Typos: courtesy of Itamar Danziger

Sept. 4, 2008

Issues to be discussed in class

• theoretical foundation of qualitative research
• community and cultural context
• qualitative research methods and tools:
• interviewing
• focus groups
• how to design a study

stages

• conceptualization
• methods
• data collection
• data analysis
• results
• future research

-here, the people are the center of our research.

Learning outcomes:

• discuss theoretical foundation of qualitative research
• differentiate researchable qualitative and quantitative social work research questions
• articulate the process of development of qualitative research in SW
• design a qualitative research study
• critically analyze the material you read

Assignments

1. assignment 1 – position yourself as a researcher
1. sept 25
2. paper outlining your research question
1. oct 16
3. interview and analysis assignment
1. oct 30
4. student presentation
1. nov 13/20
5. research proposal
1. nov 27

written assignment guidelines

• APA manual`
• Academic integrity
• Due dates
• Communication
• Break

Why are we here?

-improve effectiveness/efficacy of practice

-understand problems better

-Policymaking is influenced by research. It is very important to fit in the qualitative… it is still lacking in the policymaking world

-seeing if practice and policy decisions regarding appropriate assessments and treatments

What is qualitative research

Multi-method

• Non-numeric/text data
• Look at naturalistic setting – not out of context
• Interpretive

Multi-source

• myriad methods to describe moments and meanings

-sees subjectiveness as important

-natural source =is most important – more than manipulating situation.

The observer is the key instrument. This has reflexivity;

-in qualitative studies, the studier is mitigates what he says. Therefore we have to know his position

Descriptive: data is the words and not the numbers.

Concerned with the process rather than outcome. Asks how instead of why something happens

Inductive analysis/interpretive - seeks to discover – not to test seeks to understand experiences - looks for themes

Meaning and subjectivity are central – how do people make sense of their lives? What is the participant's perspective?

Tradition of inquiry

1. case study – exploration of a system or case (1+). There are many kinds of case studies: interviews/focus groups/etc…
2. Ethnography: to learn about a group: implies long term engagement
3. grounded theory: looking at the group and then building a theory
4. narrative: look at the stories which are told – look at single person's experiences
5. phenomenology: an examination of meanings of people's experiences – look at a singular phenomenon – i.e. parent to ADHD – not individual's experience but the experience of someone who has X

Qualities of a good researcher

-people mentioned things similar to individual intervention things – empathy, humor, able to listen, etc…

Sept. 11, 2008 class

Qualitative research has 3 principles:

1. you can learn something from interactions/relationships. Values human interaction and human perspective. The participant is central/the way they say/represent/describe.
2. Compared to quantitative, qualitative likes to use their language/quotes.
3. The participants shapes the study– voice/representation

Historical influences

1. anthropological tradition
2. reflexive paradigm – questioned positivism
3. feminist
4. postmodern

you gotta have:

• sociocutural understanding which is different from physical understanding
• shared assumptions about objective reality/observer neutrality
• notion of naturalistic inquiry
• outsider/stranger within a setting
• participant observation technique
• naturalism neglects subjectivity of researcher

reflexive paradigm

• questioning of positivism (that there one objective truth). Is everything observable and measurable?
• Raised concern over effects of the researcher
• Feminist critique: questionings of positivist methods, because I could be used for political ends for/against the participants. Neutrality/objectivity is questioned
• Need the research to be useful- not just “understanding” need to be applicable to bring about action and change to address the issues of oppression in its various forms

Post-modern critique:

• rejection of realism
• notion of deconstructivism (Derrida) questioning idea that researcher cab capture who people act
• meaning is not stable/not properties of individuals but reflect constructions of subjectivities through language
• writing is not a transparent medium
• Foucault: social research is used to control/surveillance of people
• Products of social research reflect its social character. What is seen as true/false in social research is seen through the exercise of power

Third methodological method:

1. mixed methodology (mix of qualitative and quantitative)
2. assumption is that quantitative came before qualitative
3. each “side” is seeking superiority
4. multiple ways of contextualizing research through integration of approaches

5 guiding assumptions

1. multiple nature of reality (ontological)
2. Close relationship between researcher and participant (epistemological)
3. Value laden aspect of inquiry (axiological issue)
4. Personal approach to writing the narrative (theoretical issue) – social workers study things because “We think it is important!”
5. Emerging inductive methodology of the process of research (the methodological issues). i.e. you speak more about who the person was in the study. This approach allows the researcher to be more open to the possibility that the participants will set the tone and not the researcher will set the schematic category.

Video

Questions of qualitative research

-interested in circumstances of every day life.

Ethnography uses

• Observation
• Participant observation (observing while participating)
• Interview

Textual observation: uses

• Language analysis
• rhetorical analysis
• content analysis – look for patterns in the content – i.e. who is being quoted/what is the themes?/etc… could be interpretive – looking for deeper meeting. Or looking at plain meaning

applications examples:

1. circulation of audio-visual material from terrorists. You can study them better with qualitative and less from quantitative methods
2. doing business in a foreign country – gotta know the culture

gatekeeper: the contact person – the middle man between the researcher and his participants from different “cultures” or social groups

Sept 18, 2008

Ethics of concealment: deception & disclosure

Usually comes up in the consent form. The question is how much you tell.

Range: disclosureàneglectàdeceit

Neglect is when you don't tell people at all that you are observing… i.e. if you are observing people on the street.

Deceit: when purpose of study and researcher's role in unknown (=full deceit).

Confidentiality and privacy

Anonymity –researcher doesn't know the participant's details or what category that they are in. the confidentiality –someone knows who you are but the person being evaluated (i.e. teacher) does not know who said what -Tenet: never reveal the identity of the responder

Distress and Emotional Harm

Principle of least harm – you have to make arrangements for professional referral prior to your study!

Informed consent

• how to say how long the study will be/where
• identifying the researcher/organization [include address and phone number for future contact]
• be very clear about reassurance of voluntary participation
• assurance of confidentiality
• knowledge of risks and benefits of the study – i.e. you may become upset
• people under 18 – need to phrase for them as well as their parents' signature
• for people do don't want to sign –i.e. illegal immigrants, you have a second form which you need a witness to sign
• need 2 consent forms – one for you and one for the participant

coercion/deform consent

• you need to be aware of potential for coercion
• assure the right to withdraw
• researcher shouldn't study clients/colleagues

incentives/payback

• most funded studies involve remuneration for respondents' time. But this could be seen as coercion. Addicts could used for drugs (when addicts are studied). Some claim: they are adults, so they will have to decide how to use their money

risk to the researcher

• emotional concerns - ups and downs of fieldwork
• exhaustion – of too many interviews
• reactions
• threats to physical safety

power

• can't be too voyeuristic – ethical problem
• be aware of your organization's power

Violation of the moral code

• illegal activity – you need to guarantee that you won't report it, despite your own values

socially responsible research

• framing your results in an unharmful way
• guarantee confidentiality

Sept 25, 2008

Examining research design

-2^nd half of class – mapping the concepts – for out research design

Reseach components

• abstract – like 100 words
• introduction – presenting your study
• problem – stating the issue that led up to the problem. This states your area of interest. Acquaints your reader with what your topic is about. It also introduced key concepts and what has what already studied in regards to your topic. Then you narrow your approach to specific research question. Also convey the importance of your proposed study – point out lack of info in the literature/gap in info) any inadequacy in what is already known concerning your topic. The need for this study should be obvious to the reader.
• statement
• research questions
• research context – i.e. literature review
• research methods
• plans for collecting data
• plan for analyzing data
• plan for employing standards of quality
• Conclusion: Implications for findings, plan for dissemination of findings

using philosophic assumptions/theoretical frameworks/guiding assumptions

• multiple nature of reality (ontological)
• close relationship b/w researcher and researched (epistemology)
• value laden aspects of inquiry (axiological issue)
• personal approach to writing the narrative (rhetorical issue) – i.e. what language you use
• emerging inductive methodology of the process of research (methodology issue)

conceptualizing the research process (methodology)

• how we conceptualize the entire process
• researchers start inductively, studies topic within its context and then describes in detail. Works with details before generalizations and continually revise questions from experience in field. i.e. looking at elderly àelderly in Montreal àethnic differences between caregiver and elderly. The point is tat the concept in the research is evolving. àneed to explain your structure – so that issues of credibility doesn't go crazy
• Frameworks with distinct ideologies stance seeks to empower participants, such as a feminist, post-modern or critical approach, which situates study within a context, historical framework or socio-political perspective àyour ideological stance shapes your study – but should not be restrictive.
• Ideological perspective provides conceptual lens for designing a qualitative study to bring about change and action
• Researchers may employ social science theory to guide qualitative studies

Introduce the study

• This phase in designing your research proposal involves:
• Stating the problem or issue leading to your study
• Formulating the central purpose of the study
• Providing the research questions
• Identifies your area of interest
• Acquaints your reader with what your topic is about
• Introduces key concepts and what has already been studied in regard to your topic

Picking who/what you interview depends:

• your research question
• what you want to know
• access to potential respondents
• population (group from which you select participants)
• sample (your chosen participants)

sampling stratagies

• Purposeful or theoretical sampling
• based on the ability of respondents to provide the needed information
• Convenience sampling
• based on ready availability of respondents (ethics of using captive populations?)

àproblems include ethics (i.e. participating in your professor's study) and it might not include all the population sample you want (i.e. a study in an old age home might include only those who are mobile enough

• Snowball sampling
• respondents who know others in similar situation provide suggestions to the researcher
• researcher may advertise for volunteers (ethics approval of materials is needed)

à downside: may reduce the information that you get (the diversity). Others will want to keep their identity hidden

Family Pathology

àprotective factors = no repetition of pathology

àn protective factors = mediating factors in learning the pathology

Question: does next paper include lit review?

Oct 2, 2008

-get your questions ready for next class

The research question

• specify what you want to understand by doing your study
• shouldn't be formulized in detail until purposes and context of design are clarified
• remain adaptable to other parts of the design

characteristics of research questions

• open-ended
• non-directive/evolving
• restate the purpose of the study in more specific terms

• Start with “what” or “how” rather than “why”
• Also, “in what ways”
• Are few in number
• One strategy: reduce entire study to a single over-arching (central) question and several sub-questions. – i.e. phrasing your statement/question in one phrase
• Bhattacharjea (1994)
• What is the nature of the expectations that affect female administrators’ actions?
• What strategies do female administrators adopt to deal with these constraints in the context of a gender-segregated and male-dominated environment?

àassumption: there are constrains

• Comments?

Example

• Bhattacharjea (1994)
• What is the nature of the expectations that affect female administrators’ actions?
• What strategies do female administrators adopt to deal with these constraints in the context of a gender-segregated and male-dominated environment?

• Comments?

Your research questions need to take account of

• Why you want to do the study (your purposes)
• What is already known about the things you want to study and your tentative theories about these phenomena (your conceptual context)

Assignment #2

• Describe issue being addressed and why it is important to address it (problem statement)
• Describe how you became interested in the research topic.
• Provide purpose of the study (purpose statement) and show path you took to formulating your research question (include preliminary research questions).
• Justify use of qualitative methodology to explore it
• Use clear, well-edited writing style, APA standards for references

Who will help you enter the field?

Gaining entry: common questions from those in the setting

1) What are you actually going to do?

• Be honest; try to explain your intentions clearly; don’t be too lengthy in your explanation.

2) will you be disruptive?

• Part of being successful is being non-disruptive; assure participants you will not make excessive demands; be sensitive to their problems and requirements.

Gatekeeper: people within the community who will help you accessed them.

3. What are you going to do with your findings?
1. Decide how you will use the material; share this decision/information with your participants.

4. What will we get out of this?
1. Decide what you are prepared to give.

Gaining entry and staying in

• finding a gatekeeper
• gaining formal permission
• maximize your acceptance (fitting in)
• gain rapport and build trust
• by sympathetic to the goals of the group
• provide safeguards for people – i.e. if person gets upset, turn off recorder/take a break/safe space

Difference between research questions and interview questions

If you are conducting interviews in your study:

• Your research questions identify the issues you want to understand
• Your interview questions will provide the data that you need to understand these issues.

Research interviews are different than clinical interviews:

• Goals – data collection and not assessment and intervention
• content – just narrowed to studies material
• relationship

framing your study

• Begin with puzzlements and jottings
• What is your interest/what is pussling you

• Do a global sorting and ordering
• Identify the subtopics/themes

• Do a section sorting & ordering
• Decide the order of your concerns and questions to arrive at the overall research design – the idea is to narrow it down

Achieving coherence;

• Dichpronic: look at development and change over time – i.e. the attached study of parents –how attachment develops
• Synchronic: looks at themes – i.e. occupationally successful men – what makes them successful

What type of information will you collect? (Lofland, 1984)

• Who is s/he? What does s/he do?
• What do you think is meant by that
• What are they supposed to do?
• Why did she do it
• Why is it done
• What happens after____?
• What does she think that?
• Who is responsible if___?
• etc..

structure of interview

• structured – all are asked the same questions
• semi-structured
• informal

structured interview

• formal structure of interview questions
• all are asked the same questions
• less room for exploration
• assumptions:
• Questions are sufficient and comprehensive
• Wording is relevant and will elicit desired information
• Meaning is the same for every subject
• Examples:
• Is your housing adequate?
• Do you have enough money to live comfortably?

Unstructured interview

• Characteristics:
• Open-ended
• No schedule of questions
• Process of developing and adapting questions (probes) along the way

• Assumptions:
• Research does not know relevant questions
• Meanings of questions will differ for participants
• Example:
• Describe a typical day in your work week…walk me through it
• Useful when the researcher is unfamiliar with respondent’s lifestyle, customs, etc.

-unstructured interview is harder to code – but you are looking for themes in such interviews

Semi-structured interview:

• Characteristics:
• Located between structured and unstructured interviews
• Involves predetermined questions or special topics
• Interviewees are permitted to digress
• Interviewers are permitted to probe
• Assumptions:
• Interviewer has idea of where interview should go but does not feel restricted to this path

Thinking of what to ask:

• experiences/behavior of a certain issue
• opinions/values
• specific knowledge/facts [depends on if you are looking for that kind of data
• feeling/emotion [in response to a situation]
• sensory impressions
• background/demographic info [could be problematic – and we'll discuss it]

Factors that shape the questions you ask

• Problem you are investigating (What do you want to know?)
• Material you want to collect and boundaries of your study (overall frame)
• Repertoire of understandings based on previous work and study, familiarity with literature and experiences
• Pilot research (in practice, we then go on to develop better ideas about what to ask) –they are made form people who fall under the same group as population
• Sense of what will give substance to eventual report

Who do you interview?

Depends on...

• your research question
• what you want to know
• access to potential respondents
• population (group from which you select participants)
• sample (your chosen participants)

Types of questions

• Basic descriptive
• Can you tell me about your car accident? Tell me what happened that evening. Describe how you felt?
• Follow-up
• You mentioned that “planning time” is important to you. Can you tell me how you use planning time?
• Experience/example
• You mentioned that you loved going to Paris. Can you give me some examples of what you loved about the city?
• Simple clarification
• You used the term “constructivist teacher” today. Can you clarify that for me? What can you tell me about your own constructivist teaching?
• Comparison/contrast
• You said there’s big difference between a great principal and an ordinary one. What are some of these differences? Can you describe them for me?
• Structural/paradigmatic
• You stated that this class was problematic. What would you describe as the cause of these problems? Of all the things you have told me about being a critical care nurse, what is the underlying premise of your workday? In other words, what keeps you going everyday?

àbetter to record – so you won't be stuck taking notes. You may want to write down key words

Questions to include

• Essential questions
• Focus of the study - questions that interest you
• Place them together or scatter them through out the interview

• Extra questions
• Equivalent to essential ones but worded differently- i.e. if one question was worded in a way not clear to the interviewee
• Included to check the reliability of responses or impact of changing the wording of questions
• Throw-away questions
• Used to develop rapport at the beginning of an interview – i.e. how was your day?
• Used to change the pace or focus on the interview – if the person needs to become more involved or to slow down [but you can't use something wild and unrelated]
• Demographic questions could be used as a throw away question- but some don't like to answer things like age, especially right in the beginning
• Probing questions
• Way to draw out more complete stories- i.e. can you describe this? How does this happen?
• Allows participants to elaborate on their responses

Questions to avoid

• Affectively worded questions
• Questions that are known to/or likely to arouse negative emotional responses – i.e. even subtly putting an emotional twist to your questions
• Neutralize the topic in order to improve the response. Sometimes, you have to build rapport before asking sensitive questions/phrases
• Complex questions
• Keeping questions brief and concise helps to ensure that respondent understands them
• Double-barrelled questions
• Asks a respondent to answer two questions simultaneously
• Results in confusion or failure to answer one or both questions
• Far better to ask two questions instead
• Wording
• Questions must be worded so respondents understand them and so they provide a necessary focus for the respondent
• Order
• Sequence of questions may affect responses
• Begin with non-threatening questions or demographics
• Develop rapport through demeanour and eye-contact

• what do you do about x
• describe your views on
• what are your views on
• what concerns you most about
• in your opinions, what is the public's view of
• What do you think are types of challenges people with have? how do you think that these challenges could be addressed

Questions to ask yourself and others about your interview questions (Manson, 1997)

• What does this question mean to me?
• Is the intended meaning easily understood?
• How would I ask this question of someone else?
• Might I encounter resistance to obtaining an answer to this question?
• How could I rephrase the question to reduce such resistance?

Small group work

1. on own, write 1 problematic interview question
2. discuss the problematic question
3. revise the question and share with group

1. so, did you actually abuse your child, and how could you?

October 9, 2008

Data collection

Components o the research design

• using philosophical and theoretical assumptions
• writing an intro to the study
• collecting your data
• analyzing your data
• employing standards of quality and verifying results
• writing the narrative

preparation for the research interviews

• before you do the actual interviews:

• be aware of your issues and where you stand on them [before you start your interviewing]
• prepare an information letter [introducing/study/inviting participants]
• prepare a consent form – one for you and one for them.
• Prepare a face sheet – to collect demographic data àhelps us organization – may sign a pseudonym/care number/etc…
• Develop an interview guide, including probes à i.e. a list of questions
• Get a tape recorder/tapes/batteries àmake sure they all work!
• Arrange a referral source [or participants] and debriefing [for yourself] àconfidentiality is paramount for the participant

Some interview tips –Berg, 1992, 57-58

• never start the interview cold àdepends where the interview takes place
• take a few minutes to establish yourself
• remember your purpose
• have the interview guide handy and stay on track àhelps your organization and time management
• present a neutral front
• know your questions well enough so you can ask them in a neutral manner
• demonstrate through non-verbal responses that you are listening ài.e. smile when something funny is said àpeople need to know that your are listening. Therefore, give same attention and energy levels as you did with first interview
• think about appearance and how you look to others
• dress appropriately for your interviewer
• interview in a comfortable place – select a place where the interviewee feels comfortable – i.e. not a place which the interview can be overheard, since its probably very personal
• do not be satisfied with single word answers – so you gotta ask “can you tell me more about that? What else happened?”
• be respectful – ensure that participant feels included and knows you value the answers à you may not agree with the answers, but you still need to be respectful
• Practice!!! Develop your own repertoire of action [style]
• Be cordial/appreciative: thank the participant and answer questions about the research

Using documents as a data source

• examples: administrative records [i.e. birth certificates] databases, reports [case reports/annual reports]
• provide contextual information àrepresents a particular reality àit is an important source of triangulation
• represents a specific version of reality constructed for specific realities and for specific purposes
• for quality assessment: authenticity/credibility/representativeness and meaning àyou have to note where the information came from for the benefit of anyone reading the report

Visual data

• photographs/film/video
• who decides what events/aspects worthy of being captured? Researcher has to think about that!!!
• Visual data vs. “the truth” àwhat is considered “the truth”?
• What is in the picture/what is left out/what is the focus? ài.e. is the media being altered?
• analysis is often in triangulated with other data collection and sources àstrengthens credibility of finding

participant observation

• doing fieldwork – what is participant observer?
• Ethnographic strategies
• Researcher for data collection that place researcher in midst of what they study
• Researcher's concern is to learn about and understand everyday life of a community, settings, actors, etc… from the perspective of the participants (Berg, 1995)
• Resulting narrative provides a thick description [verbal snapshot of what is going on – a different perspective] or cultural description [Wolcott, 1973] of some aspects of daily life

Gold's typology of observer roles – 1958

-roles are not static àpeople may move positions depending on interactions

• complete observer ài.e. watching a videotape of an intervention
• observer as participant
• participant as observer
• complete participant àmight lose a critical perspective/losing one's impartiality/maintaining some distance needed for impartiality.

how does the participant-observer role differ from our everyday role as an observer?

• dual purpose à you are participating, but also taking notes
• heightened or explicit awareness to task àwe have a purpose, so you are paining more attention to details àallows for richer note taking
• viewing setting as wider-angled lens
• Experience an insider/outsider tension while in field ài.e. are you a teacher or an observer? Might be a hard position to be in?
• Reflect on ‘presence of self’ àseeing how you might influence the outcomes
• Keep records of our observations

How do we select people/places to study?

• Does the setting complement your general interest? Will this answer your questions?
• Do you have easy access to the setting?
• How easy or difficult will it be to ‘blend in’? àthis is a tough thing to do!
• Is the setting sufficiently active or busy? Need to be able to collect data!
• What ethical issues arise in relation to your choice? Some sensitive areas, like ER has to be thought through, since it is a sensitive place

Recording information:
writing field notes

• Written account of what researcher

• Hears

• Sees
• Experiences
• Thinks

-in the course of collecting/reflecting upon data

àyou need to separate what you see and what you think/interpret. Some people write what is observed and what is interpreted on 2 sides of a page

Field notes

• Important supplement to other data collection methods (contributing toward triangulation, to be discussed later) àstrengthened dindings –field notes become a data source
• Help keep track of development of project
• Help to visualize how research plan has been affected by data collected
• Help you to be self-conscious of how you may have been influenced by (and may have influenced) data àkeep notes of what you specifically were thinking àkeep those notes!!!!!! This is important in qualitative research

Fieldnotes: Descriptive/Substantive and Reflective

• Descriptive aspect of fieldnotes is intended to capture a word-picture of setting, people, actions, conversations àcan encompass different ares, i.e. physical setting, snippets of dialogue

• Descriptive fieldnotes encompass the following areas:

• portraits of participants

• reconstruction of dialogue
• description of the physical setting
• accounts of particular events accounts
• depiction of activities
• observer’s behaviour

• Reflective aspect of fieldnotes is meant to capture observer’s frame of mind, ideas, concerns.
• Emphasis is on speculation, feelings, problems, ideas, hunches, impressions, prejudices

• This set of notes will contain:

• reflections on analysis
• reflections on method
• reflections on ethical dilemmas and conflicts
• reflections on observer’s frame of mind
• points of clarification

àhave a field log – source of data –needs to be methodical: data time/who the people are/methodological issues

Recording the information you gather

What are substantive field notes?

• your records from the field [jotting, key words, phrases, sketches, maps, etc…]
• log from which your analysis will emerge [Lofland and Lofland, 1995]
• description without interpretation

what are you looking for?

• physical space and setting
• actors and their words/actions
• interaction/relationships/emotions felt and expressed àor what you think they are

What should be in a field log? (Burgess, 1982)

1. Substantive field notes
• main observations, description of events
• details (names, dates, places, conversations, interviews)

Making the notes:

• Start your log ASAP, certainly within 24 hours
• Remember that 1 hour in field could produce 10+ pages
• Pay attention to language
• Make note of actual language used
• Note key words and phrases
• Move from condensed to expanded language
• Do not translate your notes (keep them in their original language)

-keep notes in your original language as you are listening àtranslate later àbetter to keep the original language

Clues for recalling data (Berg, 1995)

• Record key words and phrases in field
• Makes notes about sequence of events
• Limit amount of time you stay in setting
• Write your notes immediately after leaving setting àbefore sharing with colleagues
• Write your notes before sharing your experience à sharing can influence the way an event is recalled

Interview assignment

• Choose a partner for Assignment 3.
• Interview your partner for approximately 30 minutes based on your interview schedule.
• Take notes if you are not able to audiorecord the interview.

Dr. David Este – guest lecturer – focus group

Focus group

-generated by group – members have data – thoughts/emotions, etc…

Kruger: carefully planned discussion

• orienting oneself to a new field
• getting data about complex behavior
• evaluating different research sites/populations
• getting participant interpretation f results of previous studies or studies in progress
• generating information for questionnaire constructing

focus group myths

• they are cheap àactually they are not because it takes a long time to set up
• people speak about sensitive issues in group. Yeah right!
• Focus groups tend to produce conformity
• Must be validated by other ways of data collection

Characteristics of focus groups

• focus groups involve people
• focus groups is a data collection procedure
• focus groups make use of qualitative data

Advantages of focus group interviews

• socially oriented research procedure
• format allows the moderator to probe
• relatively low cost
• can provide speedy results
• increase the sample size of qualitative research

limitations

• researcher has less control in the group than individual therapist
• data is more difficult to analyze
• the technique requires training
• groups can vary considerably
• difficult to ensemble
• the discussion is focused

planning

• what is the purpose
• what is the right method
• identifying information rich participants
• how many groups to conduct

-gotta make sure that the place is set up comfortably – not in public. Some interview in two – best sitting across from each other – to see everyone around.

Hosting the group

• refreshment
• incentives
• other people coming with the participants

co-moderator:

1. roles

recording

• quality of equipment
• familiarization
• checking before and after group

recruiting participants

-gotta explain why you are doing this research – i.e. its purpose

Selection strategies

incentives

• money
• food
• gifts
• positive/upbeat invitation

quality of good focus group questions

• use they words
• clear
• short
• open ended question
• easy to state

àties in to “safe environment”

Examples of probes

• would you explain more
• anything else you would like to share
• is there anything else
• I do not understand

Beginning of the focus group discussion

• welcome
• overview of the topic
• the ground rules
• the first question

ethics

-ethics boards are part of such studies

-we need to make sure no hard is done to anyone

àconsent for is important. The downside is that it

Role of moderator

• facilitation of the discussion process
• encourage diversity of perspectives
• encourage discussion from quiet participants
• keep on track

moderator roles

• seeker of wisdom
• enlightened novice
• expert consultant
• challenger
• referee
• writer
• team discussion leader/technical expert
• the therapist

-need to be respectful to all participants ài.e. set the tone for entire session

-anticipating the flow of discussion

-moderator's reactions to respondents

Closing the session

-Wind up discussions

-thank the participants

-Describing follow-up/debrief

Capturing data

• video
• audio
• etc..

-be ready for the unexpected! i/e/ no one shows up/showing up with kids/etc…

October 23, 2008

Data analysis

• we do data analysis after we collect our data
• it involves breaking it down to manageable chucks/synthesizing/searching for patterns/discovering what is important to others
• it has to be meaningful to anyone we explain it to
• we can explain our goals/conclusions

• 2 modes of data analysis as pertaining to field work
• Concurrent to the fieldwork
• Start only after data collection

àin practice, it is best to mix between the two

• make analysis an ongoing part of data collection and be ready for final analysis as soon as leaving the field
• force yourself to make decisions that narrow the study
• force yourself to make decisions concerning the type of study that you want to accomplish
• develop analytic questions, substantive and theoretical questions [linking theory and practice]– it is advised for beginners to use substantive questions and leave theoretical questions for the final analysis
• plan data collection sessions in light of what you found in previous observations

analysis in the field:

• write as many observer's comments about ideas that you generate
• write memos to yourself about what you are learning
• try out your ideas and themes on subjects
• begin exploring literature when you are in the field
• play with metaphors/analogies/concepts
• use visual devices
• general tips [in and out of the field] –do not be afraid to speculate, vent energy created by mulling over ideas. Jot down lots of ideas while reviewing your data
• have a rest after data collection – as long as it is reasonable

developing coding categories

-important stage in data analysis

• you can use words/numbers
• influences by paradigm/literature/research ài.e. your background. Examples:
• situation
• process/words/sequence of events
• relationship/social structure

mechanics of working with the data

• having a scheme – regardless of which scheme, as long as you have a logical scheme – i.e. using color schemes
• making a list of codes – i.e. of their abbreviations
• go through data and marking each unit with appropriate coding
• important to number your pages/possibly number your paragraph and line too!
• Subdivide your themes and sub-sub divide your themes
• Variations on coding:
• Open coding [separate/non-applied codes, looking for the themes without pre-defining things]

Basic questions

1. what is it about here? What phenomena are mentioned here?
2. Who – which actors/persons are involved?
3. By which means/tactics/strategies are we going to reach our goals
4. How much? What is the intensity/strength?
5. Why? – what are the reasons which are given or could be reconstructed
6. What purpose? What are the intentions

Moving from codes to themes

-look for relationships between the codes

àwhich themes jump out?

àidentify strong and important quotes

Data processing and transformation from grounded theory

• Abstraction – prominent features/characteristics
• Generalizing –connect/link similar features/patterns
• Categorizing –link/sort/organize/connect together

October 30, 2008

Moving your analysis forward and writing up your results

Components of the research design

1. Using philosophical & theoretical frameworks

2. Writing the introduction to the study

3. Collecting data

4. Analyzing data

5. Employing standards of quality & verifying results

6. Writing the narrative report

Finding a balance

-Making sense of data entails movement from description to analysis to interpretation.

Description

What is going on here? Data consist of observations made by researcher and/or reported to researcher by others

Analysis

Addresses identification of essential features and systematic description of inter-relationships among them - in short, how things work.

In terms of stated objectives, analysis may also be employed to address questions of why system is not working or how it might be made to work better.

àmoving to higher level of abstraction – i.e. looking at other theories that exist

Interpretation

-Addresses questions of meanings and contexts - What does it all mean? What is to be made of it all?

• Turn to theory - examine case in terms of competing theories
• Connect with personal experience - this is how research affected me
• Analyze interpretive process - What is puzzling? What is missing?
• Interpret analytic process – introspection
• Explore alternative formats (media, poetry, etc.)

• Extend the analysis - offer comparative perspectives
• Mark and make the leap - interpret (with caution) despite uncertainty
• When you come to end, stop - go as far as you can with confidence and them stop
• Do as directed - work toward immediate goal (thesis passed, article published)

Basic definitions

• What is a concept? – looking for themes
• Units of abstract thought which identify common aspects of phenomena.
• What is theory? – abstracting from the themes
• Abstract knowledge which has been developed as an account regarding a group of facts or phenomena.

Using theory in your analysis

Theory directing – testing a theory

Using pre-chosen theoretical positions that direct your research and against which you place your findings

Theory generating

• Draw range of theoretical frameworks from literature

Analytic strategy: visual depictions (Strauss & Corbin, 1998)

• Visual representation of something – good for visual people
• Integral to analysis process
• May be used to illustrate categories or themes
• Forms of depiction
• Typology (what are the topic’s types?)
• Matrix (rows & columns)
• Concept charting (envisioning relationships)
• Flow chart (time sequence)
• Tree diagram (connections)

CONSTRUCTING A THEORETICAL NARRATIVE FROM TEXT
(Auerbach & Silverstein, 2003, 42-76)

PHASE I: Making text manageable

• Working at level of text itself – make clear your connections how your data and conclusion
• Filtering process in which you choose which parts of your transcript you will include in your analysis and which parts you will set aside
• In this phase, you use your research concerns to select relevant text

Step 1: Explicitly state research purpose and theoretical framework.

• Your research purpose is what you want to learn about and why.
• Simple act of explicitly stating your research concern focuses you on what you want to know and why. Some text will be left out –it narrows the focus. Pulling themes helps this àwhat you want to learn about and why
• Your research purpose provides a blueprint for making coding decisions.

Step 2: Select relevant text for further analysis. You need to cut down mass of raw data to manageable proportions

• Method is simple and direct - when you encounter passage that contains an idea relevant to your research concerns, “highlight” it
• Continue in this way until you have worked through entire transcript
• Relevance is subjective; keep in mind when reading transcript: - if it is relevant, note it, and how it relates to your research purpose
• Does it relate to your research purpose?
• Does it help you understand participants better?
• Does it clarify your thinking?
• Does it simply seem important, even if you can’t say why?
• Can approach highlighting by 1) quickly and impressionistically, trusting that meaning of what you are doing will be clear in the end, or 2) more deliberately, writing memos to yourself each time you underline a passage, recording why you thought selection was important and any thoughts or ideas stimulated by text

Step 3: Discover repeating ideas by grouping together related passages of relevant text

• A repeating idea is an idea expressed two or more times in the relevant text.
• These are beginning building blocks from which you will eventually assemble a theoretical narrative
• Aim is to create a master list of repeating ideas
• Name your repeating ideas, using short quotes from relevant text, in participants’ words as much as possible

PHASE II: Hearing what was said

Step 4: Organize themes by grouping repeating ideas into coherent categories.

• A theme is an implicit idea or topic that a group of repeating ideas has in common
• Reduce # of repeating ideas to average of 15
• Name your themes with easily understood phrases that express common thread (i.e., conceptual similarity between repeating ideas)
• Once you have identified all themes, check out your work with a member of group you are studying who is not involved in your project
• Working at a more abstract level to group themes into more general concepts called theoretical constructs
• A theoretical construct is an abstract concept that organizes a group of themes by fitting them into a theoretical framework
• Use constructs to create a theoretical narrative

• Code book is a master plan of codes and what they mean
  

PHASE III: Developing theory

Step 5: Develop theoretical constructs by organizing themes into more abstract concepts.

• Theoretical constructs move analysis from description of subjective experience found in repeating ideas and themes to a more abstract level
• Name your theoretical constructs: When grouping your themes, you had thoughts about organizing principle(s) that hold themes together. Recall these and now draw on literature and your general knowledge to find a theory or concept that explains this organizing principle - names of theoretical constructs should use language of the theory on which you draw
• Verify your work with consultants, make adaptations as relevant, that have emerged from their feedback

Step 6: Create theoretical narrative by retelling participant’s story in terms of theoretical constructs.

• A theoretical narrative – this is important for qualitative research – it makes sense of the stories, and makes it scientific valid
• Describes process that research participants reported in terms of your theoretical constructs.
• Uses your theoretical constructs to organize people’s subjective experiences into a coherent story.
• Employs people’s own language to make their story vivid and real.
• You create narrative by organizing your constructs into personal story that describes subjective experience of research participants
• Begin narrative by describing research concerns then describe first theoretical construct as it applies to research participants

Break down construct into its thematic experiential components, using language of repeating ideas to make experience vivid and real - using repeating ideas tells the story in participant’s own words.

• Do this with each theoretical construct

Determine your daily writing goal

• sit there method - write for a certain amount of time every day
• inspiration method - write until you come up with three ideas
• many pages method - write a certain number of pages every day

thinking on paper

• Freewriting
• Write whatever comes into your head, as fast as you can type, without reference to outlines, notes, data, books, or any other aids.
• Object is to find what you would like to say, what all your earlier work on topic or project has already led you to believe.
• Learn to write in order to think, to encourage thought, to tease thought out of chaos or out of fright.
• Start with a small task, learning to write for 15 minutes a day, no matter what.

• Making a mess
• When you sit down to begin writing, aim to “make a mess” - to say anything that comes to mind, on the subject or off it
• Don’t worry about logical connections
• “Play with your subject the way you used to make mud pies, to do no fine detail work, to spell poorly if that’s your natural inclination, and to generally forget about standards all together.”

• Keep the flow going
• If the writing doesn’t sound good to you while you’re writing, it’s fine to make a note to yourself about this.
• Put that commentary in midst of text, using square brackets, or a different colour ink or pencil, so that when you come back to revise, you can recognize and engage quickly with problems already noted.
• Don’t stop to hunt for words. If you can’t get just the right word, list 3 or 4 alternatives, choices, words. Use thesaurus while working on second draft, when doing that won’t threaten to interrupt flow of thoughts and feelings (Fine, p. 35).
• Writing as thinking
• You have already made many choices when you sit down to write but may not know what they are.
• Can lead to confusion—a good thing!—and to mixed-up early draft.
• Knowing that you’ll write many more drafts, don’t worry about one draft’s crudeness and lack of coherence. Think: “This one is for discovery, not for presentation”
• You begin to notice you are coming back to the same ideas in different forms; this lets you see what’s there; it makes sorting out your thoughts easier and lets you see what you want to say (Fine, p. 56).

Myth of the perfect first try (Fine, p. 49)

• Face your fears and the risks of writing
• Opening yourself to scrutiny involves coming to trust yourself and colleagues
• Constructive feedback from colleagues makes it possible for you to trust yourself
• Why is it hard to trust colleagues?
• Who do you think you are?
• Every piece of writing is evidence of your work
• Denigrating others can ease our own insecurities
• Comments made about us (professional image)
• Sharing drafts is risky…few people understand what working drafts are

Develop your own work process

• Devise strategies that suit who you are, how you work
• Take your work habits seriously and scrutinize them to see if they need revamping
• Ask yourself what kind of research by other people you find most interesting and enjoyable to read
• Might mean finding a good model in a related area

Once you start

Internal Interruptions:

• Ambivalence
• Part of you wants to finish, other does not, nothing happens—recognize it
• Static
• Unrelated thoughts, feelings, other distractions pass through your mind
• Try focusing techniques such as meditation
• By giving static bits of time, you might prevent it from taking all your time
• Writing scared
• Anxiety can be healthy/unhealthy
• Use this and continue to write
• Learn to work despite your anxiety
• You don’t have to get un-scared first

Tips for writing

Park on downhill slope to keep the flow going

• Writing on downhill slope on Monday is a wonderfully simple way to jump-start your writing on Tuesday because you’ve already done hardest part of beginning: you’ve decided what you are going to write about.
• If you change your mind partway through, just write about your new topic and make sure you park on downhill slope at the end of Tuesday (Fine, p. 97).
• Frequency is key (if you lose it, it takes half a day to re-enter)
• Use positive reinforcement - reward yourself each step of the way as you accomplish small goals
• Set realizable goals: better 2 pages than 10 (too overwhelming)
• Write even if you feel tired or have nothing to say

deadlines

• Deadlines are essential
• Focus on estimated date of completion
• Remain aware of deadlines, know how you work and what you need

Moving from Zero to a first draft

• Pick out words or sentences that seem interesting, provocative or resonant and try starting with them
• Ask yourself what stands out for you in what you’ve written
• Read to organize, marking potential themes with codes or colours
• Identify argument within the mess
• Ask yourself: Do I think what I’ve said is what’s going on? Do I still believe it?
• Try writing repeatedly in 5-10 minute spurts of free writing...what am I really trying to say in this argument, chapter, section?
• Read to extract a provisional outline

Techniques for exploring ideas (Hacker, 1996) – organizational tools

• Listing
• Brainstorm ideas as they occur to you
• Clustering
• Write your topic in centre of sheet of paper, draw a circle around it, and surround that circle with related ideas connected to it with lines
• If some of satellite ideas lead to more specific clusters, write them down as well
• Branching
• Put main idea at top of the page and list major supporting ideas beneath it, leaving plenty of space between ideas
• To right of each major topic, branch out to minor ideas, drawing lines to indicate the connections
• If minor ideas lead to more specific ideas, continue branching
• Asking questions
• Ask relevant questions to generate many ideas: who, what, when, where, why, how
• Goal statement
• Write one statement that summarizes the paper and what you intend to cover

Audience

• Determining the target audience
• One audience vs multiple audiences - thesis readers, community
• Type of audience - course, colleagues, policy makers, etc.
• What will the report be used for – stored in a filing cabinet, practice guidelines, policy, etc.
• Common presentation style for target audience – academic paper, practice report, creative medium
• Reaching the audience
• Structure text so that it appeals to the audience
• Examples of such structures: lay meanings, popular terms, snazzy titles, footnotes
• For publication, devices such as display of academic credentials, references, footnotes, methodology section, familiar academic metaphors and images
• Approach depends on target audience.

Getting support

• Types of support
• sounding board, reading drafts, etc.
• Trusted friends/colleagues, one or two people to pass things to
• be willing to return favour!
• Formal group
• down the line and for longer projects – i.e. advisory boards

• Support group: questions to consider
• Have you reached a point where you’re ready to talk about your work?
• Do you know what to expect from such a group? Are these expectations realistic?
• Are you looking for students inside your department or outside?
• How frequently do you want to meet?
• How large a group do you want?
• Should the group have a leader or not?
• Have you decided where to look for such a group?

Revising

Types of revision

1. Reduce
• shorten chapters, sections & paragraphs; condense or move sentences; replace longer words with shorter ones
2. Rethink and rearrange
• Tone, verb tenses, parallel structure, word order, emphasis
3. Reword
• According to personal style, best word…active verbs, rhythm and sound, figurative language

Revision strategies

• One chapter at a time, then struggle with whole
• Consider revising introduction and conclusion last
• When unsure of argument or shape of chapter, try making outline of what you have
• Use outline on a smaller scale: try reducing each paragraph to a sentence - when you can’t do it, you’ll discover paragraphs are fuzzy
• Leave editing at individual word level until last (avoid wordsmithing)
• Leaving smooth transitions at later stage
• Saddest rule of editing: less is more
• Use your ears - reading it aloud is a good way to find sentences that tie themselves in knots
• Use your breath: 1-page paragraph is daunting to read
• Have a proof reader look for phrases you’ve overused or repeated arguments
• Watch for writing quirks (e.g., overusing punctuation)
• Don’t use complex words or jargon when simple words will make the point just as well
• Read it repeatedly—it will never be perfect and you will have to quit fiddling with it (hard for the perfectionists among us…)

Author representation: questions of concern – i.e. how will you deal with a different social location

• How much of the self is present in the narrative?
• What is the authorial stance of the writer (i.e., location)?
• What biases, values and contexts have shaped narrative?
• Have these been disclosed?
• Author may be present in following ways: epilogue, comments on role as researcher, etc. (reflective footnotes, interpretive commentaries)

quotes

• Short eye catching quotes àshort take as little space as possible
• Take up little space, stand out from narrator’s text but verify it
• Embedded briefly quoted phrases – APA has a standard about this
• Prepares reader for a shift in focus
• Displays a point and allows reader to move on
• Longer more complex quotes
• Give a sense of participant’s voice
• Use to convey more complex understandings
• Difficult to use sometimes (e.g., space limitation in publications)

Telling a story

Desirable features of a qualitative report (Lofland & Lofland)

• Basic organization & presentation – you might want to go by chronology [life in the day of ____, theme, around an event]
• Data and methods – quality
• Analysis – quality
• Overall evaluation - overall value of study

Ways of telling a story

• Chronological order - events as they occurred
• Narrator order - informant’s way of unveiling their life story
• Progressive focusing - from broad context to particulars or vice versa
• Day in the life of - reconstruction of a day
• Critical or key event - focus on one or two groups in interaction
• Follow an analytic framework - impose structure on account
• Demonstrate that there are a number of varying accounts

Qualitative data analysis software

• NVIVO 7
• HyperResearch
• Atlas/Ti
• NUD*IST N6
• Ethnograph 5.0

Nov 6, 2008

We will speak about research design and valid [credible]

Components of the research design

1. Using philosophical & theoretical frameworks – describes the planning [proposal] before the actual research àcould be for a sponsor/donor

2. Writing the introduction to the study

3. Collecting data

4. Analyzing data

5. Employing standards of quality & verifying results àmaking sure that our study is credible

6. Writing the narrative report

Different methodological approaches (Auerbach & Silverstein, 2003)

Quantitative:

• Tries to exclude subjectivity, interpretation and context from scientific practice
• Requires that data analysis procedures be ‘objective’, that theories be universally applicable
• Translates requirements of objectivity and universality into statistical concepts àlooks for probability sample and looks for statistical significance in order to generalize
• Objectivity corresponds to concepts of reliability and validity
• Universality corresponds to concept of generalizability – generalizability is paramount to quantitative research

Qualitative:

• Views subjectivity, interpretation, context as essential elements of research practice inextricably interwoven into every study
• Argues for evaluation standards consistent with qualitative research paradigm
• Replaces quantitative concepts of reliability, validity, generalizability with qualitative terms of dependability, credibility, transferability

Standards of qualitative research (Howe & Eisenhardt, 1998)

• Do research questions drive data collection and analysis rather than opposite?
• Are data collection, analysis completed in technical sense?
• Are researcher’s assumptions made clear?
• Does study have overall warrant?
• Does study have value in informing and improving practice?
• Does study protect confidentiality, privacy and truth telling of participants?

Standards of qualitative research -(Lincoln, 1995)

• Three commitments in qualitative research paradigm:
• To emergent relations with respondents
• To a set of stances
• To a vision that enables and promotes justice

Credability

• Having guidelines: Rigor, communication, way of working toward consensus; these guidelines also serve to legitimate social science research knowledge
• Positionality: Text should display authenticity about its stance and position of author – yeah! No hiding your position
• Community/context: All research takes place in, is addressed to and serves purposes of community in which it is carried out (e.g., feminist thought, black scholarship, Indigenous studies, etc.)
• Voice: Qualitative or interpretive research must give voice to participants so that their views are not silenced, disengaged, or marginalized; Multiple voices need to be heard in a text
• Critical subjectivity Researchers need to have heightened awareness in research process and work toward personal and social transformation; enables researcher to understand psychological emotional states before, during and after research
• Reciprocity between the researcher and those being researched; this means intense sharing, trust and mutuality
• Sacredness Of relationship in research-to-action continuum; this means that researcher respects collaborative and egalitarian aspects of research and makes space for the others’ ways of being
• Sharing of privilege: Researcher shares rewards with persons whose lives they portray; sharing may be in form of royalties or publication rights

“Validity”/Credibility (or, how might I be wrong?) (Maxwell, 1996, 86-98)

• Alternative conceptions of validity in qualitative research paradigm are a response to concern with accuracy in empirical sciences.

• Do the findings and conclusions represent reality?
• Do constructs measure actual experience?

• Validity refers to correctness or credibility of a description, conclusion, explanation, interpretation or other sort of account.

Types of validity/credibility in qualitative research

• As a component of your research design, validity consists of strategies you use to rule out threats to credibility of your study (i.e., ways that you might be wrong).
• Each category of understanding in qualitative research - description, interpretation, theory, generalization and evaluation - has its own type of research question.
• Descriptive questions ask about what actually happened in terms of observable behavior or events.
• Interpretive questions, in contrast, ask about meaning of these things for people involved, their thoughts, feelings and intentions.
• Theoretical questions ask about why these things happened, how they can be explained.
• Each of the main types of understanding has distinct “threats” to its validity/credibility

• Description
• Interpretation
• Theory

Main threat in description

• Main threat to credible description, in sense of describing what you saw and heard, is inaccuracy or incompleteness of data
• If your description of what you were observing, or of interview you conducted, is invalid, then interpretations or conclusions you draw from these descriptions are questionable
• Audio or video recording of observations and interviews, and verbatim transcripts of these recordings, largely solves this problem
• observational notes need to be as detailed, concrete and chronological as possible

Main threat in interpretation

• Main threat to credible interpretation is imposing one’s own framework or meaning rather than understanding perspectives of people studied and meanings they attach to their words and actions – i.e. is it your ideas or the participant's voice being herd?
• Most important check on such credibility threats is to seriously and systematically attempt to learn how participants in study make sense of what’s going on rather than pigeonholing their words and actions in own framework
• Member checking is one of main strategies for avoiding this threat - this means that you go back to the participants and ask them if this was meant by them

Main threat in theory

• Most serious threat to theoretical credibility of an account is not collecting or paying attention to discrepant data or not considering alternative explanations or understandings of phenomena you are studying

àso you can't say it's statistically valid or objective. But it does reflect what is going on.

Specific threats to credibility:

Researcher bias

• Temptation to filter one’s observations and interpretations through lens overly influenced by preconceptions and opinions
• Limiting selection of data to fit researcher’s existing theory or preconceptions
• Impossible to deal with these problems by eliminating researcher’s theories, preconceptions or values - this impossibility is one aspect of inherent reflexivity of qualitative research
• Qualitative research is primarily concerned with understanding how a particular researcher’s values, etc. influence conduct and conclusions of study - explaining your possible biases and how you will deal with these is key piece of your research proposal

Reactivity

• Potential effects of researcher’s presence on setting or individuals studied
• Eliminating actual influence of researcher is impossible - goal in a qualitative study is to understand it and use it productively
• For participant observation studies, reactivity is generally not as serious a validity threat – in natural settings, observer is generally much less of an influence on participants’ behavior than is setting itself
• For interviews, in contrast, reactivity is a powerful and inescapable influence; what respondent says is always function of interviewer and interview situation
• Although there are some things to prevent more undesirable consequences of this, such as avoiding leading questions, trying to minimize your effect is not meaningful goal for qualitative research
• What is important is to understand how you are influencing what the informant says, and how this affects validity of inferences you can draw from interview

Procedures for a credible study

• Although methods and procedures do not guarantee credibility, they are nonetheless essential to process of ruling out credibility threats and increasing credibility of your conclusions
• The following strategies primarily operate not by verifying conclusions, but by testing credibility of your conclusions and existence of potential threats to those conclusions
• Fundamental process in all of these tests is trying to find evidence that challenges your conclusion, or that makes potential threat implausible

Prolonged engagement and persistent observation

• Build trust, “learning” the culture, checking for misinformation

Triangulation

• Make use of multiple and different sources, investigators and theories to achieve a comprehensive picture
• Triangulation reduces risk of chance associations and of systematic biases due to specific method and allows better assessment of explanation that you develop

Peer review or debriefing

• Solicit feedback from others is a useful strategy for identifying credibility threats, your own biases, assumptions, flaws in your logic or methods
• Try to get feedback from those familiar with the phenomena or settings you are studying those who are strangers to situation

Member checks

• Researcher takes analysis, interpretations and conclusions back to respondents who should be asked to provide alternative language, critical observations and interpretations
• Single most important way of ruling out possibility of misinterpretation of meaning of what they say and perspective they have on what is going on
• Important not to assume that participants’ pronouncements are necessarily valid; their responses should be taken simply as evidence regarding validity of your account

Searching for discrepant evidence and negative cases

• Identifying and analyzing discrepant data and negative cases is key part of attempt to falsify proposed conclusion
• Instances that cannot be accounted for by particular interpretation or explanation can point up important defects in that account
• Basic principle is that you need to rigorously examine both supporting and discrepant data to assess whether it is more plausible to retain or modify conclusion, being aware of all pressures to ignore data that do fit your conclusions
• In particularly difficult cases, best you maybe able to do is to report discrepant evidence and allow readers to evaluate and draw their own conclusions

Clarifying researcher bias

• Reader must be able to understand researcher’s position
• Researcher comments on past experiences, biases, prejudices and orientations that have shaped interpretation

Audit trail and external audits

• Document each step, leave a trail
• Have an external consultant or editor examine process and account, assessing their accuracy

‘Rich’ data

• Data that are detailed and complete enough that they provide a full and revealing picture of what is going on
• Such data counter dangers of respondent duplicity and observer bias by making it difficult for respondents to produce data that uniformly support a mistaken conclusion, just as they make it difficult for observer to restrict his observations so that she/he sees only what supports her or his prejudices, expectations
• Key function of rich data is to provide a test of one’s developing theories, rather than simply a source of supporting instances

Generalizability/Transferability

• Qualitative researchers typically study single setting or small number of individuals or sites, using theoretical or purposive sampling rather than probability sampling
• Qualitative researchers rarely make explicit claims about generalizability of their accounts but may refer to the circumstances in which the accounts may be transferable

Transferability

• Internal transferability, which is key issue for qualitative case studies, refers to transferability of conclusion within setting or group studied
• External transferability refers to its transferability beyond that setting or group
• Descriptive, interpretive and theoretical validity of conclusions all depend on their internal transferability to case as a whole (e.g., if you’re studying patterns of interaction between teacher and students in single classroom, your account of classroom as whole is seriously jeopardized if you have selectively focused on particular students or kinds of interactions and ignored others
• In contrast, external transferability (generalizability) is often not crucial issue for qualitative studies
• Value of study may depend on its lack of external transferability in sense of being representative of a larger population; it may provide account of a setting or population that is illuminating as extreme case or ideal type
• Doesn’t mean qualitative studies are never generalizable beyond setting or informants studied; certain characteristics provide credibility to generalizations
• Studies often have face transferability—there is no obvious reason not to believe that the results apply more generally
• Transferability of studies usually is based not on explicit sampling of a population to which results can be extended, but on development of theory that can be extended to other cases
• Features that lend plausibility to transferability from case studies or nonrandom samples include respondents’ own assessments of transferability, similarity of dynamics and constraints to other situations, presumed depth or universality of phenomenon studied, and corroboration from other studies

Criteria for the justifiable use of subjectivity to interpret data

Transparency

• Means that other researchers can know steps by which you arrived at your interpretation; does not imply agreement--only that they know how you arrived at it. If your interpretation of data is transparent, it means that it is justifiable, that you did not simply make it up to suit yourself.

Communicability

• Means that your themes and constructs can be understood by, and make sense to, other researchers and to the participants themselves; does not imply agreement, only that what you have done can be understood

Coherence

• Means that your theoretical constructs fit together and allow you to tell a coherent story; does not mean that story you develop is only one possible, but rather that your story helps to organize data

Questions for discussion on standards, verification, credibility

How would you judge a qualitative study?

• voice of participants is central
• stated limitations
• stated methods, process (how study evolved)
• honest reflections of researcher
• match between persons selected and research topic
• connections to field (builds on or questions what is happening)
• well supported claims
• negotiated/reflective piece (were your questions answered; if not, why not/ how did you revise or account for this?
• curiosity
• well written

How do you know an account is accurate?

• have a colleague review it
• have participants review it
• use case examples (in interviews)
• ask questions in different ways (language, etc.)
• clarify and/or rephrase in interviews

What makes for a credible study?

• allow participants space to discuss their experiences
• check back with participants
• note process and methods
• solicit information from colleagues and ‘knowledgeable’ persons
• reflect on your past and present experiences/social locations/beliefs
• develop a transparent process
• methods (recorded interviews, transcription and use of quotes)
• extended involvement….trust
• discuss limitations of study

And still more questions to consider…

• How do you know if the study is valid/credible? To whom?
• What is your position or social location (insider/outsider)? How does this affect your thinking?
• How have you addressed power? Inclusion? Diversity?

November 13, 2008

Participatory Action Research/ Community Action Research
Fusing research and social justice

Participant Action Research -Some definitions of PAR

• A method of social investigation of problems involving participation of “ordinary” people in problem posing and solving
• An educational process for the co-participants (researcher and participants) who analyze the structural causes of named problems through collective discussion and interaction
• A way for researchers and people marginalized from access to resources to join in solidarity to take collective action, both short and long term, for (radical) social change

Community Action Research -Some definitions of CAR

• Useful, action-oriented, community-based research approach for change
• An approach to research that can promote self-discovery of individuals, build communities, serve as catalyst for social change
• A value-driven activity founded upon integrated set of concepts that, taken together, provide direction for planning, implementation, and utilization of CAR

PAR is not “a” research method

• PAR is a “systematic investigation, with the collaboration of those affected by the issue being studied, for the purposes of education and taking action or effecting social change”
• PAR is an approach to research partnerships and applications.
• PAR is about power whose location in various stages of the research process is key.

PARTICIPATORY ACTION RESEARCH

Comparing Participatory & Conventional Research

àthis approach allows for people to take ownership over the project at hand

SWOT analysis: analysis: strength/weakness/opportunities/threats

Co-Participant/ Co-Researcher

• Community participants engage in all stages of research process (decision to do research, identification of research focus, selection of methods, implementation of inquiry, analysis, interpretation, synthesis, verification of conclusions, and decisions to take action). Attention is given to reduce barriers to participation, especially for those who have been excluded or under-represented in past.
• Formally trained researcher stands with and alongside community, not outside as an objective observer or external consultant. Researcher contributes expertise when needed as a participant in process. Community participants contribute their physical and/or intellectual resources to research process.

November 20, 2008

Sampling:

1. purposeful/theoretical sampling – think about who you want
2. convenience sampling – based on what is available – but you also have to note the downsides of it
3. snowball sampling – find a suitable respondent for further contacts

data collecting

1. individual interviewing
2. key informants
3. focus groups
4. visual data
5. participant observation
6. reviewing documents
7. writing a personal field log

àby using more kinds of data collection, one increases triangulation

Data analysis

• open coding (separate/non-applied codes)
• line-by-line coding – not theme
• constant comparison – grounded theory
• initial codes – first steps
• analytic codes [more precise codes]
• pre-determined themes [theme of the paper]

Resolving field issues: research ethics

1) Ethics of concealment: deception & disclosure

• range from disclosure - neglect – deceit
• neglect (not telling people they are being observed)
• full deceit (concealing the study & investigator’s role)

2) Confidentiality & privacy

• no guarantee of complete anonymity - respondents are known
• never reveal the identity of respondents
• exception to this is a legal requirement (e.g., child abuse protection)

3) Distress & emotional harm

• principle of least harm
• make arrangements for professional referral prior to your study
• refer but leave service provision to others

4. Informed consent: research with vulnerable populations

Elements of informed consent:

• description of study &procedures (duration, interviews)
• identification of researcher and organization (include address & phone number for future contacts)
• assurance of voluntary participation
• assurance of confidentiality
• knowledge of risks/benefits associated with the study
• special precautions for persons under 18
• signed consent may be waived to protect identities & individuals
• Have 2 copies when consent is given (for you and for respondent).

5. Coercion & deformed consent

• be aware of the potential for coercion
• assure respondents they can withdraw at any time
• researchers should avoid studying clients & colleagues

6. Incentives & payback

• most funded studies involve remuneration for respondents’ time
• too little is not an incentive; too much risks coercion

7. Risks to the researcher

• emotional concern - ups & downs of doing fieldwork
• exhaustion
• reactions/involvements - comments may strike a nerve
• threats to physical safety, innuendos

8. Power

• consider ethical implications of power relations
• pay attention to power as a service provider (research linked to agencies)

Specific threats to credibility:
Researcher bias

• Temptation to filter one’s observations and interpretations through lens overly influenced by preconceptions and opinions
• Limiting selection of data to fit researcher’s existing theory or preconceptions
• Impossible to deal with these problems by eliminating researcher’s theories, preconceptions or values - this impossibility is one aspect of inherent reflexivity of qualitative research
• Qualitative research is primarily concerned with understanding how a particular researcher’s values, etc. influence conduct and conclusions of study - explaining your possible biases and how you will deal with these is key piece of your research proposal

Procedures for a credible study

1. Prolonged engagement and persistent observation
• Build trust, “learning” the culture, checking for misinformation
2. Triangulation
• Make use of multiple and different sources, investigators and theories to achieve a comprehensive picture
• Triangulation reduces risk of chance associations and of systematic biases due to specific method and allows better assessment of explanation that you develop
3. Peer review or debriefing
• Solicit feedback from others is a useful strategy for identifying credibility threats, your own biases, assumptions, flaws in your logic or methods
• Try to get feedback from those familiar with the phenomena or settings you are studying those who are strangers to situation
4. Member checks
• Researcher takes analysis, interpretations and conclusions back to respondents who should be asked to provide alternative language, critical observations and interpretations
• Single most important way of ruling out possibility of misinterpretation of meaning of what they say and perspective they have on what is going on
• Important not to assume that participants’ pronouncements are necessarily valid; their responses should be taken simply as evidence regarding validity of your account
5. Searching for discrepant evidence and negative cases
• Identifying and analyzing discrepant data and negative cases is key part of attempt to falsify proposed conclusion
• Instances that cannot be accounted for by particular interpretation or explanation can point up important defects in that account
• Basic principle is that you need to rigorously examine both supporting and discrepant data to assess whether it is more plausible to retain or modify conclusion, being aware of all pressures to ignore data that do fit your conclusions
• In particularly difficult cases, best you maybe able to do is to report discrepant evidence and allow readers to evaluate and draw their own conclusions
6. Clarifying researcher bias
• Reader must be able to understand researcher’s position
• Researcher comments on past experiences, biases, prejudices and orientations that have shaped interpretation
7. Audit trail and external audits
• Document each step, leave a trail
• Have an external consultant or editor examine process and account, assessing their accuracy
8. ‘Rich’ data
• Data that are detailed and complete enough that they provide a full and revealing picture of what is going on
• Such data counter dangers of respondent duplicity and observer bias by making it difficult for respondents to produce data that uniformly support a mistaken conclusion, just as they make it difficult for observer to restrict his observations so that she/he sees only what supports her or his prejudices, expectations
• Key function of rich data is to provide a test of one’s developing theories, rather than simply a source of supporting instances

APA:

Number pages

No contraction

Left justification ànot full

No spaces between the paragraphs – not even before subtitle

Indent first line

References – double spaced

Second lines and onwards - indented

END of COURSE!