Lecture_2

Question 1

What is qualitative research?

Answer 1

• Non-numerical data: Text, image, video, artefacts.

Question 2

Why use qualitative research?

Answer 2

• To gain an in-depth understanding of a phenomenon in its context
• To guide managerial decision making and the development of new theories; to allow the “real”
  world inform theorizing and decision-making
• To elaborate or extend existing theory
• new concepts, processes, or relationships between concepts, theory applications
• To get new insights
• In practice: new trends, customer needs, untapped market potential
• To make (more) sense of quantitative data and results
• Understanding trends in data, e.g., “Why do we observe a decline in customer satisfaction?”
• Explain “strange” results (statistical outliers) and contradictions

Question 3

Sampling in qualitative research

Answer 3

• Purposive sampling: Theory-driven.
• Small samples: Contextual depth.
• Evolves: Participants suggest others.

Question 4

Depth interviews

Answer 4

• Objective: Understand experiences, beliefs, perceptions.
• Formal: Usually >1 hour.
• Core activity: Standalone or combined.

Question 5

Types of qualitative data

Answer 5

• Archival data: Written, visual materials.
• Interview data: Transcripts.
• Observational data: Field notes, videos.

Question 6

Projective methods

Answer 6

• Goal: Indirectly reveal feelings.
• Examples: Word association, sentence completion, symbolic matching.

Question 7

Focus groups

Answer 7

• Participants: 6-12, homogeneous.
• Setting: Relaxed, informal.
• Strength: Interaction adds richness.

Question 8

Qualitative data analysis

Answer 8

• Coding: Categorization for patterns.
• Iterative: Start early, refine codes.
• Tools: NVivo, Atlas.ti, MAXQDA.

Question 9

Linking qualitative and quantitative data

Answer 9

• Qualitative: Conceptual development.
• Quantitative: Representative sampling.
• Together: Clarify and validate findings.

Question 10

Applications of LLMs in qualitative research

Answer 10

• Automated interviewing: Scalable, reduces bias.
• Sentiment analysis: Nuance detection.
• Synthetic data generation: Privacy-compliant.

Question 11

Key Drivers of the Current AI Revolution

Answer 11

• Massive increase in the amount
  of data available worldwide
• Technological progress in GPUs
• Development of the transformer
  architecture
• 135,017 citations
  ➢ Of these, citations in the top 50
  management journals: 6

Question 12

Basic Principle of Large Language Models

Answer 12

1. Tokenization – The input sentence is split into smaller parts (tokens), such as words or characters.
2. Embeddings – Each token is converted into a numerical vector, where similar words have closer values.
3. Sequence Processing – The model processes the sequence of embeddings to understand the context.
4. Prediction – The model predicts the next word based on probabilities, generating meaningful text.

Question 13

LLM and Temperature

Answer 13

In the context of Large Language Models (LLMs), temperature is a parameter that controls the randomness of predictions when generating text.

🔥 How Temperature Works
* It adjusts how confident or creative the model is when choosing the next word.
* Affects the probability distribution of the next token in a sequence.

📊 Temperature Settings:
* Low temperature (e.g., 0.2) → The model is more deterministic and picks the most likely words (useful for factual responses).
* High temperature (e.g., 1.2) → The model becomes more creative, selecting less likely words more often (useful for storytelling or brainstorming).
* Temperature = 0 → The model will always choose the highest probability word, making responses predictable.

🎯 Example:

If the model is predicting the next word in “It was a dark and…”:
* Temperature = 0.2 → “stormy”
* Temperature = 1.0 → “mysterious” / “gloomy” / “cold”
* Temperature = 1.5 → “exciting” / “adventurous” / “peculiar”

💡 Lower temperatures = better for accuracy, higher temperatures = better for creativity!

Question 14

Stochastic Parrots

Answer 14

• Training Data Bias: LLMs inherit biases from datasets (e.g., gender and demographic imbalances).
  
  • Evaluation Challenges: Bias is subtle and requires careful analysis.
  • Automated Bias: Human biases get amplified in AI-generated text.
  • Marginalization Risks: LLMs may disadvantage certain groups.
  • Illusion of Understanding: LLMs don’t “think”—they just predict words probabilistically.

Outcome & Impact:
* Sparked global debate on AI ethics, bias, and corporate influence in research.
* Raised concerns about Google’s commitment to responsible AI.
* Led to more focus on transparency and fairness in AI development.

Question 15

Multimodal AI

Answer 15

Multimodal AI refers to artificial intelligence systems that can process and understand multiple types (or “modes”) of data, such as text, images, audio, and video. This makes them more advanced than traditional AI, which typically focuses on a single modality (e.g., just text or just images).

Case study: Customer Value Prediction with Multimodal Data
take -> Product descriptions, Product images, Transaction data and put into AI (Transformer) -> get Customer Value
Case Study: Predicting Performance for LinkedIn Posts
Text of a LinkedIn post + Picture of a LinkedIn post -> into Embeddings and Machine Learning -> get:
* # Likes
* # Clicks
* # Shares

Question 16

AI and Customer Service

Answer 16

➢ Replaceable activities of employees in customer service: 63% - 80 % (own calculations)

Question 17

Applications of LLMs in Qualitative Research

Answer 17

Automated Interviewing
* Concept: LLMs conduct initial screenings or gather preliminary data through natural language interactions.
* Advantages: Scalable, time-efficient, reduces interviewer bias.
* Challenges: Risk of hallucinations, lack of human intuition and context.
* Example: Using GPT-4 to conduct preliminary interviews for market research on a new product launch.

Question 18

Applications of LLMs in Qualitative Research

Answer 18

Sentiment Analysis
* Concept: LLMs provide qualitative insights into consumer sentiment beyond simple positive/negative categorization.
* Advantages: Rapid processing of vast data, identification of subtle nuances.
* Challenges: Potential misinterpretation of cultural or contextual subtleties.
* Example: Analyzing social media responses to a new advertising campaign using BERT-based models.

Question 19

Applications of LLMs in Qualitative Research

Answer 19

Open-ended Survey Responses
* Concept: LLMs process and categorize open-ended survey responses for actionable insights.
* Advantages: Quick theme identification and pattern recognition in consumer feedback.
* Challenges: Potential to miss outlier responses or unique insights.
* Example: Using Claude to analyze customer feedback on a new software feature.

Question 20

Applications of LLMs in Qualitative Research

Answer 20

Qualitative Coding Assistance
* Concept: LLMs support researchers in coding qualitative data by suggesting potential codes and themes.
* Advantages: Accelerates coding process, promotes consistency across multiple coders.
* Challenges: Risk of overlooking context-specific nuances, over-reliance on suggested codes.
* Example: Using Claude to assist in coding interview transcripts for a grounded theory study.

Question 21

Applications of LLMs in Qualitative Research

Answer 21

Synthetic Data Generation
* Concept: LLMs create realistic, diverse synthetic qualitative data for research and testing.
* Advantages: Provides privacy-compliant data, allows exploration of edge cases, enhances dataset diversity.
* Challenges: Ensuring data quality and relevance, potential biases in generated data.
* Example: Generating synthetic customer reviews to test a new sentiment analysis model.

Question 22

Applications of LLMs in Qualitative Research

Answer 22

Content Analysis
* Concept: LLMs assist in analyzing large volumes of textual data from various sources.
* Advantages: Efficient processing of diverse content types, identification of complex themes.
* Challenges: Potential oversimplification of nuanced content.
* Example: Analyzing years of annual reports to identify shifts in corporate strategy and culture.

Question 23

1. To gain an in-depth understanding of a phenomenon in its context.

Answer 23

Venomenon vs Context

Question 24

1. To elaborate or extend existing theory

Answer 24

Goal as qualitative researcher (in academia): elaborate or extend existing theory, by identifying
* new concepts (e.g., x3),
* relationships between concepts (variance theory),
* or new processes that explain how outcomes are generated (process theory).

Question 25

1. To get new (market) insights

Answer 25

* Established companies use qualitative research to foster innovation and new product development. * Often in cooperation with agencies. * As entrepreneur… * find economically answers to question such as: Who are my customers? What are their concerns? What elements of my product do they struggle with? * Example: Airbnb found out that the #1 reason not to use their platform was a lack of information about the homes. * Solution: offer providers (“hosts”) to book photographers free of charge.

Question 26

2. Types of qualitative data

Answer 26

* Archival data: written and visual material such as newspaper articles, blog posts, reports, promotional material. * Interview data (i.e., transcripts) * Observational data: are collected during field work in the research context (e.g., firm) and involves documenting observations through field notes (text) and/or other means (e.g., video). Researchers often combine different types of qualitative data in their analysis.

Question 27

3. Sampling

Answer 27

* Sampling is the first crucial step of the analysis * Initial choices limit later conclusions and how confident you feel about them * Sampling may look easy... * Many researchers study a single case (e.g., a firm). * Yet, settings have sub-settings (organizations have departments, departments have teams, teams have individuals…)

Question 28

3. Sampling: Key features in qualitative research

Answer 28

* Typical **small samples** of people nested in their context * Should be **purposive** rather than random * Usually theory-driven (upfront or progressively) * Initial definition of universe more limited * With small samples, random selection can deal you a decidedly biased hand * Typically **not entirely pre-defined** but evolves over fieldwork * Participants recommend other participants; looking at one case makes you consider looking at a similar but slightly different one with respect to some theoretically relevant dimensions * Be sure to set **boundaries**: usually done by looking at your research question and initial conceptual framework; consider time limit

Question 29

4. Depth Interviews

Answer 29

* Definition: A conversation aimed at eliciting detailed information about a research topic * Objective: Gain an in-depth understanding of a topic achieving nuanced insights into participants' experiences, beliefs, or perceptions * Characteristics: Core activity of qualitative research (stand alone or combined); formal and long (usually more than an hour) * Application: Can be utilized to understand various phenomena, such as consumer-brand relationships, management strategies, etc.

Question 30

4. Depth Interviews: Preparation

Answer 30

* Research Guide: Develop a guide containing topics, key questions, and probes ➢ Keeps you focused during the interview and helps coming up with new questions * Adaptability: Tailor the structure based on research stage, goals, and participants ➢ Structured approaches may be suitable for manager interviews; less structure could be apt for consumer conversations * Initial Questions: Employ warm-up questions to understand participant backgrounds and establish comfort (especially valuable with consumers) ➢ Example: “Can you tell me about yourself?” * Crafting Questions: Formulate questions that participants can answer ➢ Translate your research questions (often abstract concepts) into interview questions and use the appropriate language (that of participants in the context)

Question 31

4. Depth Interviews: Conducting the Interview

Answer 31

* Environment: Opt for a conducive and comfortable interview setting ➢ Avoid background noise; interview at work might be inappropriate * Introduction: Introduce yourself and the research purpose * Confidentiality: Stress the importance of privacy and confidentiality ➢ Might stress, that they are allowed to say things “off record” * Positioning: Sometimes, positioning yourself as less knowledgeable about the topic can elicit more detailed explanations * Don'ts: the interview is not about you – attention is on the interviewee, but you are directing the conversation ➢ Interviewee share personal stories, opinions, experiences – interviewers don’t. ➢ Avoid making interviewees feel self-conscious * Flow: Gradually move from general to specific questions ➢ Reduces bias * Avoid: yes/no questions ➢ No time to reflect on answer to prepare next question ➢ You can also do this in a survey * Exploration: Be receptive to tangential topics while maintaining focus on the central theme * Probing: Short verbal or non-verbal responses that request more of an answer ➢ Use to gain deeper insights without disrupting conversation flow

Question 32

4. Depth Interviews: Probes

Answer 32

* Repeater Probe: Repetition of a phrase or word to coax more details ➢ Repeat something the interviewee said and turn it into a question by raising the voice at the end * Non-verbal Probes: Use body language (e.g., nodding, raising eyebrow) to invite further elaboration * Expressing Interest: Utilize affirming sounds (e.g., “hmmm”) or words (e.g., “OK”/“I see”) without endorsing any response * Requesting Detail: Politely seek more detailed explanations or emotional responses ➢ Examples: “Can you tell me more about that?” or “How did that make you feel?” * Utilizing Silence: Allow silent moments to coax more sharing from the participant ➢ Is uncomfortable and eventually (it may be a few seconds, but can feel longer) the interviewee adds more detail

Question 33

4. Depth Interviews: Common Mistakes

Answer 33

* Interrupting: Don’t. But circle back to earlier topics for greater depth * Over/Under Empathizing: Maintain a balance – be empathetic without overshadowing the participant's experience ➢ Under Empathizing: e.g., a participant contradicts himself and the interviewer points this out in a judgmental way. ➢ Over Empathizing: novice interviewers often complete the sentences of participants (following the norm of active listening). Leads to useless data. * Biased Affirmation: “absolutely”, “I completely agree/understand how you feel…”. Closes down the dialogue

Question 34

4. Depth Interviews: Post-Interview Reflections

Answer 34

* Reflection: Ponder over the conversation and note key insights and areas for improvement ➢ Do the questions I ask really address my RQ? ➢ Was the interview “on topic” or unrelated to what I am interested in? > May indicate a mismatch between phenomenon (what are interested in) and context (what you are studying) ➢ Choice of words: am I using the language of participants? ➢ Specific questions: where did the participant struggle? Why? * Adaptation: Update your guidelines or protocol based on reflection outcomes * Pacing: Spread out interviews to allow adequate time for reflection and adaptation

Question 35

5. Projective Methods

Answer 35

* Origin in psychotherapy * Idea: People are sometimes better able to project their feelings onto others than to attribute those feelings to themselves * Sometimes respondents not even aware of their own attitude (strangers to ourselves) / respondents give socially desirable answers * Allows respondents to say things indirectly * Can be used in interviews to give participants a break and alter the pace of the interview

Question 36

5. Projective Methods: Examples

Answer 36

* Word association * Goal: reveal top-of-mind associations * Give participants a series of words and have them respond with the first things that come into their mind. * Sentence completion * Goal: reveal top-of-mind associations * “Christmas is a time when..”; “Work is a place where…”; “Owning a car means…” * Thematic stories * Participants are given a drawing or photography (e.g., of the CEO) to tell a story about it * Symbolic matching * Goal: quick visual metaphor for a person, brand, or company * Symbolic Matching, example: If Donald Trump were an animal, he would be…

Question 37

6. Focus Groups

Answer 37

* Special type of interview * Often used in practice * Managers often behind one-way glass mirrors * Work well for making smell, taste, and package design tests

Question 38

6. Focus Groups – Characteristics

Answer 38

Group Size: 6-12 Group Composition: Homogeneous, prescreened, strangers Physical setting: Relaxed, informal atmosphere Time duration: 1-3 hours Recording: Audio and video Moderator: Observational, interpersonal, and with good communication skills

Question 39

6. Focus Groups – Strengths and Weaknesses

Answer 39

Strengths * Relatively quick and inexpensive * Provides concentrated amounts of rich data, in participants’ own words * Interaction of participants adds richness to the data that may be missed in individual interviews Weaknesses * Lack the depth of interviews * Group dynamics can be a challenge if moderator is inexperienced * Interpretation is time-consuming, complicated by group dynamics, and requires experienced analysts * For a critical view on focus groups, see guardian podcast/long read: “Talk is cheap: the myth of the focus group: Focus groups make us feel our views matter – but no one with power cares what we think”

Question 40

7. Qualitative Data Analysis: Coding

Answer 40

Definition Coding: The systematic categorization of qualitative data to facilitate pattern identification and analysis Key Components * Data Chunks: Assigning labels to varied sizes of text, ranging from single words to entire paragraphs * Meaning Assignment: Labels or codes are not mere tags; they encapsulate the intrinsic meaning of the data chunk Characteristics of Codes * Hierarchy: Codes can be stratified in terms of specificity or abstraction (e.g., “diet” vs. “vegetarian diet”) * Flexibility: Codes can be adaptable to encompass emerging themes or concepts The Iterative Process of Coding * Initial Coding: Begin with a foundational set of codes, derived from theoretical framework or research questions * Revisiting and Refining: As data analysis progresses, revisit, refine, or expand codes to encapsulate emerging themes Ensuring Code Relevance * Interconnected Codes: Craft codes ensuring they interlink, forming a coherent thematic network that directly ties back to your research objectives * Alignment with Framework: Continually ensure that your codes and emerging themes resonate with your initial conceptual framework and research question(s) Timeliness in Coding * Early Start: Commence coding early in the data collection phase, potentially after the initial 1-2 interviews, to allow ongoing refinement of codes and adjustment of interview questions * Ongoing Process: Coding is not a standalone phase but a concurrent activity during data collection, enabling dynamic adaptation of the research approach Reliability * Consider incorporating strategies to enhance coding reliability, such as using multiple coders Coding with Software * Leverage qualitative data analysis software like NVivo, Atlas.ti, or MAXQDA to streamline the coding process, manage data, and enhance analytical capabilities Leveraging LLMs in Coding: A Novel Approach * Automated Assistance: LLMs, such as GPT-4.0, can assist in preliminary coding by identifying recurring themes or keywords within the data, serving as a base for further detailed manual coding * Data Management: Utilize LLMs for managing and retrieving relevant data chunks, enhancing efficiency in handling voluminous qualitative data * Pattern Recognition: LLMs might assist in initial pattern detection, providing a starting point for in-depth thematic analysis by researchers

Question 41

7. Qualitative Data Analysis: Coding Example

Answer 41

Interviewee: "Initially, I was a bit skeptical about using GPT-4.0. But when I started interacting with it, I realized it could understand my inquiries quite well and provide relevant responses. However, sometimes it misinterprets complex sentences or queries loaded with jargon. Yet, overall, it's quite impressive how it manages to create human-like interactions. I use it mainly for gathering quick information and occasionally for some light entertainment, like generating creative stories.“ Preliminary Coding Process Example: Initial Interactions and Perceptions * Skepticism * Surprise/Impressed * Relevance of responses Usage and Functionality * Information retrieval * Entertainment * Creativity in responses Limitations and Challenges * Misinterpretation of queries * Struggles with complex sentences/jargon Quality of Interaction * Human-like interactions * Ease of use

Question 42

8. How to Link Qualitative and Quantitative Data

Answer 42

* Quantitative data takes the form of numbers; Qualitative data the form of text. We need both to understand the world. * Doing both (qualitative and quantitative) by yourself can be demanding… * Quantitative data can help qualitative researchers during.. * Study design by finding a representative sample and locating deviant cases * Data collection by supplying background data * Analysis by showing the generality of specific observations (testing!) * Qualitative data can help the quantitative researchers during * Study design by helping with the conceptual development * Data collection by making access and data collection easier * Analysis by validating, interpreting, clarifying & illustrating quantitative findings as well as through revising theory

Question 43

9. Qualitative / Quantitative Research?

Answer 43

Apply qualitative research to… * Generate new insights * Explore emerging phenomena (e.g., generative AI). * Explore concepts or relationships between concepts which are poorly understood (and are therefore difficult to measure). * Explain phenomena which are difficult to model (e.g., “processes”) Don’t apply qualitative research when… * You are interested in (hypothesis) testing, effect sizes, and significance!

Question 44

Summary

Answer 44

* In the social science, qualitative research it is usually done to understand (especially emerging) phenomena in their contexts, to extend or elaborate existing theory * Qualitative and quantitative approaches help us to answer different questions * Sampling in qualitative research should be purposive * Data analysis should start early in the process of collecting the data