Lecture 3: Acquiring evidence

Question 1

2. Whom to Ask

Answer 1

Who is best place to answer these Q’s?
Who has addressed this Q previously? What did they find?

Choose practitioners with relevant expertise, meaning:

• Training, education, and hands-on experience in the issue’s domain
• Direct and frequent feedback in their work

The quality of the input depends on selecting the right experts for the question being explored.

Question 2

1. What to Ask

Answer 2

• How should we manage organisational change?
• How do our customers react to our advertising and promotions?
  The list is finite……

Practitioners should be asked:

• If they agree with the problem definition
• Whether alternative causes exist
• Whether the problem is serious
• Their views on proposed solutions, including:
  
  • Feasibility
  • Risks
  • Potential alternatives

Question 3

3. How Many to Ask

Answer 3

The number of practitioners depends on:

• Desired accuracy of results
• Resources (time, budget)
• Method:
  
  • Qualitative (e.g. interviews, focus groups) = smaller samples
  • Quantitative (e.g. surveys) = larger samples

Watch out for selection bias: if your sample doesn’t reflect the broader population, results may be skewed or misleading.

Question 4

How To Prevent Selection Bias

Answer 4

• True random sampling requires considering who and when to ask.
• The few or the many?
• Locally, regionally, nationally or internationally?
• Inclusive (wide, public, and open) or exclusive (private, limited and ‘invite-only’) groups?
• Who is accessible and who is not accessible?
• Do you have the capability and resources to manage all evidence / data?
• For example, surveying only early-start Monday workers skews data, especially for topics like flexible working hours.
• Selection bias can lead to unrepresentative or misleading conclusions.

Question 5

How to Ask: Methods of Data Collection

Answer 5

Walking Around and Asking
- - Quick and informal but prone to bias
- Good for initial exploration.

Surveys
- Efficient for large groups.
- Often quantitative, using closed-ended questions (e.g. Likert scales).
- Can include open-ended questions for deeper insight.
- Ensure clarity, anonymity, and purpose are communicated to increase honesty.

Delphi Method
- Structured qualitative approach with rounds of anonymous expert input and feedback.
- Aims to reach consensus.
- Useful in policy, strategy, and forecasting.

Group Decision Room
- Tech-supported, real-time focus groups or Delphi processes.
- Enables anonymous responses, instant visibility and discussion.

Mobile Voting/Audience Response Systems
- eal-time engagement using devices.
- Useful for interactive meetings and live polling.

Question 6

What is the Delphi- method?

Answer 6

The Delphi Method is a qualitative data collection technique where a panel of experts answers a series of questionnaires anonymously and in multiple rounds. After each round, a facilitator provides a summary of the responses. Experts then revise their answers in light of the group’s feedback, gradually moving toward consensus.

Question 7

What does ‘The fallacy of centrality Battered Child Syndrome (1948), refer to?

Answer 7

• A radiographer in the US identifies ‘inconsistencies’ in the reports of parents v x-ray results of injuries
• Conceivability of evidence issue among paediatrician [clinical] practitioners is a problem

The “fallacy of centrality” refers to the mistaken belief by those in power or central authority that if something significant were happening, they would or should know about it. This idea suggests a dangerous overconfidence in one’s own awareness or control over events.

The fallacy of centrality applies here because medical professionals, social workers, and authorities failed to recognize patterns of child abuse—believing that if widespread abuse were happening, they would already be aware of it.

Ergo: Practitioner validation and acceptance of evidence from other practitioners can take time

Question 8

What is The Whistleblower’s Dilemma?

Answer 8

The whistleblower’s dilemma is a moral conflict faced by individuals who consider reporting unethical or illegal activities within an organization.

refers to the pointing out malpractice in the workplace:
- Career cost
- Personal cost
- Emotional cost
- Identity cost - risk of stigma, betrayal of organizational loyalty, untrustworthy etc.
- Future employability….?

organizational silence

Question 9

Professional expertise importance

Answer 9

• The romance of leadership (Meindl, 1995)
• Personal and professional legitimacy (Suchman, 1995)
• It is hard to develop valid and reliable professional expertise. Why?
• Professional judgement and expertise and most prone to cognitive bias (Barends & Rousseau, 2018)

“One way of looking at this might be that for 42 years, I’ve been making small, regular deposits in this bank of experience, education and training. And on January 15, the balance was sufficient so that I could make a very large withdrawal.”

• Captain Chesney Sullenberger (US Air Pilot)

Question 10

Developing Questions and Tips for Asking Valid & Effective Questions

Answer 10

• Questions must accurately measure opinions, judgements, or experiences.
• Poor wording can lead to misleading results.

1. Keep it Simple
2. Avoid Double-Barrelled Questions
   3.Avoid Negatives/Double Negatives
3. Avoid Vague Terms
4. Define Terms Clearly
5. Avoid Loaded/Leading Language
6. Prevent Social Desirability Bias
7. Pilot Test the Questionnaire