exam 2

Question 1

Superintendent Brooks’ reform plan (4)

Answer 1

1. eliminating (or reclassifying) 101 central office positions
2. reorganizing the district’s central office bureaucracy to create a clear line of authority from the superintendent to school principals
3. the closure of 4 elementary schools in the predominantly low-income and African American southern part of LR
4. using a mixture of private and district funds, Brooks supported a pilot program to pay bonuses to Meadowcliff Elementary School teachers and staff

Question 2

purpose of the LRSD implementation research (2)

Answer 2

1. to explore the reorganization policy of the LRSD within the context of policy implementation
2. analyze data collected from the school district to better understand the barriers to successful implementation of this policy

Question 3

describe the LRSD reorganization policy (2)

Answer 3

create a more efficient bureaucracy by
1. establishing a clear line of authority from the superintendent to principals
2. streamlining the organization by eliminating central office positions and redistributing resources to schools

Question 4

Freeman’s 4 points of a needs assessment

Answer 4

1. constructing a precise definition of the problem
2. assessing its extent
3. defining and identifying the target of intervention
4. accurately describing the nature of the service needs of that population

Question 5

emison’s key points for control (6)

Answer 5

1. have a real work plan
2. meet the deadline
3. getting people with the right skills and temperament
4. expect something to go wrong
5. use more than one set of eyes
6. know your core values

Question 6

emison’s key points for communication (4)

Answer 6

1. lead with communicating ideas, not details
2. write and speak in short words and sentences
3. preview, provide, and review messages
4. visuals that support, not supplant, the briefing

Question 7

fink’s presentation points (visuals) (6)

Answer 7

1. do the talking and explaining, let the audience listen
2. focus on the key points of the presentation
3. each slide has a title
4. keep tables/figures simple
5. do not simply read a slide
6. rehearse the presentation

Question 8

solomon and shortell communication points (5)

Answer 8

1. understand the cognitive styles of decision makers
2. timely and available results
3. respect the stakeholders’ commitments
4. utilization/dissemination should be part of the design
5. overcome organizational resistance and politics

Question 9

emison’s characteristics of a work plan/timeline (4)

Answer 9

1. it must be complete; no important activities should be overlooked
2. it must be specific; products to be delivered and the timing of that delivery must be clear
3. it must connect actual humans on the evaluation team with the acitivities and products; names should be tied to obligations
4. it must be usuable; both understandable and updated as conditions change

Question 10

main difference between qualitative and quantitative research

Answer 10

qualitative research relies on words, a narrative, and/or a story; quantitative research relies on counts and statistical analysis

Question 11

advantages of qualitative research (5)

Answer 11

1. greater awareness of the perspective of program participants
2. awareness of time and history
3. sensitivity to context
4. ability to learn what is happening without prepared instruments
5. greater flexibility of perspective

Question 12

range/types of qualitative research (6)

Answer 12

1. ethnography
2. participant-observation
3. observation (non-participant observation)
4. informal interviewing
5. focus groups
6. documents

Question 13

ethical issues with qualitative research (3)

Answer 13

1. deception
2. confidentiality
3. bias

Question 14

combining qualitative and quantitative studies (5)

Answer 14

1. study different parts (in different forms)
2. study different stages of a program
3. to help evaluation design and measures
4. multiple methods
5. interpretation of results

Question 15

3 areas one wants a process evaluation to provide data/information

Answer 15

1. description of program operations
2. comparison between sites
3. conformity of the program to its design

Question 16

program organization

Answer 16

comparing the plan with what the program should be doing and what is actually being provided

Question 17

service utilization

Answer 17

understanding or examining the extent to which the target population receives the intended service

Question 18

coverage

Answer 18

the extent to which participation by the target population achieves the levels specified in the program design

Question 19

bias

Answer 19

the degree to which some subgroups participate in greater proportions than others

Question 20

3 perspectives of process monitoring

Answer 20

1. evaluator - implementation issues
2. sponsors - program accountability
3. program managers - corrective actions

Question 21

how is the problem described when assessing the extent (needs assessment)

Answer 21

in terms of the size of the problem; how widespread it is

Question 22

ways to describe the target population (3)

Answer 22

1. risk/need
2. incidence/prevalence
3. rates

Question 23

risk/need

Answer 23

a population at risk would consist of persons or units that have a significant probability of developing this problem or condition that the program is trying to address

Question 24

incidence

Answer 24

the number of new cases that are identified in a specific time period

Question 25

prevalence

Answer 25

the total number of existing cases

Question 26

rates

Answer 26

a standardization; as you describe incidence or prevalence, you might use it as a rate

Question 27

importance of understanding the nature of service needs of the target population

Answer 27

if you don't understand the services needed, the target population cannot get the proper help

Question 28

Management Information System (MIS)

Answer 28

what data the organization is already collecting

Question 29

ways to develop relevant outcomes (3)

Answer 29

1. stakeholders 2. program theory/impact model 3. prior research

Question 30

how process evaluation findings are used per Fox, Grimm, and Calderia

Answer 30

to shape the intervention/recommendations

Question 31

key point of distinguishing monitoring from normal process/outcome evaluations

Answer 31

repeated measurement over time

Question 32

approaches to setting levels of program performance (3)

Answer 32

1. often based off of impact model/program theory 2. identify which aspects of program performance are more important 3. look at administrative standards to judge what good performance is

Question 33

2 characteristics of outcome evaluation per Rossi

Answer 33

1. outcomes = observed characteristics of the target population or social conditions 2. outcomes don't mean the program targets have changed/programs have caused them to change

Question 34

implementation type for each group and the strategy for successful implementation (LRSD) (4)

Answer 34

1. superintendent & executive staff = administrative implementation; key to success = providing sufficient resources (training) 2. principals = political implementation; key to success = power of district leadership to force compliance 3. teachers = experimental implementation; key to success = providing sufficient resources (understanding that additional resources were a result of the reorganization) 4. central office & school-based staff = administrative implementation; key to success = providing sufficient resources (training)

Question 35

why academic research time and evaluation time differ

Answer 35

academic time = more perfect evaluation time = time constraints/sacrifice data

Question 36

parts of an evaluation presentation (5)

Answer 36

1. address the evaluation purposes 2. describe the program/participants 3. describe the main methods (data collection) 4. provide main results 5. provide a conclusion (include limitations) and recommendations

Question 37

2 issues evaluators might face in communicating results

Answer 37

1. agency/organizational structure 2. agency/organizational politics

Question 38

pros noted in the LRSD case study article (4)

Answer 38

1. reorganization --> greater accountability (Larry Berkely) 2. efficiency & effectiveness 3. reduce top-heavy administration (bloated central office) 4. cost saving

Question 39

cons noted in LRSD case study article (5)

Answer 39

1. moved too quickly 2. disproportionate effect on black personnel ($913k v. $28k loss for white personnel) 3. transition team determined who to fire in secret with no justification 4. Brooks failed to connect with black community & grass roots organizations 5. poor relationship with teachers union (brought in to "destroy" it)

Question 40

stakeholders involved in relation to their motivations of an evaluation (2)

Answer 40

1. pro-Brooks school board members (Berkley, Fox, Kurrus): justify support and continued support of Brooks reforms 2. anti-Brooks school board members (Mitchell, Armstrong, Curry, Daugherty): justify removal and reversal of Brooks' decisions

Question 41

importance of defining target population

Answer 41

helps set boundaries of the program; different stakeholders may have different ideas of who the target population is

Question 42

concepts of specification of services and accessibility (delivery of services) (2)

Answer 42

1. specification of the services: the actual service provided by the program in operational terms (measurable terms) (an explicit definition of a service and how you go about measure that) 2. accessibility: are the program's actions consistent with providing access?

Question 43

on-going problem with evaluation studies in terms of timeline

Answer 43

data collection; collection of data can take longer than one estimate, which then pushes back how much analysis there is