Day 1 with Stewart Donaldson: Intro to Eval
Summative Evaluation to determine if the program should keep running
Key questions to use?
Appreciative Inquiry was mentioned earlier. Also talked about how to politely evaluate a program – sometimes the owner will call you into evaluate their “baby” and you can’t start off by saying that everything sucks. Best way to evaluate is a clinical trial: implement something and test the before and after. Have a control group and an experimental group.
Comparative Analysis: before and after
How do you decide the outcomes that you’re looking at?
How to determine the primary outcome of choice?
Is it cost-effective? Should it continue?
Q: What kind-of evaluation design would you use?
A: Qualitative, Quantitative, Mixed Methods to bring data and qualitative together, don’t use an RCC for ethical reasons,
Day 2 Educational Evaluation
Two types of Eval: formative and summative
– Formative: how are things running? Something changes over time? How are they doing? Is X working but need more on Y? How to improve things at implementation. This is more like as the program forms.
– Summative: goal tends to be whether or not to continue the program, like a summary afterward. “Determine merit or worth.” Show evidence for if it’s good or bad. Side effects. Cost.
Needs assessment: assess which needs are met or not.
Met need = electricity
Unmet need = nutrition
Need to do an evaluability assessment. Is this program able to be evaluated? Are they ready?
Developing Eval questions, i.e.: What is the quality of our program?
Know the stakeholders, though she’d rather find out from the front line staff. They tend to know what’s going on.
Can do: case studies, observations, panel interviews, interviews, correlations, mixed methods (quant and qual) — and there needs to be a discussion with client about the approach. have a big tool kit. have a range of methods.
CQI: continuing quality improvement
Where do you start with logic models?
Step 1: what does research say about the problem? what is the problem?
Step 2: Identify activities to address the problem.
– how do you know it was implemented well?
Ferrington Report: lit review
growth mindset vs. fixed mindset
Day 3 Qualitative
Qualitative has multiple realities and is subjective (unlike quant). Often happens in a natural setting. Researcher is an instrument and is very much a part of the study in interpreting/making sense of what people are saying.
Qual and Quant can be both inductive and deductive, but quant tends to be deductive, meaning that you’re testing a theory and qual tends to be more inductive, meaning that you’re generating a theory. Be conscious of your own biases and how they might help or hurt. Respect your participants’ privacy.
– narrative approach i.e. life story approach.
– phenomenological approach: to explain an event
– grounded theory approach
– ethnographic approach: understanding the experience of a particular culture or sub-culture
– case study approach: how you define the case
What makes an economic system? -Scarce resources
Brain is lazy organ. 3% of body weight, 30% of calories
Breathing – can control up to some period of time