Categories
evaluation

Why Evaluation?

In a recent blog post published during the AEA 2019 Conference, Cameron Norman, initiated a list of things that evaluation is for. Recalling this list was helpful during a recent family gathering when I was asked about work. Rather than focus on the means of evaluation, I found myself talking more about the ends of evaluation which coincided with many of the things on his list shared below. Evaluation is for…

  • Decision making
  • Seeing the future
  • Design and innovation
  • Asking better questions
  • Creating conversation
  • Speaking truth to power
  • Honoring our work
  • Learning
  • Leading system change
  • Telling stories about who we are as a people
  • Promoting health and preventing harm
  • Reinforcing democratic ideals
  • Provoking curiosity
  • Recognizing humanity

Make sure to check out Cameron’s blog post that elaborates on each of these.

Categories
assessment mathematics

Designing Mathematics Assessments

Using released items from state and national mathematics assessments can be helpful in designing a grade level assessment instrument for students to be used as one source of data in the evaluation of teacher professional development or student mathematics interventions.

The list below identifies sources of released assessment items that might be aligned with the content of your program. This list will be updated as additional sources are identified.

New York State Department of Education Released 2017 Grades 3-8 ELA and Mathematics State Test Questions

New York State Department of Education Released 2016 Grades 3-8 ELA and Mathematics State Test Questions

PARCC Released Items

NAEP Released Items / NAEP Questions Tool Take a look at this Facebook video for help in using the tool.

Once a pool of items are established, you’ll need to consider next steps to ensure the validity and reliability of the assessment.

Categories
evaluation

Eva the Evaluator

Based on the book Eva the Evaluator by Roger Miranda, Wendy Tackett’s enactment helps Eva understand all the different roles that an evaluator might take on in an evaluation project. This light-hearted video and the book in which it is based on demystify the complex work of evaluation.

Categories
design programs

Defining a Program

What makes a program a program?

A recent webinar led by Boris Volkov hosted by AEA’s Organizational Learning & Evaluation Capacity Building Topic Interest Group gave me pause to think about important qualities of a program.  His talk titled From Purist to Pragmatist: Expanding Our Approaches to Building Evaluation Capacity opened with a shared definition of program, evaluation, and program evaluation. He used the CDC (2011) definition that states:

A program is any set of organized activities supported by a set of resources to achieve a specific and intended result (p. 3)

Inherent in this rather broad definition are three important moving parts:

  1. Set of resources (inputs)
  2. Set of organized activities (activities described by outputs)
  3. Specific and intended result (outcomes and impacts)

Each of these provide parameters that define and focus a program and its evaluation. Yet, embedded in the definition are the qualifiers- organized, specific, and intended. Activities must have some sort of underlying organization. Perhaps the activities are organized developmentally, sequenced by difficulty, or arranged in some manner as a result of previous research. Results must be specific and intended. That is, there is a desired end that can be clearly articulated so as to justify the means.

A program exists when the qualifiers (organized, specific, and intended) are understood and there is a logical relationship between resources, activities, and outcomes. This logical arrangement is often described in a program’s logic model. Without understood qualifiers and an articulated logic, a program is not a program, but rather a loose set of activities.