Evaluation model

An evaluation model is a list of the series of steps and things to consider that should be included in an evaluation. Models for evaluating educational products vary in their components and what the model is focused on, but most models at the least provide a blueprint or directions for things that an evaluator should attend to when developing their evaluation plan.

Definition

An evaluation model is a type of conceptual framework or theoretical model that lists all of the concepts, ideas, and factors that are necessary for conducting an evaluation. Such factors include a person's behavior, evidence of learning, competency, and psychological states, as well as factors related to teachers, other students, and environmental or contextual factors.

An evaluation model provides general descriptions of the concepts, factors, and processes that should be accounted for in an evaluation plan. Because of this, a theoretical evaluation model is used to generate a specific evaluation plan for an educational product.

Additional Information

An evaluation model will highlight a list of things that should be examined in an educational product. In other words, an evaluation model is a blueprint to follow to account for useful concepts, themes, and procedures in an evaluation so that the evaluator can make strong claims about how the product was successful. Ideally, the model helps the evaluator capture all of the necessary components to get clear, reliable, and valid observations of learning and participation within an educational product. Whichever model is chosen or developed, it is used by the evaluator to guide their work in writing the specific evaluation plan for the product and attend to important theoretical factors, variables, and research questions that are deemed most important to make strong claims.

Evaluation models therefore tie theory to practice for educational evaluation by providing a rationale for why certain factors of a person's learning, psychology, competence, or behavior should be examined and how these factors are related to learning outcomes and educational product use.

Models are built on best practices, theory, and logical progressions for making strong claims about the research questions. Best practices in evaluation have emerged over the years as certain research questions and methods for answering them have been demonstrated of reliably and validly offer evidence of people's learning and how educational products are used as expected.

Common components of an evaluation model

Evaluation models differ in what is focused on, but the most rigorous models have at least the following elements:

• Needs, interests, and priorities of learners and educators. It is important for most evaluations to at least account for and interpret findings based on the unique needs, interests, and priorities of the participant audience, as well as the intentions and priorities of the decision makers and educators who choose to use the product.
• Specific types of research questions that are asked. Individual models all focus on different research questions and aspects based on the theoretical framework held by the evaluator on what aspects are important to measure and analyze. Typically, research questions focus on (1) how people learn and change and (2) how people used the educational product as expected. Research questions are thus tied to the learning objectives and the designer's intent of how the product will be used.
• Theories of learning and change. It is important to account for the theories of change and learning upon which the product under evaluation is designed. Theories are used to predict the types of activity and learning that will occur and serves as the rationale for the product. With evaluation, the theory upon which the product is built can be tested as to whether it accurately predicts learning and identify why or why not it occurs. Additionally, an evaluation theory itself may be built on specific theories of evaluation that dictate how to measure and interpret learning, change, competency, and the use of an educational product. Theories in both categories of (1) learning and change and (2) evaluation should be accounted for in an evaluation model.
• Analysis methods and data instruments that are used. A model will at least describe in basic terms the types of analyses and data instruments that are used to generate insights about learning, change, and product use. These approaches are described more in the specific evaluation plan for a product, but are also an important part of the theoretical model for how evaluation occurs.

An example basic evaluation model

A basic evaluation model for educational product or experience evaluation. Multiple steps are involved with evaluating an educational product or experience, which are identified in an evaluation model. Evaluation models inform and document specific research questions and steps so that the evaluation activities are systematic, that the evaluation provides useful information for decision making, and that the insights from evaluation are trusted, valid, and reliable.

For beginners to evaluation, a basic, simplified evaluation model such as the one depicted in the image above is an excellent a starting point for an evaluation plan. Each step of the process in the basic evaluation model above accounts for the primary research questions that often provide evidence for whether a product was successful or not, or whether people learned from it. The steps of this basic evaluation model are based on best practices in the field.

Four common categories of research questions for evaluations are also referenced in this basic model, which follow four basic forms of questions that educational evaluations tend to answer to determine if the product was "good" or whether it achieved its intended goals or not. These questions include (1) questions related to learning, competency; (2) questions related to participants' perceptions; (3) questions of usage, participation, and quality of product; and (4) questions related to cost, efficiency, and comparison of products.

To use this model for a specific educational product, a beginner evaluator only needs to write an evaluation plan that identifies and specifies each of the steps outlined in the model - starting with the research questions of the evaluation.

All evaluation plans begin with well-written, clear research questions. The questions are in part influenced by the learning objectives of the product or experience being evaluated (i.e., what are people expected to learn from the product?) and the contexts in which the product or experience is implemented (i.e, who is the audience and industry/field using it, and what is the scope of the product?).

From the research questions, the instruments and analysis methods are identified so that the questions can be answered with data and systematic analysis. After the evaluation is conducted, the evaluator generates insights and findings, which can be used for making decisions about the product, identifying areas for how the product can be improved, and making claims with ample evidence about how well the educational product achieved its goals.

Tips and Tricks

• When just starting out in an evaluation, it is useful to select a simple plan and just follow it before tweaking it or amending it. The evaluation model provides a blueprint of the things and factors that an evaluator should attend to.
• It might be useful to select the sample evaluation model provided above for your first evaluation plan, as it touches on many important questions that examine the value of an educational product and looks basically at how people learned from the product.

Related Concepts

• Evaluation and assessment
• Evaluation plan
• Research question
• Instrument
• Analysis method
• Learning theory

Examples

Examples of evaluation models in education

• The Basic Educational Evaluation Model, which is detailed in the section above.
• The Knowledge-Attitude-Behavior (KAB) framework, a model that suggests individuals should be measured along the three dimensions of knowledge, attitude, and behavior.
• The Knowing What Students Know framework (Pellegrino, Chudowsky, Glaser), a model of evaluation to know how to identify what people know and what they have learned along three dimensions of student.
• The Authentic Assessment Model (Wiggins), a model that investigates learner performance by only evaluating their authentic tasks and learning activities, not by testing or performance on an unrelated external task. Also known as "activity as assessment."
• The Kirkpatrick Model of Evaluation, a model that evaluates educational products on four dimensions of learner reactions, learner achievement, behavior, and results.
• The Kaufman Model of Evaluation, a model that builds on the Kirkpatrick Model to include more analysis of the inner workings of the educational product and to include considerations of value of the product within society.
• Learner-Centered Assessment, in which the objectives to be evaluated are the tasks of the activity and participants' activity during a course of study.

External Resources

None yet - check back soon!