Evaluation and assessment - Revision history

Drriel: /* Additional Information */

2024-02-11T02:10:44Z

Additional Information

← Older revision		Revision as of 02:10, 11 February 2024
Line 17:		Line 17:
	Two primary types of [[Research question\|research questions]] are answered during the process of evaluation and assessment: (1) answering research questions about [[Learning objectives and outcomes\|learning objective achievement]] by [[Participant, student, or learner\|learners]] (who are the primary focus of an [[educational product]]) and (2) whether the product was [[Design feature use evaluation\|used as intended]], by both the participants and the people who [[Implementation\|implement]] the product. Evaluations also sometimes ask questions about the quality and value of an educational product, such as whether it is a "[["Good" educational design\|good product]]" and if it was useful or better compared to alternative products.		Two primary types of [[Research question\|research questions]] are answered during the process of evaluation and assessment: (1) answering research questions about [[Learning objectives and outcomes\|learning objective achievement]] by [[Participant, student, or learner\|learners]] (who are the primary focus of an [[educational product]]) and (2) whether the product was [[Design feature use evaluation\|used as intended]], by both the participants and the people who [[Implementation\|implement]] the product. Evaluations also sometimes ask questions about the quality and value of an educational product, such as whether it is a "[["Good" educational design\|good product]]" and if it was useful or better compared to alternative products.

	Evaluators take a ~~[[System / Systematic\|~~systemic approach]] to investigating what, how, and why things happened as a result of using an educational product. They use an [[evaluation model]] to guide them as the do the step-by-step process of developing their [[evaluation plan]] and later following through on the plan. Evaluation models are step-by-step processes that are commonly used and are based on best practices in the field.		Evaluators take a systemic approach to investigating what, how, and why things happened as a result of using an educational product. They use an [[evaluation model]] to guide them as the do the step-by-step process of developing their [[evaluation plan]] and later following through on the plan. Evaluation models are step-by-step processes that are commonly used and are based on best practices in the field.

	The product of the evaluation process is a collection of insights and findings about how and why people used the educational product and how and why they demonstrated competency or learning from the experience. Evidence about the use of the product and its effects is used to describe the value of the product, as well as to ~~[[Iterative Design Process\|~~improve the product in its future versions]]. Additionally, evaluation and assessment is often used to providing a grade, certification, or other credential to a learner to signify their competence after participating with the product.		The product of the evaluation process is a collection of insights and findings about how and why people used the educational product and how and why they demonstrated competency or learning from the experience. Evidence about the use of the product and its effects is used to describe the value of the product, as well as to improve the product in its future versions. Additionally, evaluation and assessment is often used to providing a grade, certification, or other credential to a learner to signify their competence after participating with the product.

	===== Common process and steps of developing an evaluation plan =====		===== Common process and steps of developing an evaluation plan =====

Drriel: /* Additional Information */

2024-02-11T02:10:15Z

Additional Information

← Older revision		Revision as of 02:10, 11 February 2024
Line 9:		Line 9:

	==Additional Information==		==Additional Information==
	[[File:ADDIE evaluation.png\|alt=A graphic representation of the ADDIE model of design - the evaluation section is highlighted\|center\|thumb\|450x450px\|Evaluation is the last step of the [[ADDIE Model\|ADDIE model]] of design]]		[[File:ADDIE evaluation.png\|alt=A graphic representation of the ADDIE model of design - the evaluation section is highlighted\|center\|thumb\|450x450px\|Evaluation is the last step of the [[ADDIE Model\|ADDIE model]] of design ]]
	Evaluation is the final step of the [[ADDIE Model\|ADDIE model]] and seeks to answer questions about whether the product worked as intended and if people who participated achieved the learning objectives of the product. Using an [[evaluation model]], evaluators will develop an [[evaluation plan]] and conduct the evaluation plan when the product is implemented. After an evaluation is completed, the evaluator will write reports and share the information with anyone who is interested.		Evaluation is the final step of the [[ADDIE Model\|ADDIE model]] and seeks to answer questions about whether the product worked as intended and if people who participated achieved the learning objectives of the product. Using an [[evaluation model]], evaluators will develop an [[evaluation plan]] and conduct the evaluation plan when the product is implemented. After an evaluation is completed, the evaluator will write reports and share the information with anyone who is interested.

Drriel: 1 revision imported

2024-02-09T00:10:18Z

1 revision imported

← Older revision	Revision as of 00:10, 9 February 2024
(No difference)

Drriel: Created page with "'''Evaluation and assessment''' is the final step of the ADDIE model of design. In this phase, educational products are evaluated for whether they met the intended goals and whether learners benefited from the experience. ==Definition== '''Evaluation''' is the process of analyzing and making judgement on whether an educational product achieved its goals. An evaluation, as a noun, is the ac..."

2023-06-16T01:57:25Z

Created page with "'''Evaluation and assessment''' is the final step of the ADDIE model of design. In this phase, educational products are evaluated for whether they met the intended goals and whether learners benefited from the experience. ==Definition== '''Evaluation''' is the process of analyzing and making judgement on whether an educational product achieved its goals. An evaluation, as a noun, is the ac..."

New page

'''Evaluation and assessment''' is the final step of the [[ADDIE Model|ADDIE model of design]]. In this phase, [[Educational product|educational products]] are evaluated for whether they met the intended goals and whether [[Learning objectives and outcomes|learners benefited]] from the experience.

==Definition==
'''Evaluation''' is the process of analyzing and making judgement on whether an [[educational product]] achieved its goals. An evaluation, as a noun, is the activity of evaluating, or the final results and report from an evaluation.

'''Assessment''' is typically the process of evaluating people for competence, learning, or change. Assessment, as a noun, can also mean the [[Instrument|instruments]] used to assess a person.

Although there are slight differences between the terms of ''evaluation'' and ''assessment'', they are often used synonymously. The word ''evaluation'' tends to be used more to describe, compare, and judge the quality of educational products, where the word ''assessment'' is more frequently used when concerning an individual and measuring [[Competency vs. learning|their learning]] or other important changes (such as [[participation]] or [[Attitudes and affect of the learner (KAB)|affect]]).

==Additional Information==
[[File:ADDIE evaluation.png|alt=A graphic representation of the ADDIE model of design - the evaluation section is highlighted|center|thumb|450x450px|Evaluation is the last step of the [[ADDIE Model|ADDIE model]] of design]]
Evaluation is the final step of the [[ADDIE Model|ADDIE model]] and seeks to answer questions about whether the product worked as intended and if people who participated achieved the learning objectives of the product. Using an [[evaluation model]], evaluators will develop an [[evaluation plan]] and conduct the evaluation plan when the product is implemented. After an evaluation is completed, the evaluator will write reports and share the information with anyone who is interested.

Scientifically, the best evaluations are pre-planned and are subsequently completed according to the plan. Any deviations from the plan are documented and shared as a part of the results. This is to increase the [[reliability]] and [[validity]] of the findings, to improve the ability for others to reproduce and follow the steps that the evaluator took to come up with their findings, and ultimately improve the trust people have when they read the results and [[generalizability]] of the findings to other contexts.

===== Purposes of evaluation and assessment =====
Two primary types of [[Research question|research questions]] are answered during the process of evaluation and assessment: (1) answering research questions about [[Learning objectives and outcomes|learning objective achievement]] by [[Participant, student, or learner|learners]] (who are the primary focus of an [[educational product]]) and (2) whether the product was [[Design feature use evaluation|used as intended]], by both the participants and the people who [[Implementation|implement]] the product. Evaluations also sometimes ask questions about the quality and value of an educational product, such as whether it is a "[["Good" educational design|good product]]" and if it was useful or better compared to alternative products.

Evaluators take a [[System / Systematic|systemic approach]] to investigating what, how, and why things happened as a result of using an educational product. They use an [[evaluation model]] to guide them as the do the step-by-step process of developing their [[evaluation plan]] and later following through on the plan. Evaluation models are step-by-step processes that are commonly used and are based on best practices in the field.

The product of the evaluation process is a collection of insights and findings about how and why people used the educational product and how and why they demonstrated competency or learning from the experience. Evidence about the use of the product and its effects is used to describe the value of the product, as well as to [[Iterative Design Process|improve the product in its future versions]]. Additionally, evaluation and assessment is often used to providing a grade, certification, or other credential to a learner to signify their competence after participating with the product.

===== Common process and steps of developing an evaluation plan =====
Evaluation typically occurs over a series of general steps. Of course, each [[evaluation plan]] is different based on the [[Research question|research questions]] being asked and the context in which the product was implemented.

# '''Define the research questions.''' The beginning of any evaluation involves defining the questions that will be asked and why they matter. Evaluators must define specific questions whose answers will provide useful information toward improving the product and demonstrating that the product works.
# '''Identify the audience.''' Each audience is different and may have varied needs. To interpret the evaluation [[Validity|validly]], [[Reliability|reliably]], and [[Equity|fairly]], the evaluators should consider the unique composition of any audiences that will use the product, as well as the contexts in which the product will be implemented.
# '''Determine analysis methods and instruments.''' To answer the research questions, the evaluators must define both the [[Analysis method|analysis methods]] and the [[Instrument|instruments]] that will to be used to collect and analyze [[data]].
# '''Conduct the evaluation'''. After the planning is complete, the evaluation is conducted while the educational product is being implemented. Learners are assessed using both [[Summative assessment|summative]] and [[Formative assessment|formative]] approaches and data are collected on how the product and its [[Design feature|individual design features]] are used.
# '''Conduct the analysis'''. Using the [[data]] that were collected during implementation, [[Analysis method|analyses]] are completed based on the [[evaluation plan]] and results are generated after implementation. However, with [[formative assessment]], the analyses and interpretation are completed ''while'' the product is being implemented so that the experience can be improved for learners as they participate.
# '''Interpret and review the results.''' From the analyses and the results they generate, the evaluators will generate insights about what happened, how people learned, and how people used the product as intended. The evaluators will interpret the findings using standard industry practices and make claims about what the evidence means.
# '''Report and share findings'''. The results and findings from evaluations are typically shared with parties interested in knowing how a product worked, including funders, potential implementers, educators, and purchasers. Results from evaluations are most often shared in the form of reports, descriptive studies, and research papers that describe the product, the research questions, the data, the analysis approaches, and the results.

===== When to evaluate =====
Conducting an evaluation occurs when an [[educational product]] is being implemented. From the first moment of its use, data are being collected on the participants' learning and how they are changing as a result of their participation. Additionally, data are often collected on how people use the product throughout implementation. However, evaluations and analyses are conducted throughout the [[implementation]] of an educational product.

[[Formative assessment]] is conducted actively while the product is being implemented. This real-time evaluation approach is used to identify how and why participants are learning with the system and what factors are influencing their learning, including their [[Participation|level of participation]] with the product's various [[Design feature|design features]]. With formative assessment, the [[Educational product|product]], [[learning environment]], and [[Learning experience|overall experience]] can be improved ''while it is being used,'' based on the learner's performance and perceptions.

[[Summative assessment]] is conducted at the end of the product's use. Data that are collected at the end of the product's use provides an "end result" of a participant's competence and learning. Analyses that are conducted at the end of the product's use can investigate, as a whole, how the product was used and what effects it had on learning. At the end of a product's use, evaluators make claims about how well the product worked, based on the analyses and evidence shown in the results. Evaluations at the conclusion are typically written as a report and shared with interested parties.

===== Types of evaluations and common research questions =====
There are common families of [[Research question|research questions]] that appear frequently in evaluation studies. Each responds to a particular research need for evaluators to answer about the quality of the product and how or why the participants learned from it.

* [[Analyzing cost and efficiency]]
* [[Comparing educational products and alternatives]]
* [[Evaluating learning and competency outcomes]]
* [[Evaluating the quality and standards of educational products]]
* [[Identifying factors that influence outcomes]]
* [[Identifying participant perceptions and satisfaction]]
* [[Identifying product and design feature usage]]
* [[Identifying how people participate, interact, and learn]]
* [[Reporting on design processes and design specifications]]<br />

==Tips and Tricks==

* Every [[educational product]] should be evaluated for whether it was [[Identifying product and design feature usage|used as intended]] and whether people [[Evaluating learning and competency outcomes|learned from it]]. This is completed by designing an [[evaluation plan]] that asks specific [[Research question|research questions]] about how and why the product worked. Consider following the step-by-step guidance on [[Evaluation plan|evaluation plans]] on how to begin designing an evaluation for a product.
* Evaluation can occur both while a product is being used (via [[formative assessment]]) and at the end of the product's use (via [[summative assessment]]). Think about how you may use formative methods to evaluate how things are going for participants while they are active in the learning experience, as well as how teachers or facilitators may be informed via formative methods on how to improve the experience for participants.
* With summative approaches, consider what kinds of claims you would like to make about your product if you were to share it with other people. Such claims include:
** This is a good product
** People used the product as expected
** This product worked as intended
** People learned from this product
** This product is better than alternatives
* While using an [[evaluation model]] to design the evaluation plan, determine what kinds of analyses you need to conduct and what kinds of data you need to collect using [[Instrument|instruments]] to make the kinds of claims that you determined above. The evaluation model can help you think about and identify the analyses and data that you need to make strong claims.
* Evaluation is a challenging task. Many evaluations are conducted by a team. Freely share your plans and ideas with others to get feedback on how to improve your plan so that you can make the strongest claims. Consult with research methodologists to consider what kinds of analyses and data are appropriate for the kinds of research questions that you are asking.

==Related Concepts ==

* [[Evaluation plan]]
* [[Evaluation model]]
* [[Research question]]
* [[Formative assessment]]
* [[Summative assessment]]
* [[Analysis method]]
* [[Instrument]]
* [[Data]]
* [[Evidence]]
* [[Iterative Design Process]]
* [["Good" educational design]]
* [[ADDIE Model]]

==Examples==

None yet - check back soon!

==External Resources==

None yet - check back soon!