Drriel: /* Definition */

2024-02-11T02:11:05Z

Definition

← Older revision		Revision as of 02:11, 11 February 2024
Line 8:		Line 8:
	Research questions are typically answered in a [[System / Systematic\|systematic]] and [[Documentation of evaluation plan\|documented]] way so that the insights can be trusted to be able to be replicated and are not just the purely subjective opinions of the evaluator, but instead are generated empirically using evidence from actual use of the product.		Research questions are typically answered in a [[System / Systematic\|systematic]] and [[Documentation of evaluation plan\|documented]] way so that the insights can be trusted to be able to be replicated and are not just the purely subjective opinions of the evaluator, but instead are generated empirically using evidence from actual use of the product.

	[[File:Types of Research Questions.png\|alt=A graphic representation of multiple types of participant-level and product-level research questions, depicted as a checklist\|thumb\|686x686px\|A checklist of ''potential'' '''research questions''' for educational evaluations. Not all of these research questions have to be asked in an evaluation. Research questions can be categorized as ''participant-level,'' or those that focus on the evaluation of individual participation and learning, or the ''product-level,'' which focuses on the evaluation of both individual design features and the whole product or experience.]]		[[File:Types of Research Questions.png\|alt=A graphic representation of multiple types of participant-level and product-level research questions, depicted as a checklist\|thumb\|686x686px\|A checklist of ''potential'' '''research questions''' for educational evaluations. Not all of these research questions have to be asked in an evaluation. Research questions can be categorized as ''participant-level,'' or those that focus on the evaluation of individual participation and learning, or the ''product-level,'' which focuses on the evaluation of both individual design features and the whole product or experience. ]]

	==Additional Information==		==Additional Information==

Drriel: 1 revision imported

2024-02-09T00:10:19Z

1 revision imported

← Older revision	Revision as of 00:10, 9 February 2024
(No difference)

Drriel: /* Types of research questions */

2023-07-02T17:07:52Z

Types of research questions

New page

A '''research question,''' when answered, provides [[evidence]] and interpretative results in an [[Evaluation and assessment|evaluation]] of an [[educational product]] or [[Learning experience|experience]]. All evaluations ask research questions that, when answered, give evaluators insights about the product or experience being evaluated on whether the product [[Goals of design|worked as intended]] or the degree to which participants [[Learning objectives and outcomes|learned]] from their use of the product or experience.

== Definition==
A '''research question''' is a guiding question that is asked in an evaluation to produce [[data]] and insights about how a product was used, and what effects or outcomes the product had on learning.

Data that are collected for evaluations must describe the things that the research question is trying to ask for the analysis to have strong [[validity]].

Research questions are typically answered in a [[System / Systematic|systematic]] and [[Documentation of evaluation plan|documented]] way so that the insights can be trusted to be able to be replicated and are not just the purely subjective opinions of the evaluator, but instead are generated empirically using evidence from actual use of the product.

[[File:Types of Research Questions.png|alt=A graphic representation of multiple types of participant-level and product-level research questions, depicted as a checklist|thumb|686x686px|A checklist of ''potential'' '''research questions''' for educational evaluations. Not all of these research questions have to be asked in an evaluation. Research questions can be categorized as ''participant-level,'' or those that focus on the evaluation of individual participation and learning, or the ''product-level,'' which focuses on the evaluation of both individual design features and the whole product or experience.]]

==Additional Information==
Answering research questions is the primary functional goal of educational evaluation and assessment. Without a research question to answer, an evaluator has no clear path or direction on how to collect and analyze evidence to see how and why an educational product worked.

Research questions are typically the starting point for any evaluation of an educational product. Once written, research questions are subsequently used to determine what kinds of data should be collected (via [[Instrument|instruments]]) and how the data should be analyzed (via [[Analysis method|analysis methods]]). As the research question determines what kinds of insights can be generated, the combination of instruments for data collection and analysis methods for an evaluation is customized to the research question being asked.

The answer to ''what'' research questions to ask in an evaluation is that it depends based on the goals of the evaluator. In short, this means that there is no standard evaluation plan or model that is conducted for every educational product. With this in mind, all evaluations should be custom built in a way that balances what the evaluators and design team wish to know about the participants' experiences and product use, as well as what the audience(s) of the product wish to know about the product's quality and value.

Additionally, evaluations may be limited in scope and the number or types of questions that can be asked based on the available budget for evaluation, limitations in the types of data that can be realistically collected for analysis from the participants, or the level of expertise of the evaluation team.

A '''''balanced''''' evaluation or assessment of an educational product should ask enough research questions to reveal valuable insights about participant learning, participant perceptions, and whether the product was used as expected - but does not overburden the evaluation or design team with evaluation tasks or data collection.

==== '''Types of research questions''' ====
There are multiple types of research questions that are commonly asked in educational [[Evaluation plan|evaluation and assessment plans]]. Each research question category elicits a specific type of [[Instrument|data requirements]] and subsequent [[Analysis method|analysis methods]] to [[Validity|validly]] and [[Reliability|reliably]] answer the research question in a trusted, believable way.

An [[evaluation plan]] does not need to contain each of these question types, but instead it is more important to have a balanced set of questions that provide enough information to reveal what the evaluator wants to find out, but does not overburden the evaluator, design team, or participants with evaluation activities at the expense of learning.

==== '''Participant-Level Research Questions''' ====
Participant-level research questions are those that investigate individual participants' experiences and interactions with educational products, interactions with other participants, and whether their participation had any influence on competence or learning. Participant-level research questions typically collect data on individuals and their [[Knowledge of the learner (KAB)|knowledge]], [[Behavior of the learner (KAB)|behaviors]], [[Interactions and activities|interactions]], and [[Attitudes and affect of the learner (KAB)|psychological states]]. Participant-level research questions generate insights for making claims about individuals and their experiences.

Frequently, participant-level research questions are directly related to the [[Learning objectives and outcomes|learning objectives]] of a product, with questions asking whether people achieved or demonstrated the established learning objectives.

''<u>Categories of participant-level research questions:</u>''

* [[Evaluating learning and competency outcomes]]
* [[Identifying factors that influence outcomes|Identifying factors and relationships that influence outcomes]]
* [[Identifying participant perceptions and satisfaction]]
* [[Identifying how people participate, interact, and learn]]

==== '''Product-Level Research Questions''' ====
Product-level research questions investigate the product itself and its features. Product-level research questions allow evaluators to generate insights and evidence for making claims about how features of a product was used, whether products work, or how well products compare to alternative approaches.

''<u>Categories of product-level research questions:</u>''

* [[Evaluating the quality and standards of educational products]]
* [[Comparing educational products and alternatives]]
* [[Identifying product and design feature usage]]
* [[Reporting on design processes and design specifications]]
* [[Analyzing cost and efficiency]]

==Tips and Tricks==

* When writing research questions for an evaluation, start by considering what kinds of things you want to know. Think about the kinds of claims that you want to make about a product, such as "the product helped people learn" or "the features were used as expected." You need good evidence to make these claims, and research questions that are appropriately aligned to the things that you want to know can provide you with a roadmap on how to find out information and insights about a product.
* Most research questions for evaluations typically come from one of the categories listed in the "types of research questions" section discussed above. If you are writing a new evaluation plan, these basic questions are good additions to any plan.
* Your research questions will determine what kinds of data that you need to collect and what kinds of methods you will use in your analysis. Don't try to force a method or data collection instrument that you wan't to use into a research question for which they are ill-equipped to answer! The instruments and analysis methods respond to the research question, not the other way around.

==Related Concepts ==

* [[Evaluation and assessment]]
* [[Instrument]]
* [[Analysis method]]
* [[Evaluation plan]]
* [[Evaluation model]]
* [[Learning objectives and outcomes]]

==Examples ==

None yet - check back soon!

==External Resources==

None yet - check back soon!

Research question - Revision history

Drriel: /* Definition */

Drriel: 1 revision imported

Drriel: /* Types of research questions */