Asking "if" vs. "how"
From The Learning Engineer's Knowledgebase
When doing asking research questions, it is sometimes more productive to ask "how" something happens instead of "if" something happens. This is due to the quality of data that can be collected and the increased ability to make richer investigations to answer research questions.
Additional Information
Asking "if" something occurred within the context of a research question for evaluation is very limiting. Primarily, it provides only a discrete, binary set of outcomes - that is, there can be only one of two answers: yes it did, or no it did not. As such, researchers can lose out on opportunities to better understand not only the range or degree to how much something happened (by having an amount of how much something was used), but also explain how and why something happened by investigating the varied levels of product use and learning outcome achievement.
Designers often believe that answering how and why something happened is far more valuable toward improving educational products and demonstrating whether how well their products achieved their goals. Many designers also agree that their designs can always be improved, so they are more interested in knowing how much better could the design be, or how much more should something have been used.
Asking "if" versus "how" also has structural challenges associated with evaluation. The two types of questions influence the way data are analyzed and the type of response that a designer can give in response to the question. By asking the more expansive "how"-type research questions in evaluation, a designer can give more nuanced claims about the product and better theorize ways in which it can be improved.
Thus, asking "how" something happened gives you far more useful information for asking a broader variety of research questions instead of whether something achieved a goal or not. When asking "if," the boundary for yes and no is often arbitrarily set as well, leaving it further up to interpretation on whether a product is "good" or not. Instead, it may be more valuable to think of asking how good a product is rather than just asking if it is a good product.
In other words, it is far more interesting from a design perspective to understand how well the product hit the mark instead of simply whether it hit a mark or not!
Tips and Tricks
- When defining research questions for evaluations of educational products, try to focus on questions that ask "how" or "how much" something happened, rather than "if" something happened. This will give you a broader range of data to work with in your analyses and more tools for answering questions about whether a program was effective or people used it as you expected.
Related Concepts
Examples
None yet - check back soon
External Resources
None yet - check back soon