Closes vs. open book evaluation
From The Learning Engineer's Knowledgebase
Closed versus open book in testing and assessment is an ongoing debate about how learners should be assessed for knowledge and skills and whether they should be allowed to access any outside media or information while they are performing the assessment tasks.
Definition
A closed-book assessment is one in which learners are allowed no outside media and information while completing an assessment activity, such as a test, quiz, or essay. The assessment activity thus requires complete memorization of the information to complete the activity in a closed-book assessment.
An open-book assessment is one where the learner can use outside media and information to complete the assessment activity.
The debate on closed vs. open book assessment exists within questions related to what students are demonstrating during the assessment activity and what knowledge and skills are deemed important to capture in the assessment instrument (such as a test or quiz). The conditions under which the assessment is completed (e.g., open or closed book) in part determines the kinds of evidence and claims about learning and competence that can be made from the assessment.
Additional Information
There is a substantial debate on the nature of knowledge and how we record information in media, such as in books, texts, websites, videos, and even wikis (like this page!).
On one side of the debate (with the closed bookers), people feel that learners should always memorize key facts and definitions about educational content topics, such as concepts, ideas, events, people, and places. If you learned something, you should be able to repeat back at any time key descriptions and definitions of that thing. It is argued that without knowing concepts and ideas, a person cannot use that knowledge in a situation where they could use it. When failing to recite or recall factual knowledge like this without the help of any outside information, this side argues that a person does not know the material and thus did not learn.
On the other side (the open bookers), people argue that it is more important to know how to apply knowledge and facts and not just memorize it. Applying knowledge includes skills in how to find, evaluate, judge the quality of, and then use knowledge in actual, real-world situations. In the case of open book proponents, information and knowledge is so readily available today that it can be accessed at a moment's notice via a mobile device or computer, or even a traditional paper-bound book.
From the open-book side's perspective, humans have the highest level of access to recorded knowledge in history, so it has become a natural part of our cognitive processes to immediately ask about and seek information when it is needed to learn. Then, that information and knowledge that is rapidly found can be applied toward authentic problem solving activities and real-world tasks. This is most common in how people do quick google searches, read Wikipedia articles, or watch YouTube tutorials to help them solve an immediate challenge that they are facing (or to solve an argument at a party!).
Some even argue that it would be inauthentic and a disservice to "close the book" on using information and media during assessment because it has become an important part of everyday problem solving. In fact, they argue that the assessment increases its validity if it captures students' everyday behaviors in an accurate way, specifically by allowing an open book. Yes, a student can memorize information in a closed-book assessment setting, but the real world does not operate in that way.
Additionally, it is argued that learners actually better acquire the very knowledge that they sought out because they used it in an authentic setting (and not on a test). Open-bookers frequently say that people forget material as soon as they take a test, as it was never used in a real way (in kids' lingo, suffering from a brain dump, or post-test forgetfulness). The argument states that because the knowledge was used for a purpose other than a quick test, it is more memorable and thus sticks with the learner longer.
Open-book proponents also argue that tests are not as accurate at capturing learners' ability for problem solving and information use skills. Instead, they argue for assessments that ask students to demonstrate their skills in finding and using knowledge from the content domain, such as through completing open-ended projects, problem solving, and working through scenarios - all of which have no single correct answer or "right" way of completing the assessment task. Instead, these tasks are assessed on the quality of the demonstrated skills in applying the knowledge.
Neither side is a clear winner in this debate. Both sides have their merits, but also their shortcomings. It is clear that knowing information concretely without the use of devices or external information is beneficial for being able to quickly retrieve information when needed from one's own brain. Also, our devices can fail or we may not have internet access, which limits the learner. From this closed-book perspective, a learner would be better off. However, the real world in which internet-connected devices are ubiquitous does not operate this way, and recorded information is plentiful, indexed, searchable, and retrievable within seconds. From the open-book perspective, it could be likewise argued that it is more important to teach the skills for finding, evaluating, and using information in real-world tasks instead of just teaching people to memorize and repeat information.
Tips and Tricks
- The side you take in this debate realistically lies in your commitments to the learner and what you feel is important. It also depends on the learning objectives that you are trying to achieve and the context or domain in which the product is being implemented. While open book works for a K-12 setting teaching creativity may not work for a business context teaching fast-reaction responses to customer requests on a help line. Sometimes, it is most important to completely memorize information, and sometimes it is more important to know how to find and evaluate the information for when you need it.
- Consider the contexts and goals of your educational product if you are debating this issue with yourself. Remember, there is no right answer - both sides are right, but toward different purposes and goals.
Related Concepts
Examples
None yet - check back later!
External Resources
None yet - check back later!