Description (analysis)

The use of description in evaluation is a data analysis method that provides rich descriptions and interpretations about the structure, design, and events or processes that were involved with the creation and implementation of an educational product or experience. Descriptive accounts help evaluators, other designers, and people who are interested in implementing the product to understand the design in the contexts for which it was made and review the decisions and objectives that were made by the instructional designers.

Definition

A descriptive evaluation is one that uses narrative, story, or descriptive terms to document an educational product's design and usage. Highly detailed descriptions are provided about the product, how and why it was built, and what happened when it was actually used.

A descriptive evaluation can also contain reflections from the design team, lessons learned during the process, and how things may be done differently if done again.

Additional Information

Descriptive evaluations give an accounting of what an educational product is, including descriptions of each of its features that make it unique. Additionally, descriptive evaluations reveal a narrative or sequence of events that led to the design, implementation, and actual use of the educational product.

Descriptive accounts are useful for designers as they consider improvements to the product or are looking for inspiration or a model for how to design a new project. Descriptive accounts are useful to learners, as they can learn about what they will do when they participate with the product.

For educators and decision makers, descriptive accounts are useful because they can reveal the contexts in which the product has been used and how people use it. It can also give decision makers information on how and for what purpose the product was made in the first place, leaving a story of the rationale for its design and information on the types of contexts and goals that the product sought to address.

Descriptive analyses do not need to be completed using only text, in the form of reports. Instead, video, audio, and other media can be used to describe the stories and concepts behind the development and usage of an educational product.

As with most analysis methods, the quality of the results from the method will depend upon (1) the quality of the data that are collected from the instruments that are used to document the design process and its features and (2) the skill and consistency of the evaluator in narrating the description and analyzing for patterns and causal mechanisms. In the case of descriptive evaluations, this largely depends on the quality of data that have been collected during the entire process from the earliest stages of design to the end of implementation. This includes capturing rich and detailed documentation of design from the design team on how, when, and why decisions were made and work was done. A detailed set of journals, logs, and design specifications should be kept, as it is impossible to remember everything that a designer or team did or thought about from each day.

Common things that a descriptive evaluation analyzes:

Design process. A descriptive evaluation uses narrative to document and describe the design process so that others may learn from the decisions, background and rationale, and work completed by the design team. It helps others understand the contexts in which the product was developed and for what purposes. It can also discuss the specific needs of learners and the communities which the product serves and how these served as rationale for specific design decisions.
Design characteristics. A good descriptive evaluation typically describes the structure and features of the product, including how it was designed and actually constructed, what technologies are used, and what the expected interactions from participants will be. The structural and audience considerations should be discussed in a descriptive evaluation, as well. It can also describe the theories that are used to predict how and why people will learn from the activities that were designed.
Sequence of events. Descriptive evaluations often tell the story of how an educational product was used in a real-world setting. It highlights the activities of learners and instructors, how and why certain features were or were not used, and the possible effects that the product or its elements had on the outcomes. A descriptive report will focus on telling the story of the events of a product and how things happened by describing what people did and how the product was implemented.
Reflections of the design team. The writing of a description gives the design team an opportunity to reflect on the overall design process and consider how they might have done things differently if they had to do the project again. These reflections can generate lessons or insights on the design process and be useful to other designers who are trying to improve their own design skills and work processes.

Common analysis approaches for performing descriptive evaluations:

Design report. A design report is a descriptive account of how and why an educational product was built. Often, design reports will include a narrative of how the design team performed its work and the challenges and successes that the team had in developing the educational product. It may also include a description of how the product was implemented and used in real-life settings. Stories about design give others information about the contexts that products were developed. It also provides inspiration and a model for how to conduct design projects in various settings and cases of how teams solved problems that emerged.
Case study. A case study examines a single case (or sometimes multiple comparative cases) in a structured way by using categories to frame the analysis. The goal of a case study is often to describe the case like in a design report, but it can evaluators can also analyze the case to identify patterns within the narrative and to reveal possible causal mechanisms that lead to certain outcomes. The writer of the case study will analyze and compare different aspects of the case and describe how and why things happened in the case. Common categories for descriptive case studies include rationale and background for design, design team composition, learning objective fit, design process tasks, development processes, and outcome achievement.
Process analysis. In this type of analysis, an evaluator will investigate the processes and tasks that people performed within a learning environment, the interactions that people performed with each other and with technology, and the steps that people took to complete tasks. Within a process analysis, an evaluator can investigate how and why people did certain things to get to their goal and complete tasks. This allows evaluators to generate hypotheses as to why a design worked or didn't work and can lead to a better understanding of the tasks within a learning experience.

Note: It is beyond the scope of this knowledgebase to expand on each of these methods. It is recommended that researchers and evaluators seek additional training, web resources, or courses on individual methods they would like to use.

Tips and Tricks

A rich description of your design and how it worked is a good first step toward evaluation. You may include other research questions, but at the least an educational product should include a highly detailed description of its design and an explanation for why each of the elements were included. This helps people make decisions about whether to use your product or not, or how the product is expected to achieve its intended goals.
Think about the description categories that you may want to include in a descriptive report of your product. This type of report commonly includes (1) descriptions about the design process and why and how you came to decisions, (2) a story or narrative about the specific steps that you took and when you took them, or (3) detailed descriptions about each of the design features of your product.
A detailed set of journals, logs, and design specifications should be kept during the entire design and implementation process, as it is impossible to remember everything that a designer or team did or thought about from each day. These documents can be used for analysis at a later time, particularly when describing the rationale for and structure of design.

Description (analysis)

From The Learning Engineer's Knowledgebase

Contents

Definition

Additional Information

Tips and Tricks

Related Concepts

Examples

External Resources