Data
From The Learning Engineer's Knowledgebase
Data is recorded evidence that represents a specific phenomena or aspect about a person. In educational evaluation, data is information that is recorded and captured from evaluation instruments and are used in analysis methods to answer research questions.
Definition
Data is a recording of some information of interest. Data is always recorded in files, logs, datasets, or some kind of document. Data are recorded by using instruments within a learning experience. Data are frequently numeric measurements of a variable, but data can also be any other form of recorded information: text, dialogue, media, documents, digital logfiles, journals, observations, and people's work products.
Additional Information
Data are a representation of something happening or existing in the world. A representation is simply a copy or a recording of that thing that is being looked at. As a result, data records are evidence of things happening or existing at specific times, as all data can have a timestamp attached that indicates when the data was recorded.
Data are typically sorted into variables to categorize the data and allow researchers to analyze it. A variable is any category that holds data. A researcher defines a variable - there is no specific way that a variable should be named or hold data. For people new to evaluation, it can be helpful to think of a variable as a container for storing whatever data you are collecting. Variables are usually very specific about the thing it describes, such as how many times a person sent an email, or the score of a survey question that asked how satisfied the learner was at the end of the learning experience. To this end, a variable name describes what values are contained within it.
Common types of variables in educational evaluation include:
- Variables with numeric values, or measures. Numeric values are assigned based on how the variable captures information.
- A numeric value can have a score (also called a "scale" variable). This is an amount of something defined by the variable. For instance, a variable can represent the total score on a test, which could range from 0 to 100. For each person, they would be assigned a number value for this variable. The variable, which measures performance as a score on the test, could then be used in an analysis.
- A numeric value can have a rank (also called an "ordinal" variable). This variable captures information about something as a rank-ordered value. For instance, a person could be asked about how much they were satisfied with the videos in a course, in which they would give a rank-ordered response of 1 (did not like at all) to 5 (liked very much). For each person, the variable would be recorded as a rank-ordered value based on their response, which then could be used in an analysis.
- A numeric value can be a categorization (also called an "index" variable). The numeric values within a variable do not need to have any meaningful progression between them - or that a "2" does not mean more than a "1". Instead it can be a way for the data to be quickly organized into categories. This is commonly done to record categories of traits that someone has or information about their background. For instance, a "location" variable can record someone who says they live in a rural area as "1", someone who lives in a suburban area as "2", and someone who lives in an urban area as "3". The differences in this case between 1, 2, and 3 do not matter - 3 is not higher than 1. They simply are numeric assignments of categories. Evaluators use numeric categories in variables because it is easier to use numbered categories in statistical software and easier for data entry.
- Variables with text (or "string") values, which provide a description about something, usually in text form. This is usually in the form of an open-ended response or information from a person being evaluated and subsequently recorded into a dataset. Text/string variables can be used in qualitative analyses or later converted to quantitative variables.
- Variables that contain an object, such as photos or small pieces of work that a person has completed. Each data entry in a dataset could be an object like a photo, video, or document, such as the assignments that students complete in a class setting.
Variables are used to sort the data into meaningful categories that an evaluator can use for analysis. Each variable contains therefore contains very specific information about things that actually happened or existed at the time it was recorded.
Evaluators will ask many research questions about how educational products and experiences worked. To answer these questions, they need to analyze data that contain information about actual things that happened or existed during implementation. Data can capture information about participants, as well as about the learning environment itself.
Common types of data that are collected in educational evaluation include:
- Competence or achievement. Data on how well someone did or the quality of their work is most frequently captured as data in a learning environment. People are frequently tested on how well they can perform skills and recall knowledge, which can be recorded as numeric values of the level of quality. Evaluators can also collect data on how much people learned, or how much new skills and knowledge they gained over time.
- Psychological and emotional states. Instruments typically capture data on people's psychological state, how they feel, and what kinds of things they are thinking about while they are participating in an educational experience.
- Demographic and background information. Evaluators often want to know information about a person who participates in a learning experience. They will ask questions about the person that can be recorded as variables, such as age, gender, where they live, family income level, and even personal questions like their interests, hobbies or career goals.
- Environmental factors. It may be of interest for evaluators to collect data about the contexts in which learners participated in the educational experience. This includes environmental factors such as room descriptions and setup in the case of face-to-face learning.
- Implementation factors. When a product is implemented, it is not implemented uniformly every way, every time. Evaluators benefit from collecting data about implementation, such as how much an implementation adhered to the intended plan and whether there were any forces or challenges that were faced by a person who was implementing the product.
- Event logs. Event logs are recordings of specific occurrences of events with a timestamp. Each separate event is a new case in the log. Typically also records the identification of the people who participated in/performed the event. Participation of people in the learning environment is the most typical event that is captured in event logs.
- The use of analytics makes heavy use of event logs and uobtrusive data collection.
- Observations, journals, and reflections. Evaluators that examine learning environments and products will often make observations about the product or keep journals and reflections on the types of things that occur and are worth documenting. The evaluators will record these observations as both structured/prompted and open text formats. Open text is just free writing by the observer, where structured formats helping the evaluator narrow the types of things that they are observing. These text documents can be used as data sources of actual events and things that existed for future analyses.
The full amount of data that are available to someone for a study is frequently called the data corpus or data universe.
In the strictest sense of the English language, "data" is the plural form of the word, so data are plural. A single piece of data is called a datum, or more common a data point. However, grammatically speaking, you can refer to data as singular or plural.
Data as evidence for making claims about research questions
To make a claim or conclusion about a research question, strong evidence should be provided. In evaluations, collected data provides most of the evidence of how, why, and with what effect an educational product worked. Ample applicable evidence and counter-evidence from data makes any conclusions about research questions strong and trustable. In other words, you can't just claim something happened or something was good, but instead need to provide data, evidence, and examples for how and why things happened. Evidence should be reliable and valid related to the research question. Evidence that is a true and trustable representation is typically that which (1) is collected systematically (i.e., with specific, reproducible steps), (2) the collection procedure is well documented, and (3) attempts to remove potential biases of the observer and evaluator are mitigated.
Data represents the evidence of something occurring or existing within a learning experience - it is a recording of a specific thing occurring or how a person feels. In the case of educational evaluation, data are collected by research instruments and used in analysis methods to answer research questions about if, how, and why things occurred related to learning and participation in a learning exercise.
Tips and Tricks
- Data are the backbone of any evaluation. You will need to consider what kinds of evidence that you need to answer your research questions in a trustable way, which translates directly into the types of data that you will collect.
- Data are collected from instruments, which are the tools for recording data. Make sure that the instruments that you use give you the kinds of data that you need to use your chosen analysis methods.
- Data come in all shapes and sizes. Consider how your data are being recorded and what the variables can tell you about the events and things that exist for which they record information.
- Researchers convert qualitative data all the time into quantitative data. In many cases, you can transform qualitative data into measurable or usable quantitative data if you need to at a later time.
Related Concepts
Examples
None yet - check back soon!
External Resources
None yet - check back soon!