Analytics, monitoring, and data capture (EdTech)

Analytics are automated data collection technologies with built-in analysis methods that provide immediate, up to the moment information on performance within an educational product. Analytics are most often used for monitoring learners' performance and subsequently promoting behaviors based on what the analytics reveal through actionable assessment.

Definition

Analytics are technologies that are usually apps and software that are operating in the background of and educational technology. Analytics software automatically capture, process, and analyze data on how people are using the technology or are participating within the learning environment.

Learning analytics is the specific field of research and design that focuses on automated and unobtrusive data capture and analysis methods that can be used for improving educational performance, usually in real time while an educational product is being implemented.

Unobtrusive data collection is that which occurs in the background of the learning experience and does not require the participant to do any special activity to provide data, such as taking a test or quiz, or completing an assignment. With unobtrusive data collection, the software that operates the learning environment automatically captures and processes data and provides the results to learners, teachers, and designers based on the types of information that is helpful to each participant.

Additional Information

Analytics are a useful tool for observing how a person interacts with a digital technology and the degree to which they are participating in the learning activities within digital learning environments.

Analytics systems are increasingly valuable as an evaluation tool because they do not require learners to take additional time to complete a data collection instrument like a test or quiz. Instead, the software automatically captures data about a person's interactions and use of the system and analyzes it automatically. This type of background data collection is called unobtrusive data collection, or that the learner is not bothered when data about their participation are collected (i.e., it does not intrude). Unobtrusive analytics methods are an instrument for collecting data for evaluation and often are integrated with analysis methods to process the results data that are collected. However, it is still up to the student, instructor, or designer to interpret the results and take action based on the findings.

Observed behavior is the focus of analytics. The automated capture of data that analytics perform can typically only capture specific behaviors that are observed by the digital system itself - that is, these systems can only capture when people clicked on things, performed certain tasks in the system, or they interacted with the interface in various ways. The analytics system can keep a log (i.e., a digital database file) of every interaction that a person does, along with a timestamp of when that action occurred.

However, analytics systems do not directly measure people's emotional states, affect, or psychological factors, such as whether they feel like they need help or that they don't understand something. Instead, researchers that study analytics often conceptually relate observed behaviors to psychological factors, in that when a specific behavior is observed it is linked (or correlated to) a psychological state. Thus, with analytics, some behaviors can serve as a proxy for a person's emotional or psychological state. This is an area of continued research and promises great potential for offering personalized learning experiences in educational products.

The field of learning analytics seeks to develop methods for analyzing data that are collected automatically from digital systems. The field is also focused on how data on how a person uses a digital learning environment might represent behaviors that are conducive to learning, or how a learner's observed behaviors might be related to psychological traits and affect that the learner is experiencing. Much research has been done in this field and evaluators can find many ideas by researching methods in learning analytics.

Tips and Tricks

• Think about how your educational product uses digital technologies and how each interaction taken by participants or teachers can be recorded in a digital log by the software you are using. You may need assistance from programmers to build an analytics logging system, but this is a possibility for having fine-grained, high-resolution data about how people use your product.
• Consider the types of research questions that you would like to ask if you had a real-time, formative analytics system in place to analyze performance. What kinds of things would you want to track and what would you want to improve?
• Brainstorm ways that the behaviors of people within a learning environment might indicate the type of psychological state they are in. Do certain behaviors indicate that they are frustrated or they need help with a task? Do certain behaviors reveal that a person is not motivated or not attentive in the activity? With these hypothetical links between behaviors and psychological states and emotions, you can begin to develop research questions that test whether the behaviors are a proxy for the psychological factors or emotions. If a link is demonstrated, then an analytics system can help you track when people need help, are frustrated, are having trouble navigating, or are dissatisfied with the experience - and thus can be corrected!

Related Concepts

• Evaluation
• Instrument
• Analysis method
• Actionable assessment
• AI-based technologies: chatbots, digital agents, analysis (EdTech)

Examples

None yet - check back soon

External Resources

None yet - check back soon