Identification of change and growth (analysis)

The identification of change and growth in a person is a family of research questions that are used for educational evaluation and assessment. The goal of this approach is to identify whether someone changed or grew as a result of their participation with an educational product, specifically whether they learned or experienced some sort of change from when they started to when they ended their participation.

Definition

Change and growth research questions ask whether a person experienced a change from when they started a learning experience compared to when they ended their participation. The change that a person could experience in an educational context typically includes whether somoene learned new knowledge or skills, changed their behaviors, or experienced a change in their affect or beliefs as a result of their participation.

Change and growth questions can also investigate whether a person retained their changes over time after their participation by evaluating participants' knowledge and behaviors at a later timepoint, or if the participation continued to influence further learning.

Additional Information

Research questions in evaluation and assessment that examine a person's change and growth gauge whether a person actually learned or changed from a learning experience. This is in comparison to measuring whether a person can simply demonstrate competence with knowledge and skills.

It is important to demonstrate change in a person's knowledge, skills, or dispositions, as a person may have already had the desired knowledge, skills, or dispositions at the beginning of their use of an educational product. If they already had the knowledge to begin with, they stood to poorly benefit from the experience in the first place. By measuring change or growth, an evaluator can demonstrate that a person actually increased their skills or knowledge, or grew as a result of their participation.

Change and growth is measured by collecting data at multiple time points on a person's knowledge, attitudes, and behavior (among other things) that capture evidence about learning objective achivement. Specifically, at least two time points are required for participants to be assessed in their change or growth: one at the beginning of using the product and one at the ending. Additional time points for assessment of participants may be added, if desired (such as from formative assessments). This type of two-time evaluation approach is typically called pre-post evaluation, or that the person is evaluated at pre-product and post-product times with those results being compared to see if the post evaluation produced a higher result. If the result is higher in the post evaluation, then the participant is said to have learned or grew during the timeframe between the two assessments.

It is of course possible that any growth between two time points is evidence for the product effectively causing that change. However, to robustly demonstrate that a product alone is what caused the change, additional rigorous experimental methods are necessary to rule out and control for any possible alternate explanatory factors or influences that could have caused a person's growth. These alternate explanatory factors include their level of participation with the product and its design features, a person's comparative performance with a similar alternative product, or the manner in which they were randomly and validly selected to participate in the study to prevent any intentional preconditions or assumptions the participant may have.

Identical instruments or methods of identifying and collecting data and responses from participants are typically used at each time point, such as giving participants identical tests, assignments, or tasks at both the pre and post timepoints. Each time point should also be scored or evaluated in the same way to reduce any potential for making incorrect inferences and inaccurate interpretations.

Common research questions related to measuring change and growth

• How much did participants gain in their content knowledge over the course of using the product?
• How much did participants' attitudes and affect change over their time participating in the experience?
• Was there an improvement in participant's skills and knowledge by using the product, or did they already have the knowledge in the first place?
• How much did behavior change over time from when the participant first started the educational experience?

Common analysis methods for analyzing change and growth

• Differences of means tests measure the difference between two or more time points to see if a statistically significant difference exists between the two (i.e., pre-post) continuous (scaled) measures for a collection of cases in a dataset. The common statistical methods used for investigating differences of means are t-tests, ANOVA (analysis of variance), MANOVA (multivariate analysis of variance).
• Time series analysis is a set of statistical methods that investigate changes over time and over multiple data points to identify areas of dramatic change. A time series is a set of repeated measures of a variable that are taken at multiple times.

Note: It is beyond the scope of this knowledgebase to expand on each of these methods. It is recommended that researchers and evaluators seek additional training, web resources, or courses on individual methods they would like to use.

Tips and Tricks

• It is always useful to investigate if a person can perform the desired skills or demonstrate desired knowledge at the end of a learning experience. However, the participant may have already had this knowledge to begin with. It is challenging to make a claim that someone learned from a product unless you can demonstrate that a positive change or growth existed for most participants with the product. If you would like to make a claim that your product helps with learning, that is it helped people change and grow, you need to demonstrate that learning occurred during the evaluation by measuring and analyzing whether a change between multiple time points occurred.
• Methods for measuring and investigating change and growth identify whether learning occurred (as measured by changes in assessments). This is in contrast to only measuring competency at the end of the experience. Thus, there is a difference between learning and competency when it comes to evaluation purposes and the types of claims that you can make about how well your educational product works.

Related Concepts

• Evaluation and assessment
• Summative assessment
• Formative assessment
• Analysis method
• Quantitative analysis
• Learning objectives and outcomes
• Learning effectiveness evaluation
• Competency evaluation
• Competency vs. learning

Examples

None yet - check back soon!

External Resources

None yet - check back soon!