14 Against Outcomes
There are two primary ways to evaluate the effectiveness of the tool in promoting your course outcomes: a data-driven comparison of student performance with and without the tool and your own potentially subjective assessment of the ease with which those outcomes were achieved. This guide recommends gathering as much data as possible, so to the extent that it’s available you will want to find ways to capture both of these measures.
Ideally, you will have some data to start with on how previous students performed in the course — or even on particular assignments. If so, you will be in great shape to conduct a direct comparison and find out whether the technology made a measurable difference. If you don’t have previous data to work with, consider whether any of your colleagues are teaching to similar outcomes without using the digital tool. You may also want to compare student data from the beginning of the course with student data at the end of the course. In this latter scenario, you will have to use your best judgment on how much of their progress can be attributed to your excellent teaching efforts and how much might be attributed to the tool. Which brings us to more subjective assessments.
Unless you are implementing this technology in the very first class you have ever taught (in which case, congratulations on making it to this point!), you have a reasonably sound understanding of how much your students typically learn over the course of a term. There are always outliers, and calling on a ‘typical experience’ has certainly a more complicated prospect since 2020. Still, you likely have a sense — something that you can back up with hard data or bolster with student reflections — of the degree to which the tool helped your students to more effectively meet the outcomes for your course. The important step here is taking that subjective interpretation and putting it into words. If you take the time in the moment to write out your perception and why you feel that way, it can be a useful opportunity to consider the nuances of your impression. In addition, that articulation becomes a valuable artifact for your future reflections.