Posted on

Text Analysis: Using Quantext to Explore Teacher Quality Surveys in Higher Education

In this self-guided research project I explored how learning analytics could be applied teacher quality surveys in higher education. This was my final assignment for the Learning Analytics course in my MET.

Evaluation surveys are controversial among educators in post-secondary. Important decisions including tenure and promotions can be based on student evaluation surveys despite concerns around their validity.

I wanted to explore how text-analysis could be applied to these surveys. I used a text-analysis tool called Quantext to explore how long answer questions in a survey or even potentially student portfolios could be analysed for the purpose of instructor evaluation. I was pleased with my early experimentation and look forward to applying text analysis to my future assessment work.

Below is an informal video walk through of my project that I created for my classmates.

You can read the full version of my report below:

Text Analysis with Quantext Report

Posted on

Splash and Ripple Framework

The Splash & Ripple framework has been an important guide throughout my career. It was my first introduction to program evaluation. I was working on a project for a nonprofit and the foundation that provided the funding required an outcomes-based assessment as evidence that their money was being well spent. I followed the guide closely as I drafted and executed and evaluation framework. Almost ten years later I still refer back to Splash & Ripple.

This guide has been published and republished in different formats by various organizations. The various editions of this framework can be found on different nonprofit and government websites. The popularity of it is due to the straightforward description of the language of evaluation. It offers clear explanations of key terms like inputs, activities, outputs, outcomes, and indicators. It is particularly helpful to nonprofits with small staff and budgets attempting their first outcomes-based evaluation to meet their reporting requirements.

The metaphor of a pebble being dropped in water is used to frame evaluation. The activity or project or program is like the act of dropping the pebble. There is an initial splash (outputs) that create ripples (outcomes and impact) throughout the water.

Coyne, K., & Cox, P. (2008). Splash & ripple: Using outcomes to design & manage community activities (4th ed.). Calgary, AB: Plan:Net Limited and Strathcona Research Group.