Tuning text analytics for research students’ writing

UTS doctoral researcher Sophie Abel has just presented her latest work to the Ascilite conference, in a paper and talk entitled: Designing personalised, automated feedback to develop students’ research writing skills. — browse her talk slides below

The ‘verbology’ of learning

Educators often write learning objectives that describe what students must do or be like to demonstrate that learning has occurred.  Bloom’s Taxonomy provides a means of classifying the associated depth of thinking.  The six levels of this taxonomy are: remembering, comprehending, applying, analysing, evaluating, and creating.   Traditionally, classifying a learning objective on this taxonomy focusses on verbs in the learning objective.

Verbs at the remembering level are usually associated with rote memorization (e.g. list, cite, define).   Verbs at the comprehending level require the student to understand something to a depth sufficient to explain something to others (e.g. describe, communicate, illustrate), whereas applying verbs require the use of skills or knowledge in practice (e.g. conduct, demonstrate, employ, follow).  Analysing verbs require the formation of connections between ideas and concepts (e.g. compare, contrast, distinguish).  Evaluating verbs take this up a notch, requiring the formation of a judgement or critique (e.g. appraise, determine, judge, recommend). At the highest level, creating verbs capture the synthesis of something new (e.g. construct, design, devise, generate).

While verbs are commonly used in Bloom’s classification, sometimes adjectives and adverbs can play a role too.  For example, to “critically analyse” something is an example of evaluating, whereas to “analyse” by itself is just analysing because the former expects a critique or judgement. Similarly, to “report important findings” is different than just “report findings” since the former also requires a judgement about the significance of the findings.  This is further complicated since some words can either be a noun or verb depending on how they’re used (e.g. design, document).  This means that it’s a little more detailed than just looking up words in tables. You also need to be able to break words in learning objective down into their respective parts of speech.

At ascilite 2018, we’ll be introducing a new app called Blooms TL for iPhone and iPad to assist with classifying learning outcomes on Bloom’s taxonomy to make all of this a bit easier. The classification in this new app uses tables containing verbs and other parts of speech that are generally indicative of a given Bloom’s level.

View slideshows with screenshots of the app in action.

The new app will be available soon. If you’d like to jump the queue and download a beta copy, contact Brian von Konsky at Curtin University (B.vonkonsky@curtin.edu.au) for instructions.

Learn more about this work:

Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives

Stanny, C. J. (2016). Reevaluating Bloom’s Taxonomy: What measurable verbs can and cannot say about student learning. Education Science, 6(37), 12.

von Konsky, B.R, Zheng, L., Parkin, E., Huband, S., and Gibson, D.C. (2018). Parts of speech in Bloom’s Taxonomy classification. 35th International Conference on Innovation, Practice and Research in the Use of Educational Technologies in Tertiary Education, ascilite 2018, Geelong, Australia.

Writing Analytics R&D

The R&D program at UTS has developed and piloted a tool to provide automated formative feedback to students on their writing. The research publications below document how we’re designing this, and what we’re learning.

In addition, the team runs regular workshops and tutorials (2016/2017/2018) bringing together some of the world’s leading researchers to reflect on the state of the art and future of automated writing analysis. Continue reading “Writing Analytics R&D”