The problem of labels in e-assessment of diagrams
Association for Computing Machinery (ACM)
MetadataShow full item record
In this article we explore a problematic aspect of automated assessment of diagrams. Diagrams have partial and sometimes inconsistent semantics. Typically much of the meaning of a diagram resides in the labels; however, the choice of labeling is largely unrestricted. This means a correct solution may utilize differing yet semantically equivalent labels to the specimen solution. With human marking this problem can be easily overcome. Unfortunately with e-assessment this is challenging. We empirically explore the scale of the problem of synonyms by analyzing 160 student solutions to a UML task. From this we find that cumulative growth of synonyms only shows a limited tendency to reduce at the margin despite using a range of text processing algorithms such as stemming and auto-correction of spelling errors. This finding has significant implications for the ease in which we may develop future e-assessment systems of diagrams, in that the need for better algorithms for assessing label semantic similarity becomes inescapable.
Journal of Educational Resources in Computing;
Jayal, A. and Shepperd, M. (2009) 'The problem of labels in E-assessment of diagrams', Journal on Educational Resources in Computing (JERIC), 8(4), p.12. DOI: 10.1145/1482348.1482351.
Article published in Journal on Educational Resources in Computing (JERIC) in January 2009, available at: https://doi.org/10.1145/1482348.1482351.
Showing items related by title, author, subject and abstract.
Jayal, Ambikesh; Shepperd, Martin (Taylor and Francis, 2009-02)A challenging problem for e-assessment is automatic marking of diagrams. There are a number of difficulties not least that much of the meaning of a diagram resides in the labels and hence label matching is an important ...
Jayal, Ambikesh; Shepperd, Martin (2010-04-14)This project deals with improving undergraduate assessment by using automated marking fordiagrammatic coursework. Automated marking of diagrammatic coursework (e-Assessment),offers many potential benefits including timely ...
Students Understanding of Alcohol Nutrition Labelling and the Subsequent Impact on Attitudes and Alcohol Consumption Brown, David (Cardiff Metropolitan University, 2017-06-01)Background Although obligated to inform consumers of the Alcohol by volume (ABV%) and Unit quantities of alcoholic products, producers rarely display Calorie (Kcal) information on their package labels. Students are ...