Comparing CEI and SET data to your peers
Introduction
The Course Evaluation Instrument (CEI) collects student feedback about their learning experiences in a course. The Student Evaluation of Teaching (SET) collects feedback about students perceptions of the teaching quality of a particular teacher. Standard reports on a CEI or SET provide the data as a frequency distribution and a mean. This information, however, can be difficult to interpret on its own. To improve interpretation, current mean data can be compared with historical data for the same course or against historical data of comparable courses. This is useful in allowing academic staff to benchmark themselves against their peers, particularly if claims for promotion are being made. They also provide useful data when applying for teaching awards.
Where does the comparison data come from?
Current CEI or SET data can be compared with data :
- from the previous three years
- in a discipline grouping of your choice (the Commonwealth DEST definitions of field education are used)
- at either undergraduate or postgraduate career level
- at either internal, external or offshore modes (using Medici classifications)
How is the comparison made?
This is essentially a step approach.
- Data from the core items of the CEI and SET from the previous three years across UniSA are collected into a common database. For SET data there is no recording of names against the data; it is pooled into an anonymous base.
- The data for each item is separated into 72 different 'bins'. Each bin represents a different context (12 disciplines, undergraduate/postgraduate, internal/external/international. (ie 12 x 2 x 3 = 72). The number of bins created depends on sufficient data being available.
- Quartiles are then determined by arranging the data into a sequence from lowest to highest. The median score is then determined. In the lowest half of the median another median is determined and similarly for the top half. This creates 4 segments or quartiles with Q1 being the lowest median group and Q4 being the highest.
- The individual's data is then highlighted in red in the relevant quartile.
How do I access my rating?
When you access your accumulated feedback within the CEI tool, you are also able to view data comparing all classes for which the CEI was run by choosing to View Course Data.
In the CEI and the SET, you are also able to compare your results against those of academics teaching similar courses within the same discipline over the last 3 years by choosing to compare course data.
Pull down menus are used to select the appropriate DEST identified discipline which is most appropriate for comparing your course data. The years of comparison are listed at the top and there are options to choose the career or mode of the comparison data.
Interpreting the data
Once submitted the comparison with the quartiles are made and displayed in a histogram. The use of quartiles is an easy way of making comparisons and establishing benchmarks. However, one must always ensure that proper conclusions are made as a result of data comparison. For example, a particular course results might be lower than the benchmark because it was delivered to a large class. Also the discipline within which the course is placed may also create some variations due to the broad nature of the categorisation. Data from each question should also be considered in relation to the overall results so that attention is focussed on the key issue for students.
For assistance in interpreting your data and/or identifying processes to improve evaluations, contact the Academic Development team.