Introduction
Assessments are integral part of any educational organisation’s course curriculum. Typical assessment mechanisms comprise written and practical examinations, presentations, etc. In most of the written exam papers in universities, subjective type questions comprise a significant share. Managing these written exams include number of complex and repetitive tasks such as maintaining different types of questions on a topic, setting a exam paper using paper patterns and question bank, conducting exams, evaluation of answer sheets, publishing of results, etc. Each task as well as the associated data is critical in nature and usually multiple faculty/administrative staff are involved in these tasks. All these tasks are bundled with lot of paper work as well. In absence of an effective management mechanism, these tasks can be tedious, time and resource consuming, and also error prone.
Online Assessments are now featuring in number of online distance learning programmes and also becoming popular in regular courses. Online assessment tools automate and ease many testing related tasks thus saving time and resources.
Most of the tools available are web-based and can be accessed using a browser on any machine connected to the system server via the network. Among the available testing tools, open source tools are gaining more popularity due to lower cost and the growing online community support available them.
Though there exist many systems which allow conducting online examinations, a very few deal with subjective type questions. The Educational Technology Unit alongwith the Open Source Software Division of CDAC Mumbai has developed a tool for Subjective Type Examinations (SuTE) to address the concerns related to managing subjective type examinations.
SuTE Our Approa ch
SuTE is an open source assessment tool mainly focussed on handling university type (written) examinations online. Many tedious tasks related to paper setting, answer sheet evaluation, etc are made online and easy using the tool. This article briefly describes how SuTE handles Question Paper Management and Exam Management (Conduction and
Evaluation) for written exams.Evaluatio n Only registered
Questio n Pa per Manageme nt As we know, the structure of typical university exam question paper consists of multiple sections such as long answer questions, fill in the blanks, match the following, essay type, short answer questions, etc. Sections can also have optional sections. Question paper setting requires preparing new questions/
reusing questions from old papers and mapping appropriate questions to sections/ sub-sections in the basic paper pattern. SuTE maintains questions, paper pattern and question paper separately, thus facilitating independent management of these tasks. It also facilitates reuse of paper attern and questions. Questions of various types can be stored in the question bank along with parameters such as difficulty level, maximum marks, topic, etc. Paper patterns can be reated comprising various sections and subsections. Each section in a paper pattern consists of laceholders for questions. The placeholders are also associated with maximum marks for he question and the type of question that it will hold e.g. subjective, match-the-following, tc. This information is useful for the paper creator to map appropriate questions to the place olders while preparing the question paper. To ease the complex task of setting the question paper, the paper pattern and question bank are displayed on a single screen. The faculty can elect a specific paper pattern, search for the questions with specific complexity, type, opic, etc and map questions to paper pattern on the same screen using the intuitive drag and rop facility [see Figure 1]. Various validation checks are done and related error essages/colour indications are displayed to ensure syntactic correctness of the paper attern/question paper when it is being constructed.
Exam Co nductio n and Evaluation
the test online within a restricted time frame. The question paper displayed on the screen has imilar format of a traditional question paper comprising of: set of instructions followed by various sections and questions within the sections, each section and question clearly ssociated with respective marks and instructions (if any). From the displayed list of questions, he student can select question he/she wishes to answer and then key in response in the given text area. Student is also provided with a facility to mark a particular question as ‘incomplete’ n case he/she wishes to revisit the answer. Authorised examiners can manually valuate the subjective questions in answer sheets of each student online. There are wo types of evaluation modes available – ‘Question-wise’ and ‘Student-wise’. ‘Student-wise’ is the traditional way of evaluating complete answer sheet of a student at a time. ‘Question-wise’ evaluation mode allows evaluation for all students for a particular uestion. This mode can be useful to do comparative analysis of students’ performance and is also useful if separate faculty are handling specific set of questions. Some useful statistics such s – number of unanswered questions, number of answered questions, etc for students and number of questions checked, unchecked, partially checked for the faculty are provided y the system. This information is useful to track progress of attempting/correcting a paper. Similarly, objective type questions in the answer sheet are evaluated automatically. This educes the time and also possibility of human errors involved in manual checking of answer heets. Paper patterns and questions papers are stored in XML format facilitating interoperability of data. Current Stat us The initial version of SuTE, along with installation anual and user manual is available for use and download from our website http://www.cdacmumbai.in/sef. We have also sent it for evaluation to variety of users – peers, teachers and students of different educational institutes (actual end users) as well as ome corporate contacts. Feedback is being collected on a variety of evaluation parameters uch as usability, effectiveness (teacher and student perspective) of the system, technical spects, etc. The current version of the system has limited support for security and recovery echanism. So the system is recommended to be used for class tests and other internal tests nly. Conducting university exams online using such a tool introduces its own challenges which cannot be overlooked. Student performance may get affected bytheir typing speed and lso by their level of comfort in computer usage. Also there is need of sophisticated answer ditors to facilitate drawing diagrams, charts, maps, etc. The security mechanism eeds to be robust enough so as to avoid intrusions, unauthorised access, etc. Also it should have a reliable mechanism for recovery in case of network failure, etc. The system, though currently usable and suitable for internal tests, is being improved in many dimensions to ake it more comprehensive to meet the university-type testing needs.
