May 2009


Views: 214


Assessments are integral part of any educational organisation’s course curriculum. Typical assessment mechanisms comprise written and practical examinations, presentations, etc. In most of the written exam papers in universities, subjective type questions comprise a significant share. Managing these written exams include number of complex and repetitive tasks such as maintaining different types of questions on a topic, setting a exam paper using paper patterns and question bank, conducting exams, evaluation of answer sheets, publishing of results, etc. Each task as well as the associated data is critical in nature and usually multiple faculty/administrative staff are involved in these tasks. All these tasks are bundled with lot of paper work as well. In absence of an effective management mechanism, these tasks can be tedious, time and resource consuming, and also error prone.

Online Assessments are now featuring in number of online distance learning programmes and also becoming popular in regular courses. Online assessment tools automate and ease many testing related tasks thus saving time and resources.

Most of the tools available are web-based and can be accessed using a browser on any machine connected to the system server via the network. Among the available testing tools, open source tools are gaining more popularity due to lower cost and the growing online community support available them.

Though there exist many systems which allow conducting online examinations, a very few deal with subjective type questions. The Educational Technology Unit alongwith the Open Source Software Division of CDAC Mumbai has developed a tool for Subjective Type Examinations (SuTE) to address the concerns related to managing subjective type examinations.

SuTE Our Approa ch
SuTE is an open source assessment tool mainly focussed on handling university type (written) examinations online. Many tedious tasks related to paper setting, answer sheet evaluation, etc are made online and easy using the tool. This article briefly describes how SuTE handles Question Paper Management and Exam Management (Conduction and
Evaluation) for written exams.Evaluatio n Only registered
Questio n Pa per Manageme nt
As we know, the structure of typical university exam question paper consists of multiple sections such as long answer questions, fill in the blanks, match the following, essay type, short answer questions, etc. Sections can also have optional sections. Question paper setting requires preparing new questions/
reusing questions from old papers and mapping appropriate questions to sections/ sub-sections in the basic paper pattern. SuTE maintains questions, paper pattern and question paper separately, thus facilitating independent management of these tasks. It also facilitates reuse of paper  attern and questions. Questions of various types can be stored in the question bank along with  parameters such as difficulty level, maximum  marks, topic, etc. Paper patterns can be  reated comprising various sections and  subsections. Each section in a paper pattern consists of   laceholders for questions. The placeholders are also associated with  maximum marks for   he question and the type  of question that it will hold e.g. subjective,  match-the-following,    tc. This information  is useful for the paper creator to map  appropriate questions to the place   olders  while preparing the question paper. To ease   the complex task of setting the question paper, the paper pattern and question bank are displayed on a single screen. The faculty  can    elect a specific paper pattern, search for the questions with specific complexity, type,   opic, etc and map questions to paper  pattern on the same screen using the intuitive  drag and   rop facility [see Figure 1].  Various validation checks are done and  related error   essages/colour indications are  displayed to ensure syntactic correctness of the paper   attern/question paper when it is  being constructed.

Exam Co nductio n and Evaluation
the test online within a restricted time frame. The question paper displayed on the screen has    imilar format of a traditional question paper comprising of: set of instructions  followed  by various sections and questions within the sections, each section and question clearly   ssociated with respective marks and instructions (if any). From the displayed list of questions,  he student can select question he/she wishes to answer and  then key in response in the given  text area. Student is also provided with a facility to mark a particular question as ‘incomplete’   n case he/she wishes to revisit the answer. Authorised examiners can manually   valuate the subjective questions in answer sheets of each student online. There are  wo types of evaluation modes available – ‘Question-wise’  and ‘Student-wise’.  ‘Student-wise’ is the  traditional way of evaluating complete answer sheet of a student at a  time. ‘Question-wise’  evaluation mode allows evaluation for all students for a particular   uestion. This mode can be useful to do comparative analysis of  students’ performance and is  also useful if separate faculty are handling specific set of  questions. Some useful statistics such  s – number of unanswered questions, number of answered questions, etc for students  and number of questions checked, unchecked, partially checked for the faculty are provided   y the system. This information is useful to track  progress of attempting/correcting a paper. Similarly, objective type questions in the answer sheet are evaluated automatically.  This  educes the time and also possibility of  human errors involved in manual checking of answer   heets. Paper patterns and questions papers are stored in XML format facilitating interoperability of data.  Current Stat us  The initial version of SuTE, along with installation   anual and user manual is available for use and download from our website We have also sent it for evaluation to variety of users  –  peers, teachers and students of different educational institutes (actual end users) as  well as  ome corporate contacts. Feedback is being collected on a variety of evaluation  parameters  uch as usability, effectiveness (teacher and student perspective) of the  system, technical  spects, etc. The current  version of the system has limited support for security and recovery   echanism. So the system is recommended to be used for class  tests and other internal tests  nly.  Conducting university exams online using such a tool introduces its own  challenges  which cannot be overlooked.  Student performance may get affected bytheir typing speed and   lso by their level of comfort in computer usage. Also there  is need of sophisticated answer  ditors to facilitate drawing diagrams, charts, maps, etc. The security mechanism   eeds to be robust enough so as to avoid intrusions, unauthorised access, etc. Also it should have a reliable mechanism for recovery in case of network failure, etc. The system, though  currently usable and suitable for internal tests, is being improved in many dimensions to   ake it more  comprehensive to meet the university-type  testing needs.

Click to comment

Leave a Reply

Your email address will not be published.

Most Popular

To Top