Manish Upadhyay, Co-Founder & Chief Evangelist, LIQVID; AmitavaMaitra, Independent Ed Tech Consultant
Between us, we have more over 30 years of experience in the field of technology aided and enabled education. Fuelled by the desire to see and evaluate the application of technology in the school and higher education in the government owned and operated schools and institutes in India, we decided to look for data on the efficacy of ICT interventions. Though we searched long and hard, we could find nothing on the MHRD web- sites. To add to our agony, we found many crucial links did not work. We then decided to use the “brahamastra”, an RTI application. We thought that a simple application would do it. Enthused by the idea we looked up the process – it was simple – a `10 postal order and the list of questions to be sent to the Public Information Officer. But that is where things got complex. The MHRD web- site left us clueless as to whom to send the RTI application. There seemed to be a lot of divisions within the MHRD that seemed to be having something to do with school and higher education and ICT interventions. It was not clear as to who exactly did what. We decided to bite the bullet and thus, sent RTIs to all, hoping that our respondents will do their bit.
And thus began the next phase, the wait and the watch game. Some weeks later, we received the first shock. From most of the people, we got answers saying that we had addressed the postal order to the wrong person. It was to be in favour of the Pay and Accounts officer (PAO) and not the PIO. Apart from polite, terse and very official sounding letters, we did not get any relevant information save for the communication about our errant ways.
The world over, educational interventions, ICT or otherwise, are judged by forming two groups – one an experimental group and one control group involving statistically equivalent populations, and then objective measures on student performance as assessed through standard tests are used as key parameters to judge the efficacy of interventions
The notable exception was a PIO who bucked the trend and actually answered some of our questions (even on an inaccurate postal order). We even had the privilege of receiving a letter from a PIO in the PMO telling us that he was forwarding our RTI application to someone in the MHRD, even though we had not sent any RTI to the PMO. Anyhow, undeterred, we sent postal orders in favour of the PAO and waited with baited breath. However, from all but one PIO we paid glowing tribute to, we got terse messages telling us that ‘no such information exists’ (the only response we got was from a mysterious sounding part of MHRD called “Schools-5 Section, Department of School Education & Literacy”). The PIO stated that evaluation reports for ICT interventions exist for four states in India and that by large these reports indicated the results were satisfactory or good. Intrigued and fascinated, we then sent an RTI report asking for copies of the four evaluation reports. We were then told that the reports did not exist in electronic form and so we would have to pay `292 to get xerox copies of the same. Again, we remained true to the cause and sent the Indian Government the money to photocopy the pages. And finally, six months later, we were in the possession of some relevant, concrete information. Curiosity had landed on Mars and we gave each other high fives.
For those of you who are befuddled by the sequence of events here’s the upshot. The Indian Government has planned to spend over 10,000 crores** on ICT in schools and higher education institutes with the belief that this would benefit teachers and students (we presume rightly or wrongly that other people were not the intended beneficiaries of these projects). Now, to verify this conjecture, the government decided to test it doing impact assessment or evaluation studies in four states only across approx 120 odd schools. The evaluation of ICT initiatives in Punjab was conducted by a leading engineering college there and that in Sikkim was conducted by a regional engineering college there. The two reports were essentially hard- ware audits and informed the reader what hardware was procured in contrast with what was ordered and in the various stages of disrepair they were in. * * `5,000 crore is being provided during the Eleventh Plan for providing ICT infrastructure in schools and another provision of `5,000 crore has been made in the Eleventh Five-Year Plan for ‘Education Mission through ICT’ for higher education institutes.
In Europe and many other developed nations, in addition to student performance in standard tests of achievement, ‘inspectors’ observe and evaluate school processes and procedures impartially to determine how a school is performing on parameters such as ICT implementation and integration
The evaluation report of schools in Meghalaya was conducted by a leading government institute in Shillong and was a voluminous one which covered everything from the history and geography of Meghalaya to the tribal nature of the population there. With over 90 percent of this large report (for which we had paid money) devoted to esoteric trivia, the last 10 percent or so declared that all was, by and large, well in the state of Meghalaya with minor points of improvement. Another leading private institute in Manipal conducted the evaluation of ICT interventions in schools in Kerala. This report also relied totally on qualitative opinions based on self-assessment based questionnaires. Overall, the four reports were well below par. While the world over educational interventions, ICT or otherwise, are judged by forming two groups – one an experimental group and one control group involving statistically equivalent populations, and then objective measures on student performance as assessed through standard tests are used as key parameters to judge the efficacy of interventions. In stark contrast, the reports made no such attempt.
In Europe and many other developed nations, in addition to student performance in standard tests of achievement, ‘inspectors’ observe and evaluate school processes and procedures impartially to determine how a school is performing on parameters such as ICT implementation and integration. As opposed to that, the reports looked at self-reportage – answers to questionnaires populated by the very people whose performance was being studied. Very dubious, to say the least
We started off by examining the rigour of evaluation studies in ICT projects in school education. Though we must admit that we were dismayed at the state of affairs, we are far more dis- appointed at the process of finding data and information. From the difficulty in getting information through RTIs, to the fact that though we got photocopies of obviously neatly word processed documents, we could not get hold of electronic versions of the same, the situations looks rather bleak. And even more critical, while all this information could have been proactively put online by the MHRD as part of the ‘Duty to Publish’ charter of the RTI which mandates the Indian Government to proactively publish information on its websites, we had to wage a long battle of sorts to get relevant information (the word relevant is
loosely used here).
Much hue and cry has been made over corruption in the government machinery. But, little attention is given to the colossal inefficiencies in the system as our experience with both the RTI process and the rather inferior quality of evaluation reports indicates. There is obviously a correlation between inefficiency and corruption, but we believe both need to be fought with equal zeal. As ordinary, concerned citizens, we have a duty to keep up the pressure.