Gilbert Importante
Asst. Professor
University of Southeastern Philippines
gimportante@beam.org.ph

The article examines how the Mindanao eLearning Space consortium has developed and implemented a quality management system (QMS) to support continuous improvement in meeting the learning needs of the Department of Education in Philippines. It begins by providing a brief background on the establishment of Mindanao eLearning Space and the impetus for change. A review of literature on quality, continuous improvement, and organisational change follows, which serves as the underpinning rationale for the quality approach taken to the instructional design of eLearning products. Discussion of the instructional design process also analyses how the QMS has been structured at each step to maximise learning and enhance the outcomes of eLearning products

Mindanao eLearning Space (MiSpace) is an organisation offering flexible distance education to public school teachers and administrators in the form of eLearning. It is a consortium of the Department of Education (DepED) and the University of Southeastern Philippines (USEP) funded by the Australian Agency for International Development (AusAID) through Basic Education Assistance for Mindanao (BEAM).

Initially, the MiSpace eLearning products on CD-ROM were trialled in blended and online delivery modes. The trial found that many of the learners in Mindanao did not have reliable Internet access so a blended delivery mode is now favoured. Interestingly, the use of CD-ROM is supported by the findings of an eLearning study of Sloman (2004) in the United Kingdom which revealed that 73% of the learners preferred CD-ROMs as the most popular form of eLearning than the other forms.

A key challenge for MiSpace instruc-tional designers and web developers was to produce quality eLearning courses more attractive to learners in terms of learning cost effectiveness, time efficiency and flexibility. This is where process mapping and the development of quality procedures and documentation has had a significant impact.

The need for quality
Learning is vital to poverty alleviation, social development and economic growth (AusAID 1996). Indeed, providing a quality education enables people to develop creatively and emotionally and acquire the skills, knowledge, values and attitudes necessary for responsible, active and productive citizenship (UNESCO 2004). It is also reflected in the goal of the AusAID-funded Philippines- Australia Basic Education Assistance for Mindanao (BEAM) project which is a member of the MiSpace consortium. Similarly, other consortium members (the University of Southeastern Philippines (USEP) and the DepEd) share a commitment to quality education and training.

Given the common commitment to educational quality, the MiSpace consortium resolved to develop quality eLearning products to address the learning needs of DepEd administrators, school heads and teachers in Regions XI, XII and ARMM of the Philippines. This represents a potential target audience of as many as 50,000 learners.

The decision to explore eLearning as a possible delivery mode was based on an assumption that it can provide a quality learning experience. It has the potential to overcome the problems of diluted understandings and interpretations through the appropriate selection of instructional design strategies for course content, interaction and assessment items which can provide consistent feedback to learners and make the learning experience experiential and reflective. Indeed, the debate about eLearning has now shifted from whether it is a suitable mode of delivery to how it can be implemented to offer a quality learning experience (Cantoni et al. 2003).

Another key driver in the need to ensure quality was to minimise variation of early MiSpace products. The initial low levels of capacity, particularly in instructional design, demanded a means of explicitly capturing the knowledge and skills of eLearning product development to allow others to follow the process. To this end, the development of a quality management system was considered an imperative in knowledge management.

The sustainability of MiSpace beyond the period of financial assistance provided by BEAM was also an important factor in deciding to pursue quality as a business imperative. Quality was recognized as being critical to the longevity of MiSpace in a commercial context. Indeed an empirical study of 3,000 strategic business units support Deming’s (1986) theory of quality which unequivocally proved that quality drives market share and profits (Buzzell & Gale 1987 cited in Omachonu & Ross 1994). This relationship leads to continuous quality improvement and increases productivity and market share.

Quality systems development
The first step in the change process was to gain stakeholder commitment to quality and to form a representative team to develop a vision for MiSpace. This culminated in the development of a quality policy statement and a commitment to a plan to map and document eLearning processes to ensure conformance to agreed standards.

A series of workshops were held by the quality team to clearly define MiSpace processes and procedures to demonstrate that quality is a managed outcome for all eLearning products. The ADDIE instructional design model, illustrated in Figure 1 below, was used as the framework for developing a detailed flowchart of MiSpace activities.

Interestingly this process revealed many differences in understandings and interpretations for developing eLearning products and reinforced the need to have a shared understanding in order to achieve the quality vision.

Templates, forms and checklists have been developed to support staff to undertake MiSpace processes and produce the necessary quality records. Non-compliant work must be corrected before approval is given for work to proceed to the next phase. Approved outputs are then document controlled so that the title of the document, author, version number, date and authority approving it are recorded. A functional quality management system (QMS) has been established and piloted with the development of a new version of a social marketing training program. Subsequent courses have been developed using the QMS in school physical resource management, constructivism in teaching and learning, change management, multiple intelligence, higher order thinking skills, educational technology and research methods and statistics.

A feature of the QMS is that it has a web interface and is not a manual which sits on a shelf that is seldom used. The advantage of having an electronic QMS is that it permits quality documents to be readily accessed by all staff and updated to capture improvements in MiSpace processes. Furthermore anecdotal evidence suggests the QMS has increased job clarity, reduced rework and significantly improved morale as staff take great pride in the products that have been released.
MiSpace product development
Before being able to develop a QMS, it was necessary to firstly define quality education and training. Quality learning occurs where learners are supported, the environment conducive to learning, the content relevant and facilitative learning processes are used and are linked to clear outcomes (UNICEF 2000). It also required MiSpace staff to research and consider the many complex ways in which technology can be applied for education and training given the broad spectrum of eLearning products that are possible.These dimensions of educational quality are addressed in each phase of MiSpace product development.

Training needs analysis (TNA) is a process of comparing actual and required levels of performance which may be habitual or have arisen as a result of a performance problem, introduction of new systems or technologies (Tovey 1997). This phase was initially given minimal attention in early eLearning prototypes resulting in materials that varied from what learners required.

To increase the quality of the analysis phase, procedures were prepared to clearly define the required competency. After identifying the competency area, the first step taken is to undertake a literature review of the competency area to ensure that the competency will reflect best practices (Clark 1995). It is followed by interviews and focus group discussions with subject matter experts (SMEs) and supervisors to define standards of work performance as well as current performance levels.

Analysis also extends to the learning environment. Many factors influence the quality of adult learners including their health, previous education, training and life experiences as well as organizational and family support (Driscoll 2002). Therefore MiSpace procedures also include a detailed task analysis of potential target groups to derive characteristics of learners as well as to define their needs and expectations.
The other consideration in the analysis phase is to consider the instructional setting to determine the suitability of a course for eLearning within the present technology capability (FGCU, 2003). This includes an assessment of possible learning strategies and the media which may be needed to support them. This entails careful consideration of the computer literacy of the target group, their access to computers, the specifications of those computers and the availability of local technical support.
The end product of the analysis phase is a draft competency standard and TNA report. These documents are then submitted to the MiSpace Management Committee (ManCom) which reviews the draft competency for compliance and approves further development.

The required output or quality record for this phase is a curriculum design document. A quality tool, in the form of a curriculum design document, ensures that the needs analysis and competency standard for each course are reviewed. It also requires the number of modules be defined and sequenced. The design document includes an abstract of the course, learning outcomes, delivery methods and an assessment plan.

Learning strategies are devised to address background, cultural and religious characteristics of learners and tackle any potential inequalities deriving from gender or ethnicity. Another key driver of the learning strategies employed is the blended learning approach that MiSpace products take. Blended learning is a mix of ‘various event-based activities, including face-to-face classrooms, live eLearning and self-paced learning’ (Singh 2003, p.52). This approach recognizes the limited information technology infrastructure and relatively basic levels of computer literacy of target learners. However an equally important reason for favoring blended learning stems for the pedagogy underpinning MiSpace eLearning programs.

MiSpace has undertaken extensive research and has consulted widely with stakeholders to determine how technology can be used appropriately to promote high quality teaching and learning. The framework guiding course development and delivery is based on seven principles of good practice in undergraduate education (Chickering & Gamson 1987) which draws on the different contributions of learning theories.

Most importantly, this approach emphasises a community building approach to support the emergence of communities of practice (CoPs) in various competency areas such as asset management, social marketing and financial management. More specifically, the blended learning model of MiSpace combines self-paced computer-based training with classroom training, job aids and reference materials. Most importantly it also nurtures relationships with experts, peers and communities of practice.

Research indicates that the community building approach of linking learners with one another, with experts and existing CoPs can be self-sustaining. In this sense, the approach makes better use of resources, promotes lifelong learning and advances practice to achieve better outcomes (Wenger et al. 2002). This claim is supported by a recent survey by the American Society for Training and Development (ASTD) which found that blended learning is considered to be the most effective approach for training (HR Focus 2005).

Broad parameters for the selection of instructional media are also set based on the analysis of the instructional setting using Bates’ (2000) ACTION model.

Reushle et al. (1999) advocate that the instructional media should allow learner control to navigate through material following their own path with a help facility and glossary. MiSpace products include these features in the user interface template allowing random navigation to modules, lessons, exercises and quizzes.
In addition, a feature of the CD-ROM is a Learner’s Toolbox section which provides some general study tips covering effective reading, writing, time management and assignment writing techniques among others. The purpose of this section of the CD is to create a supportive environment for learners to ease them into a new mode of delivery and meet program requirements.
Assessment of learner performance is another integral element of MiSpaces’ quality products. Opportunities for practice in each lesson of the course formatively assess learner performance and provide rich constructive feedback which is immediate. Consistent with Wright & Conroy (1988), self-tests at the end of each module provide answers and explanations as well as a simple calculation of learners’ test scores. In addition, written assignments are structured early in the course, usually a draft analysis or plan, to encourage interaction between the instructor and learners to get feedback.

A project plan is also prepared for an eLearning project at this stage. It documents the course purpose and rationale, stakeholder consultations and target audience characteristics. It also clearly states requirements, deliverables, team member responsibilities, a production schedule and a budget.
Similar to other quality records, the curriculum design document and project plan require the approval of the MiSpace ManCom which verifies that quality standards have been met. Once approved, resources are committed to the next phase of development.
The development phase of ADDIE requires blueprints for the presentation of content, opportunities for learners to practice and be assessed (Driscoll 2002).

Evaluation Results

Much of the quality of course content is assured by the task analysis for the specific competency, its validation with subject matter experts and approval of a validated competency standard. In this sense, the curriculum is standards based and is linked to well-defined performance criteria.

Content is developed using a storyboard template which is an important quality document. It ensures that instructional designers clearly state learning outcomes and learning objectives, use appropriate sequencing strategies and graphic organisers, and select appropriate media. Clearly, the quality of course content is also dependent upon the competency of instructional designers. To this end, substantial investments have been made in developing instructional design guidelines for codified knowledge and on-the-job training to develop tacit elements of instructional design.
The quality of the content is also dependent upon its authenticity. Good quality content uses contextualised problems to encourage skills development as well as knowledge acquisition (UNICEF 2000). Similarly content must be gender sensitive and promote peace to strengthen learners’ ability to prevent and resolve conflict at individual, group or community level (UNICEF 2000). To ensure that these dimensions of content quality, a reference panel of subject matter experts and practitioners review it. After the content is approved, it proceeds to web development.

A standardised user interface serves as a template for MiSpace products. This approach ensures that all MiSpace products have the minimum required functionality while reducing development testing and quality assurance costs through code and module reuse. Similarly, in terms of content, common study support elements are included on all MiSpace products such as the “getting started” section and “learner’s toolbox”.

One of the innovations that MiSpace has included in the QMS is an interaction library which has reduced the burden on instructional designers in developing interactions. The interaction library is a catalog of elements that an instructional designer can incorporate into a course that the learner can interact with, which addresses, in part, the active learning and prompt feedback principles of good practice in undergraduate education. Each interaction in the library contains a sample, defines the information that the instructional designer needs to provide (for example responses for incorrect and correct answers) and provides work instructions for web developers. This quality tool has paid substantial dividends by reducing rework of instructional designers’ storyboards due to omitted information, reducing the effort required to develop interactions, and improving the consistency and quality of interactive elements by providing developers with detailed implementation instructions.
Another recent innovation has been to strengthen testing of MiSpace products. Early product prototypes had mixed functionality on different platforms with some interactions that did not work or provided very limited feedback. These technical and pedagogical problems are common causes of frustration for learners (Hara & Kling 2000). This was addressed by developing quality platform, unit, integration and user acceptance testing procedures and documents. Now, when defects are identified, they are returned to the appropriate team member, whether the graphic designer, web developer or instructional designer for action. Once all defects are corrected, the course is released.

Another important implication of the blended learning approach has been the introduction of learner and trainer guides to accompany course CD-ROMs to support delivery. Wright & Conroy (1988) contend that these print-based support materials form the most basic structure for any mix of media used to deliver instruction. Implementation Implementation is arguably the most difficult phase as ‘good trainers can make a poorly developed program work well and a well developed program work great…bad trainers can make neither work’ (Clark 1995). It also poses a challenge for quality management which aims to reduce variation- MiSpace interprets this to mean minimizing the variation of learning outcomes- not to restrict variation in training techniques. The implication in practice is that all trainers must be competent as professors, facilitators and resource persons for effective delivery of MiSpace products.

The emphasis of MiSpace’s QMS in the delivery phase is therefore on the training of trainers. All trainers must undergo a program in delivering MiSpace eLearning products and be assessed as competent before they will be accredited. Certification of MiSpace trainers attests to their competency in creating conducive learning environments, ability to skillfully use a range of training techniques and assessment methods to meet the different needs of learners. The QMS also ensures that courses are structured so that student progress and trainer performance can be monitored. Where any significant variations in performance are identified, appropriate remedial action is taken.

The learning environment is another key element of implementation phase procedures. Quality tools have been developed for contracting training providers to provide minimum levels and number of equipment for learners, limit class sizes and abide by MiSpace policies to create a safe and inclusive learning environment.

Frydenberg (2002) espouses that student services before, during and after an eLearning program should constitute an integral part of eLearning quality standards. This reflects the customer focus of any quality management system. For learners interested in enrolling in an eLearning program, an accurate course description outlining the learning outcomes, content and entry requirements is issued. In addition the enrolment form includes a learning contract that is designed to secure learner commitment and supervisor support so that an environment conducive to learning is created.
The blended learning approach permits some face-to-face contact with learners, which provides the opportunity for support services to be extended in delivery. Consequently all sessions are structured to provide support services. In addition, technical support can be accessed through MiSpace or local DepEd offices. In instances where a learner’s progress falls behind, MiSpace initiates contact with the trainer and learner to assess any problems and take any necessary corrective action.
The evaluation of MiSpace programs ensures that there is follow-up with learners and supervisors post training to determine the extent of transfer of learning to the workplace as well as the organisational impact.
Evaluation
Training evaluation determines a training program’s effectiveness in meeting its intended purpose; producing competent employees (Tovey 1997). Evaluation is therefore the quality assurance component of a systematic approach to training.However, evaluation does not fit neatly as the final phase of ADDIE if considered as a linear step in the process (Driscoll 2002). MiSpace recognises this predicament as reflected by the central position of evaluation linked to all phases of the ADDIE model illustrated in Figure 2. In practice, this means evaluation is conducted in each phase of ADDIE. As described above, there are a number of quality controls employed before work can progress to the next phase. For example, the training needs analysis and draft competency standard must be approved before proceeding to the design phase. Similar controls are used in other phases.
For summative evaluation of eLearning products, the methodology used draws on the work of Kirkpatrick (1994) who advocates four levels of training evaluation: (i) reaction level, (ii) learning, (iii) transfer, and (iv) impact. Reaction level evaluation uses a quality evaluation tool drawing on the constructs of the seven principles for effective undergraduate education to gather data on the learning experience of participants. Learning is evaluated based on assessment results which are quality assured through assessment policies. Transfer of training to the workplace is evaluated by undertaking semi-structured interviews with a sample of learners and their supervisors by phone where possible to keep costs down.

There are a number of quality outputs for evaluation beginning with the evaluation of the training needs analysis and competency standard and ending with an evaluation report.

The evaluation shows that learners had rated the elearning program at a very satisfactory level having a general mean of 4.15. It further showed that the seven principles as mentioned had been presented in the programme and the learners had rated it at a very satisfactory level.

The number of hours spent by learners on browsing the CD-ROM on average is 8 hours and 28 minutes per week.
The learning of participants was evaluated based on assessment results which are quality assured through assessment policies. The participants grades ranges from 1.0 (excellent) to 1.75 (very satisfactory), they were assessed through an approved assessment policies that were presented in the learners guide.

Transfer of training to the workplace is evaluated by undertaking semi-structured interviews with a sample of learners. Learners’ feedback were recorded and showed that the program had been used in the workplace. Action planning, implementation and evaluation were among the most commonly used modules in the workplace. Impact of the programme was not yet evaluated since there was still a very short time since the end of the training. As such evaluation on this level to measure organisational performance is not yet conducted.

Quality plays a critical role in the teaching/ learning process. This implies that ‘the quality of the finished product is the direct result of the quality throughout the process used to create it’ (Nichols 2002, p.1). At MiSpace, this notion has been translated into the dogma that ‘the quality of our product is only as good as the weakest link in the chain’. Hence a total quality management culture has emerged where the QMS has become the lifeblood of the organisation

"Exciting news! Elets Education is now on WhatsApp Channels Subscribe today by clicking the link and stay updated with the latest insights!" Click here!