Evaluation Toolkit
Video Tutorial
Click on the words in the figure below to see the definitions of the variables and sub-variables
×
Usability
Information on the site is kept up-to-date, with no dead ends or stale links. The learning environment must be easy to navigate. Careful organisation of the learning resource should minimise the number of steps or “clicks”.Videos
Links
Usability 101: Introduction to Usability×
Theoretical Foundations
Constructivism acknowledges that one’s learning is an active, ongoing process. Learning is subjective because individuals’ cognitive processes differ. They perceive their environment and experiences differently and link their perspective to prior knowledge. As such, learning is unique, fluid and contextual.Social constructivism takes learning to the collective level where people learn with and from others by interacting and collaborating in social contexts, or communities of practice or inquiry.
Connectivism is a digital learning theory applicable to networked, online environments. When people connect to and participate in an online community, they interact dynamically with others using rich multimedia resources. This environment provides new ways for people to make connections between ideas and concepts, and the capacity to create knowledge by incorporating current information.
×
Ethical Considerations
Learners must be made aware of their ethical responsibilities, including issues around confidentiality and respect of other learners. Student Code of Conduct Policies regarding attendance, participation, consequences for cheating and plagiarism must be explicitly stated. Students must be made aware that plagiarism is flagrantly dishonest and that engaging in it will result in dismissal from the course and perhaps from the institution. A code of conduct policy should address attendance requirements. Absence—failure to reply to email or engage in a discussion or chat, for example—is lack of participation. One of the most effective techniques to promote participation in eLearning activities is to ensure that participation counts towards a student’s grade. If you choose to use this form of motivation, your code of conduct policy should so indicate, and both your assignment policy and your course syllabus should specify how attendance will be figured into the final course grade×
Learner satisfaction
Digital education provides education and training solutions that correspond to learner and community interests and labour market needs. Learners appreciate the convenience, flexibility, and access to resources that enhanced digital educational opportunities provide. Digital education removes physical barriers and travel limitations, empowering learners who live in remote or rural areas to participate in higher education study units or take classes in subjects that are not offered in their locations.×
Academic staff satisfaction
Academic staff benefit from the flexibility and convenience that digital education provides both themselves and their students. Academic staff and training professionals who integrate digital technologies into their teaching gain satisfaction as a result of increasing the learning opportunities, satisfaction and success of learners in a digital world.×
Increased learner knowledge and skills
Relevant, authentic content ensures that learners attain intended learning outcomes. Digital education that includes complementary core competencies to match work and study skills, and provides versatile study paths ultimately leads to securing and maintaining better employment.×
Transfer of knowledge
Carefully planned learning activities such as scenarios, problem-solving case studies, simulations, reflection, discussions and collaborative group activities enable learners to seamlessly transfer new knowledge to real-world applications.×
Positive impact on the organization
Reusable, adaptable and scalable teaching resources can lead to innovation and change within organizations. Organizations can leverage effective digital education to become competitive, relevant, and innovative and enable academic staff and trainers to attract students, compete internationally and meet the needs of learners in the 21st century.×
Relevant
Each digital education session begins with a statement of three to five learning outcomes that are the objective of the session. Study unit designers then select content to align with and lead to these learning outcomes. Relevant study unit content faithfully reflects problems and issues that arise in real-world situations. Learners engage in activities that present the same type of cognitive challenges they will encounter in an applied environment. This enables the acquisition of information, concepts, and skills that are meaningful, comprehensive and relevant to the present or future workplace.×
Organised and chunked
Content is organized into succinct segments or ‘chunks’ that present a comprehensive overview of the information needed to attain the learning outcomes. Content progresses logically, builds on previous information, and provides learners with a sense of pacing and completion. It is presented objectively, through unbiased language, matching the learner’s level of understanding. Content is organised into clear sections such as introduction, information presentation, exercises, interaction, conclusion and take-home messages. The estimated amount of time required to complete each section is noted, tracked and adjusted as needed.×
Supported with relevant resources
Additional readings and resources support the content overview to ensure that topics are covered in appropriate breadth and depth. Academic articles, videos, and websites provide background information and alternative presentation options that address varying learning styles and needs. The resources encourage learners to reflect on their own thinking and learning processes. Additional resources can encourage social negotiation and critical thinking, which can help generate insight and promote the elaboration of concepts and ideas.×
Grounded in theory and best practices
Theory and empirical research provide the basis for effective digital education content. Academic staff refer to their personal and professional experience, pedagogical best practices and adult learning theory to create activities that facilitate the transfer of theory into practice. Academic staff also take advantage of learners personal and professional experience, so they not only learn from the tutor, but with and from each other. Academic staff from accredited learning organizations share core competencies and partner with subject matter experts from private industry to design learning solutions that are closely aligned with the skills required in the workplace.×
Intentional use of technology
Selective and relevant use of technology supports teaching and learning and results in educational solutions that assist with meeting future competence needs and ensure smooth, flexible, and convenient digital study paths for learners.×
Accessibility
The online platform must be robust, easy to navigate and freely available to all students and academic staff. The online learning experience provides flexible access to customised content that allows participation anytime and anyplace. The content and online learning platform needs to be designed in such a way that it is easy for anyone to use. Course delivery utilises the full power of the Internet as a delivery tool, augmenting course content through convenient links to online resources.×
Incorporates pedagogical strategies
Learning experiences are appropriately designed and facilitated by knowledgeable educators using evidence-based teaching practices. Academic staff incorporate a variety of instructional strategies and activities and apply a range of modes of learning to accommodate diverse learning styles, and to maximise and leverage the experience of each learner. Academic staff use pedagogic frameworks specific to digital education to guide the design, delivery and evaluation of their learning sessions. Academic staff or tutors are available within the learning environment to enable the comprehension of information and encourage self-directed learning. They solicit feedback, engage learners in problem-solving experiences, and provide positive reinforcement. Pedagogical considerations include providing learners with a clear roadmap of expectations and learning outcomes, determining appropriate teaching and learning strategies, and determining the type and amount of content and appropriate level of instructor support. Pedagogical strategies also guide the development of formative and summative assessment.×
Engaging and interactive
Effective digital education promotes and facilitates interaction with the content, as well as among learners and with the tutor. Engaging activities stimulate learners’ interest and understanding and facilitate the transfer of knowledge and skill into practice. Peer-helping activities enrich the learning experience for adult learners, enabling them to assume personal responsibility and connect past learning experience to the current one.×
Promotes reflection
In effective digital education, tutors encourage learners to reflect upon concepts and ideas learned by responding to thought-provoking questions that stimulate discussion and critical thinking. Content and design are leveraged to promote independent reflection, as well as in in response to other learners.×
Facilitates teaching, social, and cognitive presence
Effective digital education implements strategies to encourage participation and engagement by enabling learners to experience teaching presence. Teaching presence includes the perception that the tutor is available, reading posts, responding quickly to questions, and providing timely, relevant, and constructive feedback. Effective digital education is designed to promote a sense of social presence or community engagement among learners and with their tutor. Having the opportunity to get to know and trust both the tutor and other learners, particularly at the beginning of a digital education study unit, enables learners to identify with the community and to communicate purposefully in a trusting environment. Effective digital education encourages the learner to be cognitively present in the learning environment by incorporating activities specifically designed to challenge them to analyse, think critically and reflect. Tutors ask thought-provoking questions and design problem-solving scenarios that require the learner to apply their new knowledge or skill to a work-related or practical situation.×
Tutoring
Success in the digital education environment is directly related to how present and engaged learners perceive their tutor to be within the virtual classroom. Learning, engagement, and enjoyment of the experience is enhanced when learners feel a responsiveness equal or superior to face-to-face instruction. Tutors communicate clear expectations and instructions, identify opportunities, challenges and agency, provide timely constructive feedback, announcements and updates, and create a learning community where learners feel comfortable and supported. Tutors address small problems before they become significant, thereby helping learners avoid stress, frustration and attrition.×
Prompt feedback and service
Within a digital education environment, academic staff or tutors provide feedback on assignments and respond to email queries in a timely manner. To facilitate the ability of learners to access support from academic staff and peers promptly and conveniently, the learning environment also provides services such as interactive question and answer forums and online office hours. Class announcements, study unit notes, and assignments are readily available.×
Technical and media training
Prior to the start of a digital education session, learners receive a thorough introduction to the learning environment, so they can comfortably access and participate in the digital education platform.×
Professional development for academic staff/tutors
Academic staff/tutors are trained to design, deliver and evaluate effective digital education. The training provides the basic knowledge to facilitate a digital education learning session while enabling staff to experience what it is like to be a learner in the environment. Academic staff also have access to professional development through their organizations to keep up to date with digital education best practices. Training and professional development that is certified by an accreditation body can help to ensure quality and stimulate participation.×
Technical support
Administrative and technical support are freely available to both students and academic staff to help them use and access the technical systems supporting the digital learning environment. Step-by-step instructional videos can be used to walk learners through downloading software or performing technical tasks. Face-to-face or video conferencing sessions can help learners who have challenges interacting with the technology.×
Collaborative learning environment
The digital education environment enables learners to work together to explore a significant question or create a meaningful project. Study units and programs enable knowledge to evolve through discussion, reflection, and collaboration. Learners have the opportunity to take initiative and responsibility to listen, question, and think critically within the community of fellow learners. Learners are empowered by learning with, from and about one another. Tutors model the sense of community by supporting and providing constructive feedback to learners and encourage learners to do the same.×
Inclusive and safe
Effective digital education establishes a learning community where learners feel comfortable and have a sense of belonging. Success, engagement, satisfaction, and learning are dependent on the support and sense of identity that comes from developing relationships within the learning environment and being a valued member of a community. Ethical principles guide tutors as they work to establish an inclusive and safe learning environment. These principles include learner confidentiality, learner development, pedagogical competence, an appropriate and supportive approach to sensitive topics, respect for learners and colleagues, and valid, fair assessment. The tutor also explains and enforces principles of internet etiquette. Options are provided to support varied learner needs and abilities.×
Participatory culture
To encourage learners to actively and safely participate in the collaborative learning environment, a set of policies and expectations are established regarding etiquette and protocols for reflection and discourse within the community. This culture of participation can be facilitated by having learners create profiles that are available to other learners, so they get to know each other. A café-style forum where learners can socialise within the context of the learning environment can stimulate participation within the community.×
Organizational funding and buy-in
The ability to create and deliver effective digital education depends on funding and buy-in from the organization planning to implement it. Developing a fully realized, robust, and scalable digital learning platform requires coordinating input from academic staff, instructional designers, software developers, videographers, and graphic designers. Delivery of successful digital education programs depend on ongoing, multi-faceted support including administrative assistance, training for academic staff, academic help for learners, as well as technology and media support for all participants.×
Scalable and customisable
Effectively designed digital education can be scaled up or down according to class or staff requirements. It is easily adaptable for changing class or learner needs and can be delivered by several tutors for many course offerings.×
Statement of ethical expectations
A clear statement of ethical expectations ensures that tutors and learners are aware of their ethical responsibilities when participating in digital education. This includes policies regarding confidentiality, attendance, participation, and consequences for cheating and plagiarism. It also communicates the expectations around demonstrating respect for other learners and academic staff.×
Learner Assessment
Assessment is integrated into the pedagogical strategies and meshes with the overall learning design. Assessment activities that encourage dialogue, collaborative activities, and problem-solving can be more effective than traditional tests. Clear guidelines about how learners should submit completed assignments, due dates, evaluation criteria, and modes for feedback prepares learners for the workload.×
Program evaluation
Program evaluation is a systematic assessment of study unit content and delivery based on feedback solicited from participants. Academic staff conduct brief formative assessments at regular intervals throughout the study unit of study. Formative assessments ask learners what is working and what is not working for them in the study unit. The anonymous feedback enables staff to understand the progress and needs of learners and address minor issues before they become significant. A summative evaluation administered at the end of each study unit assesses the overall study unit to address quality standards related to content, delivery, support structure, community and outcomes. Consistent program evaluation provides quality assurance safeguards and ensures that best practices and cutting-edge knowledge outcomes remain robust and equal or superior to traditional face-to-face learning.×
Video Tutorials
English
Maltese
Italian
Slovenian
Finnish
×
Continuous improvement and emerging design
Academic staff and designers of effective digital education continually assess study units and sessions and, when possible, implement changes to improve the quality of the study unit in situ or for the next iteration. Updating content and design on an ongoing basis ensures that digital education continues to address evolving learning requirements and technological advances.×
Needs Analysis
Understanding the needs of the learners is a prerequisite to effectively planning any digital education event. Curriculum planners should conduct a diagnostic evaluation to determine learners’ needs in terms of content, delivery and support; learning styles and preferences; background experience; current knowledge; and learning goals. Awareness of cultural, social and individual factors that could influence the learning experience are important to pinpoint in order to create strategies and tools to make the program flexible (available anywhere and anytime); accessible (access to computers); feasible (length of program and learning segments); and convenient (online access to resources, registration, support). Planners can gather this information by surveying or interviewing learners who have already participated in similar online learning events or who will be participating in the learning event being designed.×
Outcomes
Outcomes resulting from effective digital education include increased learner and academic staff satisfaction, increased learner knowledge and skills, transfer of knowledge, and a positive impact on the organization.×
Content
The content of an effective digital education program provides all the information learners need to attain the required learning outcomes. Effective contentis relevant, organised and chunked, supported by relevant resources, and grounded in theory and best practices.×
Delivery
Effective digital education is carefully designed using intentionally selected technology to ensure the usability and accessibility of the learning platform. Pedagogical strategies leverage technology to ensure that the learning experience is engaging and interactive, provides opportunities for learner reflection, and facilitates teacher, social, and cognitive presence.×
Support
An effective digital education environment requires ongoing support from tutors, prompt feedback on assignments, and responses to emails and questions within interactive forums. Learners are introduced to the digital education platform to ensure they can engage with it effectively. Academic staff and tutors are trained to deliver and evaluate the program. Continuous technical and media support is available for both learners and staff.×
Community
Effective and user-friendly digital education incorporates community-building strategies to establish a collaborative learning environment that is inclusive and safe. Tutors encourage a culture of commitment to thoughtful reflection, open discourse, experimentation, risk taking, assessment, and analysis.×