Using Online Course Analytics To Improve Learner Engagement
As instructional designers, how can you keep a pulse on how engaged your learners are in interacting with the course content and with each other? In a classroom, you can see wandering eyes or a bored expression. In an online course, it may seem challenging to see how learners are engaging behind their computer screens. Thankfully, you can use online course analytics to gain a comprehensive view of how learners are engaging with the course. In this article, we’ll discuss some indicators of low engagement in an online course, as well as intervention tactics you can use to help learners become active again.
How to View & Approach Your Online Course Analytics
Before diving into the analytics of an online course, it is important to first understand the learners’ context so that you can understand the metrics appropriately. Factors such as the topic, course type (external or internal training), and learner motivation can lead to different expectations and benchmarks for learner engagement in a course. For example, some learners who are busy professionals may join a course without anticipating completion. Others may join a course with the goal to receive a certificate of completion. You can take proactive steps to understand your learners’ motivations and expectations for taking the course through surveys or profile questions. Possible survey questions include:
What are your motivations for taking the course?
How important is achieving the course certification?
How much time do you expect to allocate to the course each week?
By understanding your learners’ motivations, you can be better informed about which metrics and thresholds in the online course analytics you will want to pay attention to. This will also lead to the types of intervention tactics you may want to use. Below are three scenarios where analytics can be used to help improve learner engagement in a course. Since every course and community is unique, we recommend that you regard these as suggestions and find ones that work best for your course.
Scenario #1: In the first week, some learners are not active
We’ve seen that the first week of a course is the most important stage for intervention, as learners who drop off here may never come back. What could it mean if a sizeable group of your learners are not active in the first week?
Learners didn’t realize or just forgot that the course is starting
Learners realize they can’t commit to participate
Learners don’t know where or how to start the course
Intervention ideas Immediate:
Send an announcement or email with action items to students, including completing their profile.
Reach out to individuals to ask if they need any assistance
Email targeting to reach out to learners who haven’t logged in
Post an interesting question in the Discussion forums and respond to some discussion threads
Send Pre-course emails leading up to the course to remind learners that the course is starting soon.
Include an onboarding message on the homepage having a list of action items
Have learners introduce themselves in discussions to kick-off social activity
Scenario #2: Learners are not submitting assignments on time
If your course using project deliverables towards completion, you can use the learning platform’s analytics to see what percent of learners submitted, read the instructions but didn’t submit, or didn’t reach the instructions at all. By using analytics and understanding why your learners haven’t submitted assignments, you can engage in targeted interventions to help them complete the course deliverables.
What could it mean if your assignment submission targets are missed? Possible Reasons
Assignment level is too difficult or too easy for learners
Learners don’t see the value of assignment (not connected to learning objective)
Learners feel that there are too many assignments or they require too much time
Follow-up with individuals through targeted messaging
Highlight exemplary submissions to the community
Create varying levels of assignments (easy/hard)
Create a team assignment for the next course (if not already)
Add peer evaluations for the assignment (if not already)
Allot more time for TAs/Instructors to provide feedback
Implement course catalyst program
Scenario #3: Teams are not forming or are not very active
If you are using teams in an online course (for example, one hosted on the NovoEd platform), you can use NovoEd Analytics to see how many teams have formed and how many have joined a team. You can also see a quick snapshot of team activity within their workspaces by looking at the number of messages exchanged, meetings scheduled, and files uploaded in the Team Activities section.
What does it mean if teams aren’t forming? It could mean that participants don’t know how to create teams or don’t know how to join one. If course-long teams aren’t active, it could mean that participants don’t know how to get started.
Learners don’t see the value of working in teams
Learners don’t realize they can find, join, and/or create a team
Learners don’t know who to ask to join their team
Learners haven’t received response from team requests or doesn’t realize they can form their own team
If course-long teams aren’t active, it could mean that participants don’t know how to get started (e.g. because of a lack of defined team roles)
Intervention ideas Immediate:
Reach out to encourage individuals who haven’t formed teams
Describe the value of forming teams (able to share knowledge, expand network etc.)
Create a step-by-step tutorial on how to form or join teams
Create a team formation discussion topic for people to look for teammates
Offer discussion starters for teams to use to get to know each other
Re-assess the way you form teams (self-formed, manually assigned, algorithmically assigned) based on the course objectives and your audience
Determine the best time in the course to describe how teams work and when you ask participants to start forming them
Create good profile questions for participants and teams that will be help participants choose teams and teammates
From measuring activity in the first week of a course to identifying the level of team activity, you can utilize online course analytics to help you identify where you can intervene to support or encourage learners to become more engaged in the course. We encourage you to try these intervention ideas and others, as you sharpen and target the tools you use to increase learner engagement in a course.
Along with hand-washing, disinfecting, and social distancing, Zoom has become part of life for many during the pandemic. However, the initial thrill of seeing distant colleagues on video conferences has given way to fatigue, anxiety, and distraction as we struggle to stay engaged.