Guannan Li
Product/UX Design Lead
Case Study: Research-Driven Iteration
Background
We introduced a new feature to allow teachers to lock/unlock each part of a lesson to individual student early 2020. After the feature was released, we heard
'The new features/changes are making teachers’ life difficult'
The feedback is learned through
-
weekly meetings with the customer support team
-
tickets from Usabilla we received directly from teachers
This is how the feature works:
Plan and Process
3 months after the feature is released, we decided to run some research to understand if it is a natural learning curve or we need to have an iteration. The following steps are taken
Step 1. Understand
We would like to understand 1) if it is a common issue experienced by a good percentage of users 2)What people mean when they say it is difficult to use
Step 2. Investigate
To investigate the issues, we turned to our analytics tools. One of our findings is there is a common pattern of teachers moving around on different pages trying to unlock the lesson for all students. By introducing the flexibilities on individual student, there are more steps to unlock for all.
All Videos
In order to validate the finding, we took a further look at the percentage
-
38% of lessons have everything unlocked for all students from beginning
-
26% of lessons are unlocked part by part but always to all students
Step 3. Solutionize
One of the steps we decided to take is to prioritize and simplify the steps on the main lesson page to
-
Unlock all parts of the lesson at one go to all students
-
Unlock one part of the lesson to all students
Besides, based on the analysis of the false click when users attempted to lock/unlock lessons from other pages, we also added the most common action of unlocking everything for all students at one go to other pages.
Iteration 1: we added the lock all/unlock all button to the overview page and they are accessible always. So users don't have to go one step further to take these actions.
Iteration 2: the action to lock/unlock all parts of the lesson for all students is added to higher-level overview pages too. Theser pages are for teachers to see all students' progress in all lessons.
Step 4. Test
Before proceeding, the proposal is tested with
-
5+ colleagues from customer support team
-
5 teachers
The new design is agreed by all participants and some minor design/copy optimization was made.
Reflection
The iteration around the lock/unlock feature was a bit of disruption because it is the daily task teachers handle. A couple of things around the process could be done to address this type of issue better in the future
-
Better product metrics (KPI) could be set up and tracked so we identify these issues earlier and understand the impacts of the changes
-
For major changes like this, we could run A/B testing or only release to a sample user first