Exploring an AI-native model for personalization and onboarding
How I turned a standard onboarding flow into an AI- and LLM-powered experience.
Coursera's onboarding is where personalization begins.
Coursera's onboarding collects learners' goals, roles, and skills to create personalized learning Pathways. Learners who complete it receive more relevant recommendations and are more likely to enroll.
The funnel analysis revealed a fundamental tension: the more information we asked learners to provide, the better we could personalize their experience…yet every additional step created another opportunity to drop off.
This led us to a bigger question:
Could conversational AI gather rich learner intent without making onboarding feel like a questionnaire?
The most recent test before this was to improve completion rate, and the hypothesis was that since learners care about content that meets their interests, we decided to move the role question earlier (Var1) vs. starting with the topic question to narrow down to role (Var2). Surprisingly, control still won, because it had the least amount of drop-offs to later questions within the onboarding flow.
During a yearly planning session, the core team and I created a quick prototype of what natural language onboarding may be to help inform next steps for the pod. We even plugged in a real GPT key so the chat could respond responsively and dynamically.
Due to the high interest and buy-in from leadership, the team gained full traction to build towards what AI onboarding at Coursera could be.
A quick prototype using Google AI Studio to gain buy-in from leadership
The best way to understand where to bring Coursera's AI direction was to develop a baseline of where AI was at the time. The field of AI moves fast, so it's best to develop meaningful ways to use AI without it going stale or it being added friction.
Capturing features on a grid, like voice, assistants
Patterns that Coursera could leverage
During our discovery phase of the project, we realized that we only had legacy research around onboarding, and a decent amount of research on how learners perceive AI. We knew certain facts like:
- Learners wanted easily answerable question on the splash screen
- Learners were willing to answer as many questions as possible to get great recommendations
- Learners would not feel comfortable answering too many questions without seeing added value
We needed to understand more about how learners perceive AI and how it can be a tool during the onboarding experience, as well as throughout discovery.
Working with my product partners to develop the wireframes and focused hypotheses, we finalized on 3 prototypes. In collaboration with my product researcher Olwen Puralena, we decided to do moderated user-testing sessions to better understand how learners interacted with these interactive prototypes. Concept 1 is a logged-in homepage experience with onboarding questions embedded in the AI sidepanel.
Concept 2 takes on the form of what we see quite often these days: a full screen that enables natural language communication between platform and the user.
Concept 3 is the existing control experience with additional AI capabilities like showcasing a summary of their answers and AI "Coach" creating a top recommendation for learners.
We entered research assuming conversational AI might simplify onboarding by letting learners express themselves naturally.
What we learned was more nuanced:
- Open-ended input increased cognitive load. Learners didn't always know what to say.
- AI needed to make its value visible. Feedback such as recommendation strength helped learners understand why sharing more information mattered.
- The strongest experience was likely hybrid. Learners wanted the flexibility of natural language with enough structure to keep them moving.
Always fun to be in FigJam!
Here are a few things we learned along the way below.
Learners' eyes lit up when they saw this interaction that I built with Claude Code.
When learners found that they could upload their resume to extract data to power recommendations, they were extremely pleased.
Learners really enjoyed that a career statement could be generated at the end because it gave inspiration.
If we introduce a pervasive onboarding experience, and use conversational AI as a familiar and natural way for users to communicate their goals and interests, then learners in discovery mode will be more likely to share explicit intent.
This will enable Coursera to deliver personalized pathways, increasing user engagement, content relevance, and overall conversion.
Setting boundaries for how the LLM would respond was a fun challenge with my product manager and lead engineering partner, Shiyun Zhu. Using a real GPT key (GPT-4), we connected the LLM to our experience. We started by creating the goals for AI's conversation, the tone of voice, the messaging, and ensuring it understood the data we needed to gather to power strong recommendations.
Guidelines and guardrails for conversational AI
At its core, we decided to test learning what the engagement was for the test design, which has entry points to the AI experience. Leveraging AI patterns, we wondered if learners would interact with the input field vs prompts.
The prompts were built with Coursera's goal in mind to personalize the homepage for the learner. The equation was: [Skill for Role] aka [Figma for Product Designers].
After the test ended, working with data analyst Xianing Xu, we found that engagement of the AI pills was quite high, though the drop-off throughout the onboarding funnel was still an issue, so we decided to iterate. Now we wanted to understand if we could have a hybrid approach for the splash screen: control vs hybrid vs full-AI.
With all of the learnings from data science and Amplitude, I built a prototype with Claude to showcase what we learned and what some next iterations may look like. The team found this useful, as it was powered by data, and influenced planning for next steps.
With the ideas above, I began iterating and creating new hypotheses. I believe that learners still want more guidance that is easy to work through when leading towards onboarding with natural language. I wanted to hone in on the fact that learners who did interact with the input field received stronger Pathways recommendations, which created conversion. However, 90% of learners used the pills. This made me think about improving the quality of the pills to the strongest and top ones, while keeping the input field for the next iteration.
This project is currently in progress and there are planned iterations to increase onboarding completion rate and paid enrollment rate. Stay tuned!
Next possible test variant: Test the input field with suggestive fields + stronger role/skill combination
Mobile stills from variant 1 experience
Next possible test variant: Leading with stronger role/skill combination, with input field below, indicating natural language
Mobile stills from variant 2 experience