AI natural language onboarding: From static to dynamic

Coursera's onboarding is where personalization begins.

When learners tell us their role and the skills they want, we build them a Pathway: a personalized collection of courses mapped to their goals. Learners with a Pathway collection on their homepage have a higher enrollment rate than those without one, and they stay on the platform longer, which increases paid enrollment rates and revenue.

The onboarding experience is built with multiple pages, asking learners about their goal, roles and skills interest, current role, and highest level of education. Coursera uses this information to personalize the recommendations on the page.

I started to dig into Amplitude with my product partner, Nikhil Raman and engineering manager Craig Pullar and we found fascinating data throughout the funnel.

Of 100% that register on Coursera, only 60% move past the first screen (goal question), then another 20% drop after the next screen (role question), followed by a smaller 5% drop (skills question), and it just goes down from there. The important thing to note is that those that do complete onboarding, they are more likely to enroll in content they care about.

The most recent test before this was to improve completion rate, and the hypothesis was that since learners care about content that meets their interests, we decided to move the role question earlier (Var1) vs. starting with the topic question to narrow down to role (Var2). Surprisingly, control still won, because it had the least amount of drop-offs to later questions within the onboarding flow.

During a yearly planning session, the core team and I created a quick prototype of what natural language onboarding may be to help inform next steps for the pod. We even plugged in a real GPT key so the chat could respond responsively and dynamically.

Due to the high interest and buy-in from leadership, the team gained full traction to build towards what AI onboarding at Coursera could be.

The best way to understand where to bring Coursera's AI direction was to develop a baseline of where AI was at the time. The field of AI moves fast, so it's best to develop meaningful ways to use AI without it going stale or it being added friction.

During our discovery phase of the project, we realized that we only had legacy research around onboarding, and a decent amount of research on how learners perceive AI. We knew certain facts like:

• Learners wanted easily answerable question on the splash screen
• Learners were willing to answer as many questions as possible to get great recommendations
• Learners would not feel comfortable answering too many questions without seeing added value

We needed to understand more about how learners perceive AI and how it can be a tool during the onboarding experience, as well as throughout discovery.

If we introduce a pervasive onboarding experience, and use conversational AI as a familiar and natural way for users to communicate their goals and interests, then learners in discovery mode will be more likely to share explicit intent.

This will enable Coursera to deliver personalized pathways, increasing user engagement, content relevance, and overall conversion.

At its core, we decided to test learning what the engagement was for the test design, which has entry points to the AI experience. Leveraging AI patterns, we wondered if learners would interact with the input field vs prompts.

The prompts were built with Coursera's goal in mind to personalize the homepage for the learner. The equation was: [Skill for Role] aka [Figma for Product Designers].

After the test ended, working with data analyst Xianing Xu, we found that engagement of the AI pills was quite high, though the drop-off throughout the onboarding funnel was still an issue, so we decided to iterate. Now we wanted to understand if we could have a hybrid approach for the splash screen: control vs hybrid vs full-AI.

With all of the learnings from data science and Amplitude, I built a prototype with Claude to showcase what we learned and what some next iterations may look like. The team found this useful, as it was powered by data, and influenced planning for next steps.

With the ideas above, I began iterating and creating new hypotheses. I believe that learners still want more guidance that is easy to work through when leading towards onboarding with natural language. I wanted to hone in on the fact that learners who did interact with the input field received stronger Pathways recommendations, which created conversion. However, 90% of learners used the pills. This made me think about improving the quality of the pills to the strongest and top ones, while keeping the input field for the next iteration.

This project is currently in progress and there are planned iterations to increase onboarding completion rate and paid enrollment rate. Stay tuned!