Launched Preply Subscriptions Pilot Project

Preply is an innovative learning marketplace and space that connects millions of learners and tutors, powered by our growth-minded team and technology.

My Role & Contribution:

User interviews
Research analysis
Design + Execution
Strategy + Planning

Goal:

We wanted to build a more robust foundation for our students to sustain the motivation and effort needed to reach their learning goals. With long-term goal of achieving higher retention, LTV & adapting our business model.

Background

We started this project with a very limited understanding of what has to be built. The first challenge we had to tackle was to align internally on the clear scope of the project: its goals, challenges, metrics and potential outcomes.
To begin solving this challenge, we interviewed our internal stakeholders, analyzed competition and customers to make sure we leveraged all the insights in regards to subscription that exist within Preply.

Challenges

The existing package model limits Preply’s long term retention to be composed only of very high intent students who are able to overcome the recurrent friction of re-committing to a new package.
As a result, most of our users do not reach their goals, churning in the middle of their learning journey. It also leaves a negative financial footprint on Preply.

Process

My key focus was to understand product goals of different types of subscription, identify opportunities, and make decisions based on the impact to students and company goals. I was actively involved in defining the strategy, goals, success metrics & roadmap. With the team, I analysed – if subscriptions can be an impactful opportunity for future business model.

Firstly,
I reviewed previous qualitative and quantitative researches and found signals related to the motivations that would power subscriptions.

By analyzing previous researches, found out, that the most successful tutors on Preply have already explored learner motivations and convince them to commit upfront to more lessons by taking advantage of high motivation levels. Also, that learners report they “expect to be motivated” during the learning process. Because motivation fluctuates over the course of the learning process, the barriers that once kept learners from starting classes can reemerge and lead to decreased class frequency and even churn.

Key insight for me was: the extrinsic motivation of a recurring subscription may counteract this drop in intrinsic motivation.

Based on that,
started to interview our internal stakeholders & research competitive landscape.

Reviewed our key competitors and understood, that almost all of them have service-centric approach with subscription as core or parallel offering in their business model. I realised, that we can focus on investigating different variations of parallel and core offerings in tutor-centric subscription, with the aim to be the top-of-the-mind product in working with real tutors, not services.

After,
organised team brainstorming session, where we generated ideas of subscription models.

Generated ideas with focus on: core marketplace product remains the same & core marketplace product has to be transformed. Exchanged ideas with the team to understand our biggest growth opportunities there. Mostly all ideas potentially may add additional value to current core business model. However, not change it.

At this point in our process, I understood that we need to discuss further and deep-dive to each idea to see the opportunity and potential problems.

That is why,
brainstormed the possible execution problems and opportunities of each idea.

This brainstorm was important for me in this step, because we can receive feedback and 360 view on the problems and opportunities from all of our team-members. It also can show the doubts, questions and suggestions of each team-member, that I can think of while designing the flow. That is why, we brainstormed as a team the questions we might have to consider in the near future. Looking back, it was a big surprise for me – how many questions and answers from this brainstorming session helped us and made their way into our MVP.

Finally,
deepdived to each idea.

Using that questions, we focused on deepdiving to each idea. Why? This step potentially can help us choose the best possible experiment to focus on in design. From the perspective of user's jobs to be done & segments, product implications, operational challenges, answers to open questions of each idea. To identify user needs and translating them into my future design solution.

However, faced with the problem. How we can, based on this tables, prioritise and choose the best subscription model? And all of our teammembers can give their vision on the final experiment.

So, came up with
framework that would help us to proceed to the execution stage.

I collaborated with core product team to discuss the possible frameworks that can score and help us to proceed to the execution stage. And explore how we can evaluate and prioritise these ideas.
‍

Solution
‍
Based on discussion with management, core product team and developers, we came up with simple framework that influenced our final decision. We scored each idea by the following parameters (each team member gave a score out of 3):
‍
- Do we believe this product will give us a strategic advantage over the competition?
- Do we believe this product can give more value to Preply users?
- How fast can we can first results? How expensive the tests are?
- Is it safe to test taking our existing business into consideration?
- Are we aligning with the main product pillars of organisation?

Using the framework,
chose the winner and detailed the next steps to launch the experiment.

‍Worked closely with our growth team manager and developers to define both the long-term strategy and the short-term tactics for our upcoming initiative. Defined and evolved our roadmap. For having the initial base of information to proceed with and better articulating to internal executive stakeholders.
‍
What questions do we have?
What possible problems we may face?
What metrics we will focus on?
etc.

For me, it was important that besides managers, also all members of our growth-team gave their opinion and scores. It helped us to see the opportunity from almost all perspectives, like development, design, copywriting & research.

Winner was:
Subscribe to a tutor

The winner had a great balance of strategic and user value, fitted into Preply’s business model and were (relatively) easy to test We can reuse many existing concepts that we know work for Preply currently, whilst pushing students into a 'journey to subscription'.

In this test, we changed packages (current business model) to a monthly subscription at the tutor’s rate. Users can subscribe to their particular tutor.

Started by

Defining success metrics, based on our company North-Star metrics.

Conversions to paid:
How many that view the offer convert to subscribe? How much is the drop vs. packages?

Hours bought:
Do we see an increase in hours bought?

GMV

Churn Rate


Additional health metric:

‍Confirmed hours
Are they actually learning more? Lesson utilisation or Breakage

& the hypothesis based on research and preparations that we did.

By changing the concept to subscriptions, we will retain students better, and affect GMV, Hours Bought while not drastically decreasing CR to paid.

Defined the expectation, that conversion will dip, while GMV will increase (logically outcome of making our product higher commitment + more expensive /upfront payment).

Firstly,
Analysed production flow.

To proceed with design, firstly I had to review our current flow to find, discuss and analyse key places where the flow will be changed. It was important to define on this stage. Without it, we will stuck with legacy problems, bottlenecks and other teams miscommunication. Also, I identified the opportunities and collected feedback from stakeholders. With the aim to fully define the MVP flow.

On this step I collaborated with product leads and developers to understand how we can make decisions that translate our subscription business goal idea into product experiences, turning complex task into simple user-centric experience.

In parallel,
defined decisions & trade-offs for A/B test with developers and data scientists.

- Whom we add and exclude from experiment. With the team, calculated the possible sample size (estimated participants & minimum detectable effect and stat. significance).
- Defined estimates.
- Payment options.
- Test constraints by parallel-running experiments.

and to
clearly define the scope of design work and plan our sprints,

we worked in close collaboration with product manager on creating experiment high-level description. Based on discussions from previous step, we started to answer high-level questions, like:

- What will change in the student experience?
- What will change in the tutor experience?
- What emails do we need to adapt or create?
- How user can manage their subscriptions?
etc.

Began
the design process with entrypoints.

First challenge for us was defining the key entrypoints, where the A/B test would start. Based on previous analysis and review of current product screens, I came up with design suggestions. Stuck with a problem, that we can not change the copy for the key action from "Buy Hours" to "Subscribe", because experiment will be triggered with this action of clicking the button. And control group will also see the wording "subscribe". Solved this problem in close collaboration with copywriter to find the unified wording for both subscribers and package buyers. Changed the wording to "Add hours", that will be more flexible for both groups.

Collaborated with designers from other teams to understand – what additional entrypoints we can also add to the experiment, so more users will see the experiment and we can achieve statistical significant results faster. Came up with solutions in core product, messages, emails and pop-ups that our students see in product.

Iterated on the
next step – pricing plans.

After entrypoints, worked on pricing plans screen – the core step that needs to be as clear as possible to motivate user go forward. For MVP came up with simple solution, collaborating with copy, management and research. With the aim to iterate on updating this design solution with copy & visual changes, based on insights that we will receive from quality and quantity data soon.

Closely collaborated
on payment flow.

With the aim of making clear experience for our subscribers and minimise development effort in MVP, I closely collaborated with payment team to discuss the possible changes, that we need to change in our payment flow. Keeping in mind code legacy & previous payment quality / quantity data and researches.

Made
user interviews for the flow (2-4).

Finished the top of the funnel flow. To be sure, that this flow is easy and clear for users – decided to do user research with the new designed screens - evaluate what users understand / expect / etc. Conducted 10 interviews to have enough amount of feedback, that have certain patterns to implement. With research team, participated as an interviewer and observer.

Key insights:
‍
- 6/10 didn’t understand it’s subscription, not packages.
- 7/10 of the students were not sure that their account would refill in a month.
- 8/10 easily found the entrypoints to subscribe.
- 4/10 didn't understand that they need to scheduled weekly lessons.

Based on these insights, iterated to make top of the funnel flow to be much easier in clarity that it is subscription, how it works and what users will get. These changes positively affected the final result.

Participated in
design critique session.

To even better understand the key problems we may face with the flow after user interviews process, I made the design critiques session. For me is very helpful to collaborate with designers from different teams and see new perspectives on current tasks and problems. To achieve better and more quality feedback, we created the unified structure of our design critiques.

Structure:
‍
I like -
I wish -
What if? -
Takeaways

This type of feedback helped me to see additional gaps in the flow. Iterate on it and made several ideas for future iterations.

Worked on
scheduling the lesson.

On this step, worked closely with User Researchers, Product Managers and Engineers from core product team to build simple user centered digital experiences of booking regular classes. Their knowledge helped me to not guess, but make design solution based on data and insights that we have.

Designed the new flow of regular lessons, with focus on edge cases. Minimised major changes in regards to current production version, so developers can implement it with minimal time effort and no complexity.

and
settings + secondary screens.

We faced the problem, that some edge cases were not defined. How user can reschedule lessons, how canceling subscription will work. How we can get that users back, etc. I focused on finishing it for MVP with secondary screens, pop-ups and the settings page, where users have the ability to cancel their subscription.

Finalised the flow and launched the test.

Finalising the flow consisted for me from several steps:
‍
1. Localisation discussion.
We localise all experiments for 7 languages. With localisaton team discussed and fixed potential bugs, so design will be adaptive and ready for language changes.
‍
2. Design review of development results.
‍Reviewed the final result with development team with the aim to fix bugs or new states that were not taken into account in the design.

3. Approval discussion with the management team.
‍I effectively communicate with C-level stakeholders the final results for MVP and what we plan to do next.

After launch of the test: Started to work on additional functionality, that can give more value for subscribers.

Ability to update subscriptions.

Based on feedback from our users and to decrease the churn rate, I worked on the ability for users to upgrade and downgrade plans. It can reduce the amount of unsubscribers: users will switch to smaller plans and not churn.

Designed the whole flow.

Ability to pause subscription & expired hours.

To motivate people to have a good pace in learning, we made certain amount of hours to expire if student is inactive during the month. We calculated the best possible amount that can expire without pain.

In addition to it, created the pause functionality, so students at holidays or other situatuons can pause their subscription and not churn.

and more. We have launched 8 versions of subscribe to a tutor to achieve our long-term goal of achieving higher retention, LTV & adapting our business model.

Results:
We received a strong signal that it can be a very good business model.

Achieved increase in LTV, with positive projection for long-term goals, GMV & retention uplift. We need to work out on our dynamics and research more, because the dynamics for the first month need to be iterated.

This experiment for me involved close collaboration inside growth, other teams and top-management. I regularly discussed design and technical tradeoffs with developers, and provided my direction and thoughts on design priorities. Collaborated with designers and researchers across Preply to continuously improve design processes and results. Worked very closely providing ideas and suggestions to each other's area of focus. Involved in design process from beginning to end, adapting insights from research and analytics into solutions that form this experiment.

let's do something cool together?

send me an email
or follow me on instagram, telegram & facebook