How would you measure the success of Gmail?

  Google
Add Your Answer
Answers (4)

Clarification

  • Gmail has both consumer users and business users. Should I focus on consumer users here? [Sure]
Product
Gmail is a free email service, where the email address is also the users’ google ID. From Gmail, user can send/receive/search for emails; chat with other Google users using text, voice and video; and use the same Google ID to access other Google services such as Google drive, calendar, photo, map, youtube, Chrome.
Goal
As Google has already reached very high user penetration, the most important goal for Gmail is to keep Google’s user engaged so that they will stay with Google’s app and services ecosystem.
User Journey
  1. Users visit mail.google.com, or alternatively, powered up an Android phone, or downloaded the Gmail App.
  2. Users create a Google ID
  3. Users login into Gmail
  4. Users use various Gmail features; or move on to use other Google services, such as drive, calendar, hangout.
  5. Users generate purchases at services such as drive, youtube, Play, etc.
Metrics
  • Acquisition
    • # PV to mail.google.com
    • # downloads of Gmail
  • Activation
    • # users that creates ID at mail.google.com/Gmail App
    • % Conversion rate from PV to Signup
    • # users who log into Gmail
    • % Conversion rate from signup to login
    • # users who read at least 1 emails in Gmail
    • # user who send at least 1 emails in Gmail
  • Engagement and Retention
    • DAU, MAU, 6-MAU
    • Avg Face time or avg session time per day
    • Avg App Swaps to front screen or avg sessions per day
    • # emails sent/read per session, per day, per month
    • # of Active days in 30 days
    • 7 Day, 30 Day, 180 Day Retention rate
    • # of other Google Services also used
    • % Gmail user who also use other Google Services
    • Average Account ages
    • Churn rate, users who didn’t log into gmail for over 30 days or 60 days.
  • Monetization
    • ARPU
    • LTV
Analyze metrics
To measure Gmail’s success, we can focus on the engagement metrics, such as
1. Avg Face time per month should be stable or increasing over time.
2. # of emails sent/read per month.
3. # of Active days in 30 days.
Metrics 1 is difficult to measure accurately, also it might be slow to detect trend that users still visit, but read/send less and less, which indicates Gmail are losing importance to the user.
Metrics has similar limitations as metrics 1.
Recommendation
I would like to use the # of emails sent/read per month per user as the success indicator.
We can also visualize the value of gmail to Google by breaking users into buckets based on their monthly read/sent mails, and measure their corresponding total Google Apps usage (facetime), # of Google Services used, ARPU, and LTV.

Feature Description : 

Gmail is the free email application used for people to connect via mails.It was launched in 2004.Since then it has made better day by day for users with features like multiple gmail accounts sign in a single tab,organizing emails more conveniently.

Goal

Gmail success goal is to drive more adoption, engagement and retention for the users. Assuming its to have more people on board and retain existing users .

Customer Journey :

  1. User sign up and create a gmail account.
  2. Sends/receives  his first mail.
  3. Repeating step 2 frequently.
  4. Using features like priortising messages , deleting unwanted mails etc.
Metrics to capture :
(Using HEART framework here)
1.Adoption
  • No of new users creating gmail account WOW, MOM.
  • No of users dropping off during sign up process.
2.Task Success
  • No of newly registered users able to send their first mail.
  • No of newly resgistred users able to receive their first mail.
3.Engagement
  • Average no of mails sent by a  user daily and weekly basis.
  • No of times users logs in to check mails in a day (following notifications or on its own)
  • Average no of mails read by a  user daily and weekly basis.
  • Average time spent in a session
4.Retention
  • No of users coming back on daily and weekly basis.
  • No of users who went inactive after a day ,week ,month or 45 days usage.
5.Happiness
  • Average NPS score for the app
  • App ratings /reviews.
Prioritizing :
I would prioritize engagement, retention and adoption metrics .

1. Describe the product: Gmail for customers is a free tool/service developed to communicate online for personal or business reasons. The email address is also the users’ Google ID. However, users are not limited to having just one Gmail ID. Using this Gmail ID, users can also use other suits of products provided by Google like Chrome, Maps, Youtube, Calendar, Drive, Photos, etc.

Some tasks that a user over Gmail can do are:

  1. Send/Receive emails.
  2. Delete/Archive emails.
  3. Snooze or set a Reminder for emails.
  4. Categorize, Organize and Prioritize emails using Tags/Folders.
  5. Integrate with other Google products/services.
  • Send invites for a virtual meetup or meeting via integrating Google Calendar and Google Meet links in the email.
  • Send files(audio/image/video) attachments in the emails via Google Drive.
  • Chat with other Google users using text, voice and video via Google Chat.

2. Assumptions

  1. Gmail has both Personal and Business users; We will focus only on Personal Users (General/Free GSuit Version).
  2. Gmail has messaging chat features; We will focus only on email.
  3. We will focus only on Existing users between Newly Acquired and Existing users.
  4. For simplicity, we will be looking at Indian users and their User journeys on Web and App both (Android & iOS).

 

3. Goal

  1. Acquisition?—?Google has reached good user penetration. Gmail being an entry-level product, may have excellent acquisition metrics.
  2. Revenue?—?As we are looking at the free version of the product, revenue is indirectly through advertising. Also, there is a paid option to get more storage, but that does not apply to large sections of the user base.
  3. Engagement and Retention?—?The most critical goal for Gmail is to keep Google’s users engaged so that they will stay with Google’s app and services ecosystem and reoccurring engagement shows Retention.

Thus, Engagement and Retention are essential for this product.

 

4. User Journey

Desktop

  • Acquisition
  1. Visit mail.google.com and click on “Create account”.
  2. Fill in personal details and set a strong password to sign up.
  3. Send/Receive the first email.
  4. Repeat the previous step multiple times (Loop).
  • Engagement and Retention
  1. Open Gmail on the browser organically.
  2. Send emails.
  3. Read emails from the Primary Inbox.
  4. Read emails from Social, Promotions, Updates, Forums, and other categories.
  5. Reply to emails.
  6. Other actions?—?forward, delete, archive, sort, snooze.
  7. Close tab/browser.

Mobile (Android/iOS)

  • Acquisition
  1. Download the Gmail App from Play Store/App Store.
  2. Sign up and create a Gmail account.
  3. Send/Receive the first email.
  4. Repeat the previous step multiple times (Loop).
  • Engagement and Retention
  1. Open Gmail app Organically or via push notification.
  2. Send emails.
  3. Read emails from the Primary Inbox.
  4. Read emails from Social, Promotions, Updates, Forums, and other categories.
  5. Reply to emails.
  6. Other actions?—?forward, delete, archive, sort, snooze.
  7. Exit app.

 

5. How do we measure? Acquisition

  • Desktop
  1. # of users who visited mail.google.com organically from the browser.
  • Mobile
  1. # of users downloading Gmail app organically from Play Store/App Store.
  2. # of users who opened the app to sign up / Total number of users who have the app pre-installed on the devices from the OS side.

Activation

  • Desktop
  1. # of users that completed signup and created a Google ID on mail.google.com
  2. % Conversion from Page Visits ? Signup.
  3. # of users who read at least one email.
  4. # of users who sent at least one email.
  • Mobile
  1. # of users that completed signup and created a Google ID on the app.
  2. % Conversion from App open ? Signup.
  3. # of users who read at least one email.
  4. # of users who sent at least one email.

Engagement and Retention

  1. # emails sent/read per session, per day, per month.
  2. # of searches in the mailbox per user.
  3. Avg Time Spent/Session Duration per day.
  4. % of users who checked their inbox organically or via notifications on Day 1, Day 7, Day 15, Day 30. 
  5. # of days the user was active in 30 days.
  6. # of users who read emails from Primary VS other categories (Social, Promotions, Updates, Forums, etc.)
  7. % Split of total users who read emails from Primary and from different categories (Social, Promotions, Updates, Forums, etc.)
  8. # of users who forward, delete, archive, sort, snooze their mails.
  9. % Split of the total users who forward, delete, archive, sort, snooze their mails.
  10. % of users who used other Google Services integrated inside Gmail.
  11. # of other Google Services also used.

Retention

  1. 1 Day, 7 Day, 15 Day, 30 Day Retention rate.
  2. DAU, MAU.

Monetization

  1. Amount spent in purchasing additional storage.

Referral

  1. # invites sent.
  2. % of signups from referral codes.

 

6. Evaluating and Prioritising the metrics:

To measure success, we can focus on the Retention and engagement metrics. Let’s evaluate the metrics based on impact, reach, relevance to goal and measurable metrics.

North Star: 

Retention Metrics.

Primary:

  1. # emails sent/read per session, per day, per month.
  2. % of users who checked their inbox organically or via notifications on Day 1, Day 7, Day 15, Day 30.
  3. Avg Time Spent/Session Duration per day.

Secondary:

  1. # of users who read emails from Primary VS other categories (Social, Promotions, Updates, Forums, etc.)
  2. # of users who forward, delete, archive, sort, snooze their mails.
  3. # of days the user was active in 30 days.

 

7. Summary

We looked at measuring the success of Gmail (Desktop & mobile app) for users in India. In the process, we looked at the major sections in the user journey to understand that Retention and engagement metrics will eventually have a higher impact. Thus we narrowed down the North Start, Primary and Secondary metrics, which we will help track the success of Gmail.

I’d take these steps to answer this Metrics question:

1. Describe Gmail product. The main function of gmail is to 1) send and 2) receive emails, with some additional features to support these 2 base functions as 1) delete email, 2) archive email, 3) set snooze/reminder of an email, 4) category emails into tags/folders, 5) integration with other G products (calendar, drive, etc.)

2. Product proposition. develop a tool for people to communicate (personal and business) via Internet.

3. user group: both business and personal consumer users

4. define success: users use gmail as a tool to communicate 1) they create account, 2) compose and send emails, 3) use email as their contact and receive emails, 4) they use it very often

5. user journey:

1) register account/sign up

2) send out first email

3) receive first email

4) send/receive more emails

5) use it everyday?evey couple of days of a week? couple days of months

6. Metrics:

1) # of sign ups in a week/month/year (depends how long the product is launched

2) # of emails sent/received in first week/month/year (if just launched)

3) # of users send out at least 1 email per week (WAU)

4) # of users receive at least 1 email per week (WAU)

5) opt out rate – how many active users stopped use it after how long (in 1 month, 3 month, 6 month, 12 months)

6) total space used per user account

7) could consider other side features as delete, category, or other G products widget but not main measurement here