Activity Stream - Post Launch Engineering Update

Jared Wein jaws at mozilla.com
Tue Dec 5 21:18:23 UTC 2017


Wow, that video shows great improvement! Nice job!

As to the engagement gains, what numbers are you comparing? For example,
"Highlights (from 1.6% to 2.6%, a +62% gain)". The charts start on November
12 and Quantum was released on November 14th. Are these engagement gains
seen between v57 and v58 users?

Thanks,
Jared

On Tue, Dec 5, 2017 at 3:52 PM, Tim Spurway <tspurway at mozilla.com> wrote:

> Hey Folks,
>
> Activity Stream was released as the default about:newtab and about:home experience
> for Firefox Quantum just about three weeks ago.  The team is proud to
> report that Activity Stream is performing well in release!
>
> Here’s a cool graph from the Activity Stream main dashboard
> <https://sql.telemetry.mozilla.org/dashboard/activity-stream-system-addon-metrics-summary_1?p_channel=release&p_granularity=week>
> :
>
> [image: Inline image 1]
>
> This is the engagement rates of the various Activity Stream
> sections/components for both about:home and about:newtab.  Note that
> Activity Stream is an almost identical experience on both pages.  The first
> thing you notice about this data is that people are using the Home and
> Newtab pages very differently  -  more people interact with Activity Stream
> on about:home (49%) than on about:newtab (34%).
>
> The engagement gains on about:home appear to be spread across the board:
>  Top Sites (from 20% to 28%, a +40% gain), Search (from 11% to 17%, a +37%
> gain), Highlights (from 1.6% to 2.6%, a +62% gain) and Pocket (from 0.13%
> to 0.34%, a 161% gain).  The explanation for this difference could be that
> people use the Newtab page mostly to go somewhere they already have in
> mind, whereas they use the Home page to discover new things or to go to
> places they have been to before.
>
> This is just one of the insights we have gained since launch.  We are
> using results like this to help improve the browsing experience for our
> users.
>
> *Performance / Perceived Performance*
>
> In the lead up to Firefox Quantum, we worked closely with the Quantum Flow
> team to understand what role Activity Stream played in core performance
> targets <https://health.graphics/quantum>.  We were eager to make sure we
> weren’t regressing any existing metrics and focused hard on the specific
> case of improving/optimizing browser startup
> <https://github.com/mozilla/activity-stream/issues?utf8=%E2%9C%93&q=label:Startup+>.
>
>
> In the 58 cycle, we have had a lot of conversations with Dominik and
> Florian about the study of the ‘perceived’ performance of systems like
> Activity Stream.  We identified the key events in the starting of Activity
> Stream and set to work improving the important browser start-up metric.  We
> found video A/B comparison tools to be invaluable for this work.
>
> The link below is a video comparing Browser Startup of Firefox 57 and 58.
> Note that to focus on just looking at Activity Stream, we do not include
> the entire startup sequence of the browser.  We have synchronized the two
> videos to start at the ‘first paint’ event of the overall Firefox startup
> sequence.
>
> [image: Video Browser Startup Firefox 57 vs. 58]
>
> <https://drive.google.com/a/mozilla.com/file/d/1-ZyNfr9ib82s_1gMLUOYFCe6WSmU-ot8/view?usp=sharing>
>
>
>
> That's pretty cool.  Notice how we 'get something on screen' early, which
> is a big perceived win, but also crushes the overall perf numbers from the
> previous version.
>
> We are going to keep this work up.  Our #1 priority over the next few
> releases is improved performance and stability.
>
> cheers,
> tim
>
> _______________________________________________
> firefox-dev mailing list
> firefox-dev at mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/firefox-dev/attachments/20171205/5a94bd79/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 431827 bytes
Desc: not available
URL: <http://mail.mozilla.org/pipermail/firefox-dev/attachments/20171205/5a94bd79/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 162887 bytes
Desc: not available
URL: <http://mail.mozilla.org/pipermail/firefox-dev/attachments/20171205/5a94bd79/attachment-0003.png>


More information about the firefox-dev mailing list