"outage" and future plans

Gregory Szorc gps at
Thu Aug 24 19:01:39 UTC 2017 is now live.

Please update your links.

Send thanks to Dustin Mitchell and WebOps for all the behind-the-scenes
operational work.

On Tue, Aug 22, 2017 at 3:52 PM, Gregory Szorc <gps at> wrote:

> tl;dr broke hard a few hours ago (but the
> docs are still available at a temporary URL) and we have plans to move away
> from Read the Docs due to ongoing reliability and control concerns.
> The Firefox source repository has in-tree Sphinx docs. We've been hosting
> those on for a few years.
> In recent months, the reliability of Read The Docs (RTD) for generating
> the Firefox docs has deteriorated slowly. ~50% of docs "builds" fail for
> "intermittent" reasons (mostly related to the scale of our docs AFAICT).
> A few hours ago, failure to generate the docs has somehow resulted in RTD
> showing a generic template page and the old docs are no longer visible. And
> instead of a useful and potentially actionable build log, it shows "An
> unexpected error occurred."
> Because of the deterioration of RTD's reliability, the unplanned fire
> drills that changes to RTD inflict upon us, and the inability to fully
> control the docs generation environment to do advanced things, we've been
> wanting to move away from for several months. (For
> the record, I have engaged RTD about reliability issues in the past. It is
> clear Firefox's docs are a 0.1% use case in terms of scale and therefore
> difficult to support.)
> We now have TaskCluster automation uploading the generated docs whenever
> they are changed on mozilla-central (bug 1382729). This is covered by
> Firefox's automation "SLA" which means people notice quickly when it breaks
> (unlike RTD). The next step is bug 1392765 to get a new hostname and x509
> certificate so there is a more official-looking home. From there, we'll
> likely do some things around hosting different versions corresponding to
> Firefox releases (bug 1390693) and auto-generated documentation for
> JavaScript files courtesy of sphinx-js (bug 1389341). And, yes, we'll
> likely try to redirect to the new home so old links
> still work.
> I'll send out an announcement when we have a new formal home for the
> in-tree docs (hopefully in a few days). In the mean time, if you must
> access the docs that were formerly available at
>, you can find them at
> That's a
> temporary home, so please don't link to it from anything official.
> I apologize for any inconvenience this has caused. The service reliability
> problems should go away once we complete the transition from RTD to
> TaskCluster.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the firefox-dev mailing list