gecko.readthedocs.org "outage" and future plans

Gregory Szorc gps at mozilla.com
Thu Aug 24 19:01:39 UTC 2017


https://firefox-source-docs.mozilla.org/ is now live.

Please update your links.

Send thanks to Dustin Mitchell and WebOps for all the behind-the-scenes
operational work.

On Tue, Aug 22, 2017 at 3:52 PM, Gregory Szorc <gps at mozilla.com> wrote:

> tl;dr https://gecko.readthedocs.org broke hard a few hours ago (but the
> docs are still available at a temporary URL) and we have plans to move away
> from Read the Docs due to ongoing reliability and control concerns.
>
> The Firefox source repository has in-tree Sphinx docs. We've been hosting
> those on https://gecko.readthedocs.org/ for a few years.
>
> In recent months, the reliability of Read The Docs (RTD) for generating
> the Firefox docs has deteriorated slowly. ~50% of docs "builds" fail for
> "intermittent" reasons (mostly related to the scale of our docs AFAICT).
>
> A few hours ago, failure to generate the docs has somehow resulted in RTD
> showing a generic template page and the old docs are no longer visible. And
> instead of a useful and potentially actionable build log, it shows "An
> unexpected error occurred."
>
> Because of the deterioration of RTD's reliability, the unplanned fire
> drills that changes to RTD inflict upon us, and the inability to fully
> control the docs generation environment to do advanced things, we've been
> wanting to move away from gecko.readthedocs.org for several months. (For
> the record, I have engaged RTD about reliability issues in the past. It is
> clear Firefox's docs are a 0.1% use case in terms of scale and therefore
> difficult to support.)
>
> We now have TaskCluster automation uploading the generated docs whenever
> they are changed on mozilla-central (bug 1382729). This is covered by
> Firefox's automation "SLA" which means people notice quickly when it breaks
> (unlike RTD). The next step is bug 1392765 to get a new hostname and x509
> certificate so there is a more official-looking home. From there, we'll
> likely do some things around hosting different versions corresponding to
> Firefox releases (bug 1390693) and auto-generated documentation for
> JavaScript files courtesy of sphinx-js (bug 1389341). And, yes, we'll
> likely try to redirect gecko.readthedocs.org to the new home so old links
> still work.
>
> I'll send out an announcement when we have a new formal home for the
> in-tree docs (hopefully in a few days). In the mean time, if you must
> access the docs that were formerly available at
> https://gecko.readthedocs.org/, you can find them at
> http://gecko-docs.mozilla.org.s3.amazonaws.com/index.html. That's a
> temporary home, so please don't link to it from anything official.
>
> I apologize for any inconvenience this has caused. The service reliability
> problems should go away once we complete the transition from RTD to
> TaskCluster.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/firefox-dev/attachments/20170824/478ad167/attachment.html>


More information about the firefox-dev mailing list