"outage" and future plans

Gregory Szorc gps at
Thu Aug 24 19:25:32 UTC 2017

On Thu, Aug 24, 2017 at 12:18 PM, Kris Maglione <kmaglione at>

> Related:
> The build environment on RTD has been one of the main things preventing us
> from getting JSDoc comments into our Sphinx docs. Now that we have our own
> hosting, will it be possible to add something like sphinx-js to the build
> process?
> [1]:

Yes. That's sphinx-js is tracked by bug 1389341. And, yes, now that we are
in full control over the execution environment, we can have nice things. We
could potentially upload Doxygen, Javadoc, etc to this site as well. Maybe
even a static version of DXR or Searchfox (if such a thing is possible).
File Core :: Build Config bugs. I can help with technical guidance and

> On Thu, Aug 24, 2017 at 12:01:39PM -0700, Gregory Szorc wrote:
>> is now live.
>> Please update your links.
>> Send thanks to Dustin Mitchell and WebOps for all the behind-the-scenes
>> operational work.
>> On Tue, Aug 22, 2017 at 3:52 PM, Gregory Szorc <gps at> wrote:
>> tl;dr broke hard a few hours ago (but the
>>> docs are still available at a temporary URL) and we have plans to move
>>> away
>>> from Read the Docs due to ongoing reliability and control concerns.
>>> The Firefox source repository has in-tree Sphinx docs. We've been hosting
>>> those on for a few years.
>>> In recent months, the reliability of Read The Docs (RTD) for generating
>>> the Firefox docs has deteriorated slowly. ~50% of docs "builds" fail for
>>> "intermittent" reasons (mostly related to the scale of our docs AFAICT).
>>> A few hours ago, failure to generate the docs has somehow resulted in RTD
>>> showing a generic template page and the old docs are no longer visible.
>>> And
>>> instead of a useful and potentially actionable build log, it shows "An
>>> unexpected error occurred."
>>> Because of the deterioration of RTD's reliability, the unplanned fire
>>> drills that changes to RTD inflict upon us, and the inability to fully
>>> control the docs generation environment to do advanced things, we've been
>>> wanting to move away from for several months. (For
>>> the record, I have engaged RTD about reliability issues in the past. It
>>> is
>>> clear Firefox's docs are a 0.1% use case in terms of scale and therefore
>>> difficult to support.)
>>> We now have TaskCluster automation uploading the generated docs whenever
>>> they are changed on mozilla-central (bug 1382729). This is covered by
>>> Firefox's automation "SLA" which means people notice quickly when it
>>> breaks
>>> (unlike RTD). The next step is bug 1392765 to get a new hostname and x509
>>> certificate so there is a more official-looking home. From there, we'll
>>> likely do some things around hosting different versions corresponding to
>>> Firefox releases (bug 1390693) and auto-generated documentation for
>>> JavaScript files courtesy of sphinx-js (bug 1389341). And, yes, we'll
>>> likely try to redirect to the new home so old
>>> links
>>> still work.
>>> I'll send out an announcement when we have a new formal home for the
>>> in-tree docs (hopefully in a few days). In the mean time, if you must
>>> access the docs that were formerly available at
>>>, you can find them at
>>> That's a
>>> temporary home, so please don't link to it from anything official.
>>> I apologize for any inconvenience this has caused. The service
>>> reliability
>>> problems should go away once we complete the transition from RTD to
>>> TaskCluster.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the firefox-dev mailing list