Intent to deprecate - linux32 tests starting with Firefox 69
jmaher at mozilla.com
jmaher at mozilla.com
Fri Apr 5 13:35:15 UTC 2019
Currently linux32 makes up about .77% of our total users. This is still 1M+ users on any given week.
There is not a lot of support in the industry for 32 bit linux, all the major vendors are only distributing 64 bit versions now and other browser vendors only distribute 64 bit versions officially.
Currently we run a full suite of unit tests on linux32 per commit (integration, mozilla-central, try, mozilla-beta) and overall this is about 11% of our total CPU usage (which is about $416/day).
Linux32 has a similar number of intermittent failures (2.06% of tasks fail) and regressions as other platforms (such as Android, Windows, OSX, Linux64) indicating that this functions as an equal in many ways. The cost associated with intermittents and failures will be a savings for everyone looking at results of a push by reducing 10% of intermittents allowing focus other failures for faster decision making.
I have looked at regressions that only appear on linux32, in fact looking deeper into the changes, there are 3 regressions since July which caused a backout and fix to the product (bug 571074, bug 1497580, bug 1499426). For reference, in 2019, osx, windows7, and android 4.3 all have ~5 unique regressions found and all other configs have 0 or 1. The larger volume of regressions we find are seen on multiple platforms or didn't result in fixing the browser.
Linux32 runs many tests in both e10s and non-e10s (see https://bugzilla.mozilla.org/show_bug.cgi?id=1433276 ), for a few reasons:
1) fennec ships non-e10s, our code base should have more coverage
2) not all tests run on fennec
3) users can disable non-e10s locally
I looked at Firefox non-e10s users, and it is 0.2% of our users for anyone that has upgraded the browser in the last 6 months. I focused on that because if we are supporting modern versions and users never see it, then our efforts to build/develop non-e10s are having little effect. In fact 96.4% of our users that use non-e10s are on Windows, so if we determine there is a need for testing this, I would encourage us to test on windows instead of Linux. As for running non-e10s tests in place of coverage on Android, we have enabled more tests on android emulators (primarily due to web-platform-tests) which has reduced the need to rely on Linux32 as a means for getting close enough coverage for Android.
Earlier I mentioned 3 regressions in the browser that we fixed, 2 of those were for non-e10s specific cases, and one was for our default browser in e10s mode. While there have been many other regressions found on linux32, they are either fixed by hacking tests or disabling the test, or we see them on other platforms and would catch them in the absence of linux32 tests.
As our next ESR is upcoming, I would like to turn off linux32 on Firefox 69 and let it ride the trains and stay on 68 ESR. This will allow builds/tests to be supported with security updates into 2021.
More information about the dev-platform