Reorganization of Firefox-UI tests in mozilla-central
gijskruitbosch at gmail.com
Fri Sep 2 22:18:26 UTC 2016
None of the test frameworks really cleanly map onto what you understand
as "unit", "functional", "integration" or "end to end". The frameworks
are based around capabilities and languages much more than "what kind of
test would I want to write with this". Those capabilities make them
maybe more suitable for particular types of tests, but ultimately it's
the capabilities that determine what kind of test gets written, not
whether it's a unit or an end to end test.
For instance, this is a marionette test:
This restarts Firefox with some magical commandline flags / env vars and
checks that the "Firefox refresh" feature behaves more or less sanely.
Likewise, browser mochitests get used for testing units or spec-level
things depending on the privilege required to easily deduce that the
spec-level thing has actually happened. For instance, here's a browser
test that checks which URLs can link to one another based on security
which is pretty much a unit test of scriptSecurityManager, but is a
mochitest-browser test because creating system- as well as
content-privileged blob URIs and seeing how they interact with a
particular other URI isn't easy in any of our other test frameworks.
IOW, I don't think renaming mochitest (and in any case, which one?) to
"functional" doesn't really help clarify things. We also already kind of
reached the conclusion that "integration" is a bad name for the fx-ui
tests, for related reasons.
On 02/09/2016 23:06, Stuart Philp wrote:
> What about renaming mochitest instead, to "functional" or similar? The
> UI tests that marionette does is in interaction with the UI from a
> "users" perspective. My understanding of mochitest is it is the
> interaction with the browser chrome via the api's. Correct?
Not really. mochitest-plain is basically just a web page with some fancy
semi-privileged APIs. mochitest-chrome is basically just a XUL page with
some fancy semi-privileged APIs. Both of them use inline or sometimes
externally-sourced JS to do testing.
mochitest-browser-chrome instead is a JS script that gets run in the
context of the privileged browser.xul window (ie a normal Firefox
window) and also uses JS to interact with stuff.
The distinction between marionette or mochitest (of any flavour) calling
element.click() (or doing so via helpers like synthesizeMouse and
whatever marionette calls its equivalent of that) seems pretty arbitrary
*** yes, yes, there are technical differences as to whether your event
will have the isTrusted flag set to true, but that's mostly not relevant
for this discussion.
More information about the firefox-dev