In order to facilitate community work
bruant at enseirb-matmeca.fr
Mon Feb 28 05:26:29 PST 2011
Here are a couple of suggestions which, in my opinion, would help the
-- In the Results page, it would be useful to have a link to the test
file in the http://hg.ecmascript.org/tests/test262/ website (may help to
spot differences between the run test suite and the latest )
-- A "report a bug" link next to each bug result (or in the "source"
popup?)would be useful too. If there is such a feature in Bugzilla, the
link could carry a preformatted bug title and pre-fill the bug report to
point to the "Tests" component. Uniform bug titles make text-search easier.
- Test searching
There are now about 10000 tests. By default, they are sorted in the repo
by source(Sputnik | ietestcenter)/chapter/section/subsection/...
Even if it certainly makes sense from a test suite producer point of
view (and directory choices has to be made anyway), it doesn't cover all
search use cases. In order to "identify test holes" more efficiently as
suggested under "Community Contributions" it would be helpful to have
some tool to search for tests. For instance, it's currently hard to find
if a test has been forgotten regarding:
* native prototype objects
For instance, in 184.108.40.206 is written "The [[Prototype]] internal
property of the newly constructed object is set to the original Array
prototype object, the one that is the initial value of Array.prototype".
There is clearly a test to write based on that to make sure that
Monkey-patched Array.prototype aren't used. Such a pattern can be found
for functions (15.3.3, 15.3.4), objects (220.127.116.11 step 4) and certainly
in other places.
* length properties
* Errors (make sure that code that should throw errors have a test for them)
* strict mode (I assume it's not going to be covered in a chapter)
* Any other cross-chapter topic you could think of.
Several different approaches could be taken to tackle that issue:
-- tags search
- Run partial test suite
Running the test suite is currently long. I'm currently reviewing FF4
fails. When I close my web browser, I have to re-run all tests. However,
most of the time, I don't care about most of them. I just want all those
that I haven't reviewed yet. I wish the website could offer an easy way
to run for instance:
- all chapter 15.2 tests (or any chapter/section/subsection obviously)
- all tests found through a search (if/when some search feature will be
- all tests that has been added or changed between two test suite
versions. If I have reviewed all tests of version x, I may not care
about them when the x+1 version comes out.
- Accept user contributed tests
I have read that "Ecma's intellectual property policies, permit only
Ecma members to directly contribute code to the project.". I however
think that this test suite would benefit from accepting contributions
the most used programming language. I think there is a need to open up
the way tests can be contributed.
If the Ecma's intellectual property policies cannot be changed/adapted
to make that happen, could ECMA TC-39 members find a way to accept user
contributions, review them and then submit them to Ecma? For instance,
by creating a TC-39 github account and a test repo, people create tests,
submit them to this repo and when there are enough of them or after a
certain amount of time, TC-39 members officially submit the tests to
ECMA. There are certainly intellectual property/copyright issues that
has to be considered, by I'm drawing some sort of main idea.
I'd be happy to help out if there is anything I can do to help with
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the test262-discuss