Supporting multiple ECMAScript test suites

Bill Ticehurst billti at microsoft.com
Mon Aug 20 23:27:17 PDT 2012


Is there an expected status of the specification, quality/coverage level of the test-suite, or level of browser implementation we expect before promoting test suites to the front page of the website?

The "best practices" suite only has 20 test-cases, with most of these being very narrow in scope (13 of them related to blocks where you can/can't declare a function).  Unless this is going to receive some investment, I'm not sure there is value in promoting it to prominence as a separate test suite on the Welcome page.

Likewise for ES6.  While we've been iterating on open issues rapidly recently, there have been no submits to the Harmony test-case repository for 9 months, and then only tests for one feature (proxies).  Do you see value in promoting this suite to the Welcome page before the spec stabilizes and/or there are more contributions?

Obviously Intl402 is a different story, as we are close to final draft and you have a number of tests with broad coverage waiting in the pipeline.  Based on your write-up below, I'm not clear how (or if) we would distinguish between test suites for finalized standards (e.g. ES5.1), and standards yet to be finalized (e.g. Intl402).  I assume we'd want to make this distinction clear?

Regards,

 - Bill


-----Original Message-----
From: test262-discuss-bounces at mozilla.org [mailto:test262-discuss-bounces at mozilla.org] On Behalf Of Norbert Lindenberg
Sent: Monday, August 20, 2012 11:58 AM
To: Rick Waldron; test262-discuss
Subject: Re: Supporting multiple ECMAScript test suites

Any other comments on this proposal, or shall we go ahead?

Rick, currently there's quite a bit of information on contributing on the pages:
- A section "Who creates and maintains test262?" on the test262 page http://test262.ecmascript.org
- The "Development" pane of that page
- Sections "Test Submissions" and "Community Contributions" on the test262 wiki:
http://wiki.ecmascript.org/doku.php?id=test262:test262

We'd certainly maintain this information in a restructuring of the web site. It probably could be improved, maybe in conjunction with Dave's work on making the wiki more community-friendly. Would you like to try?

Norbert


On Aug 13, 2012, at 10:32 , Rick Waldron wrote:

> 
> 
> On Monday, August 13, 2012 at 1:05 PM, Norbert Lindenberg wrote:
> 
>> test262 isn't a single test suite anymore: Right now there are three different test suites in the repository - ECMA-262 conformance, ECMA-262 best practices, and ECMA-402 conformance. Soon we may want to fork off an ES6 test suite as well.
>> 
>> 
>> What should the web site look like that provides access to three or 
>> more different test suites? Right now, you see 
>> http://test262.ecmascript.org and you don't see 
>> http://test262.ecmascript.org/testcases_intl402.html
>> http://test262.ecmascript.org/testcases_bestPractice.html
>> 
>> I propose we rename the site to
>> http://tests.ecmascript.org
>> and then have
>> http://tests.ecmascript.org/
>> as a welcome page with links to:
>> http://tests.ecmascript.org/ecma-262/
>> http://tests.ecmascript.org/ecma-262-bp/
>> http://tests.ecmascript.org/ecma-402/
>> Later we'll add
>> http://tests.ecmascript.org/ecma-262-6ed/
>> which will eventually become the new
>> http://tests.ecmascript.org/ecma-262/
>> 
>> 
>> What should the corresponding bug database buckets look like? Right 
>> now, there's a
>> Test262 | Test suite for Ecma-262 implementations product with 
>> components Test Harness | The infrastructure for running test262 
>> tests
>> Test262 website | The test262.ecmascript.org website Tests | The 
>> individual tests that make up the test262 suite
>> 
>> I propose we change this to
>> Test Suites | Test suites for ECMAScript implementations Test Harness 
>> | The infrastructure for running tests, including the 
>> tests.ecmascript.org web site
>> ECMA-262 Tests | The individual tests for ECMA-262 (Language 
>> Specification) conformance
>> ECMA-402 Tests | The individual tests for ECMA-402 
>> (Internationalization API) conformance
>> 
>> 
>> Comments?
> My only comment is that of support for this re-org.
> 
> One thing to keep in mind is making sure complete and up to date instructions for contribution are immediately available on each of the /test-*** pages.
> 
> Rick
> 
>  
>> 
>> Thanks,
>> Norbert
>> 

_______________________________________________
test262-discuss mailing list
test262-discuss at mozilla.org
https://mail.mozilla.org/listinfo/test262-discuss







More information about the test262-discuss mailing list