Re: Who currently executes the tests in the w3c repos?

On Wed, Aug 7, 2013 at 2:22 AM, Ms2ger <ms2ger@gmail.com> wrote:

> On 08/07/2013 02:43 AM, Dirk Pranke wrote:
>
>> Hi all,
>>
>> I am a relative newcomer to this group but I have been working off and on
>> recently (quite a bit just now) getting the tests running as part of the
>> automated tests for Blink and WebKit.
>>
>> I believe I'm probably missing quite a bit of context or history that
>> makes
>> it difficult for me to understand some of the design decisions and
>> processes around getting tests written and submitted and run.
>>
>> So forgive me if this sounds like a brash question, but it's honestly one
>> coming from ignorance and not meant to be snarky ...:
>>
>> Who actually currently runs these tests, and how?
>>
>> As far as I know, no one in Blink (or WebKit) regularly runs any of these
>> tests, even manually, with a few exceptions where we have manually
>> imported
>> some suites into our existing repos. It may also be the case that some
>> times individual developers or spec editors have run some of the tests.
>>
>>> From my limited conversations w/ Fantasai, I believe the situation is
>>>
>> similar for Mozilla. I do not know about efforts inside Microsoft or at
>> Opera, or at any other browser vendor or third party.
>>
>> Are there groups that actually do attempt to run the tests somehow on the
>> different browsers? Does that somehow happen in Shepherd in a way I don't
>> know about (or understand)?
>>
>> I would like to be able to usefully contribute to threads like
>> "consolidating css-wg and web-platform-tests repositories" and talk about
>> the pain points I'm hitting as I try to get the tests running, but it's
>> hard for me to say useful things w/o knowing more about how others are
>> using all of this. So, I'm looking to become educated?
>>
>
> Mozilla imports a subset of the web-platform-tests repository into its
> main repository (based on the MANIFEST files you might have noticed). The
> importing is completely scripted, including writing the annotations for
> failing tests. The code for that is available [1]. Those tests are run both
> in automation and by developers locally from there.
>

That's very good to know; I will take a look at the Mozilla links you've
provided.


> I understood Ryosuke Niwa was working on importing tests in WebKit before
> the fork; have you talked to him?
>

Yes, I am very familiar with the work Ryosuke did.

Thanks!

-- Dirk