- From: Dirk Pranke <dpranke@chromium.org>
- Date: Fri, 23 Aug 2013 10:50:28 -0700
- To: James Graham <james@hoppipolla.co.uk>
- Cc: "public-test-infra@w3.org" <public-test-infra@w3.org>
- Message-ID: <CAEoffTALEGYwcXaLmDUtQ2EEWtYWo8Wq=ni3w3tD3GQZ6c1Dzg@mail.gmail.com>
On Fri, Aug 23, 2013 at 6:12 AM, James Graham <james@hoppipolla.co.uk>wrote: > On 23/08/13 13:43, Tobie Langel wrote: > >> On Friday, August 23, 2013 at 12:58 PM, James Graham wrote: >> >>> I feel like I didn't explain my previous proposal very well. >>> >>> Automation harnesses need a manifest or some other external way of >>> storing which tests exist and what settings they have. That is, I >>> think, uncontroversial. It is simply not viable to parse that >>> information out of the test files when they are actually being >>> run. >>> >> >> Agreed, but this feels somewhat of an implementation detail of the >> test runner. i.e. I'm not sure what the benefit is of standardizing >> this. >> > > The benefit is that it's clear everyone agrees on which tests are in the > repo and what properties they have. If we have N different implementations > that could lead to N different conclusions about what tests we have and > what their properties are. > > Of course people are free to use their own implementation, as long as we > have some way of agreeing what the correct output is. Therefore I will > implement the proposal, check in the result, and let people either use it > directly, or check their own tools against it. > I'm inclined to agree that if we thought that every test harness needed a manifest of some form, we should probably standardize on the format. Obviously, I'm not yet convinced that we do need a manifest as part of the files in the repo, though :). -- Dirk
Received on Friday, 23 August 2013 17:51:15 UTC