- From: Robin Berjon <robin@berjon.com>
- Date: Mon, 4 Jun 2012 18:35:35 +0200
- To: James Graham <jgraham@opera.com>
- Cc: public-test-infra@w3.org
On Jun 4, 2012, at 18:14 , James Graham wrote:
> On 06/04/2012 05:46 PM, Robin Berjon wrote:
>> This *is* metadata :) Storing the information in the test means that
>> when you fix the bug, you just remove the link and any system
>> processing the test can immediately see that it is flagged as
>> bug-free. If you separate the metadata from what it applies to, it is
>> likely to go out of sync and we'll end up with tests that have been
>> fixed but not flagged as such in the system.
>
> Yes, but it can *also* mean that you break the test, if you aren't very careful. Consider a test that does document.getElementsByTagName("link")[0], for example.
While that is indeed within the realm of possible things, I would like to point out that it is a somewhat contrived example, and easy to fix at that. You would be breaking a broken test. The set of situations in which it could happen is very restricted. Overall, given the trade-off in breakage risks, it would seem to me to be a rather small risk compared to that of not having up-to-date metadata!
> I wouldn't trust an automated tool to update the test files themselves, but I would trust them to update a separate metadata file. This would allow integration with bugzilla, for example, so that the test could be flagged as "fixed" as soon as the corresponding bug was closed (and flagged as broken as soon as a bug was opened/confirmed, without the person opening the bug having to take any separate steps).
Sure, that's possible, but it requires more integration rather than less.
--
Robin Berjon - http://berjon.com/ - @robinberjon
Received on Monday, 4 June 2012 16:48:19 UTC