Re: Weekly Update

My expectation is that not all specs have a test suite that proves it
covers all lines of the spec, so therefore this would be a good report
to have when going from Candidate Recommendation (CR) to Proposed
Recommendation (PR), see
https://www.w3.org/Consortium/Process/Process-19991111/tr.html#RecsPR.

Also, if continued development of the spec could cause an update to
that report file, so it would basically alert when spec coverage
decreases, that would be a useful tool to have.

Just my 2c worth.

Cheers,
Silvia.



On Mon, Mar 13, 2017 at 8:17 PM, Benjamin Schaaf <ben.schaaf@gmail.com> wrote:
> Hello Silvia,
>
> That tool simply does normal test coverage on a 1:1 translation of the
> file-parsing section of the spec to python.
> The purpose of that tool is to ensure every part of the file-parsing
> section of the spec is touched by the tests,
> which after the pull request was merged it was.
>
> The file-parsing tests should always have 100% coverage, so I don't
> think there's much of a point of hosting the report file.
>
> Thanks,
> Benjamin
>
> On Mon, Mar 13, 2017 at 8:04 PM, Silvia Pfeiffer
> <silviapfeiffer1@gmail.com> wrote:
>> Hi Ben,
>>
>> Thanks for these updates - I think they're really useful to keep us informed.
>>
>> You mentioned a tool that can show test coverage at some point also.
>> Is the output of that tool available somewhere online?
>> I think it would be a useful piece of information to point towards for
>> moving the spec forward.
>>
>> Thanks,
>> Silvia.
>>
>>
>>
>> On Mon, Mar 13, 2017 at 7:56 PM, Benjamin Schaaf <ben.schaaf@gmail.com> wrote:
>>> Hello,
>>>
>>> This is the 5th weekly update for my progress on the webvtt tests.
>>> My next semester of university has started, so I'm transitioning from
>>> full time to part time work.
>>> This means progress should slow down slightly, but I'm hoping to make
>>> up for it with a now deeper understanding of the codebase.
>>> https://github.com/w3c/web-platform-tests/pull/5032 has been merged,
>>> meaning file parsing tests are complete
>>> I've created a pull request for updating the included vtt file parser
>>> in the tests for a incoming spec change:
>>> https://github.com/w3c/web-platform-tests/pull/5113
>>> I'm also close to completing a tool that automatically categorizes
>>> test results from a json result dump, along with categorizing all the
>>> current tests.
>>> I expect to be done with that tool by the end of the week.
>>>
>>> Thanks,
>>> Benjamin Schaaf

Received on Monday, 13 March 2017 09:31:09 UTC