Re: Weekly Update

The python parser isn't generated from the spec, it has to be updated manually.
So when the spec changes, the parser has to be updated and the
`spec_report.py` script run to see any changes.

Thanks,
Benjamin

On Mon, Mar 13, 2017 at 8:30 PM, Silvia Pfeiffer
<silviapfeiffer1@gmail.com> wrote:
> My expectation is that not all specs have a test suite that proves it
> covers all lines of the spec, so therefore this would be a good report
> to have when going from Candidate Recommendation (CR) to Proposed
> Recommendation (PR), see
> https://www.w3.org/Consortium/Process/Process-19991111/tr.html#RecsPR.
>
> Also, if continued development of the spec could cause an update to
> that report file, so it would basically alert when spec coverage
> decreases, that would be a useful tool to have.
>
> Just my 2c worth.
>
> Cheers,
> Silvia.
>
>
>
> On Mon, Mar 13, 2017 at 8:17 PM, Benjamin Schaaf <ben.schaaf@gmail.com> wrote:
>> Hello Silvia,
>>
>> That tool simply does normal test coverage on a 1:1 translation of the
>> file-parsing section of the spec to python.
>> The purpose of that tool is to ensure every part of the file-parsing
>> section of the spec is touched by the tests,
>> which after the pull request was merged it was.
>>
>> The file-parsing tests should always have 100% coverage, so I don't
>> think there's much of a point of hosting the report file.
>>
>> Thanks,
>> Benjamin
>>
>> On Mon, Mar 13, 2017 at 8:04 PM, Silvia Pfeiffer
>> <silviapfeiffer1@gmail.com> wrote:
>>> Hi Ben,
>>>
>>> Thanks for these updates - I think they're really useful to keep us informed.
>>>
>>> You mentioned a tool that can show test coverage at some point also.
>>> Is the output of that tool available somewhere online?
>>> I think it would be a useful piece of information to point towards for
>>> moving the spec forward.
>>>
>>> Thanks,
>>> Silvia.
>>>
>>>
>>>
>>> On Mon, Mar 13, 2017 at 7:56 PM, Benjamin Schaaf <ben.schaaf@gmail.com> wrote:
>>>> Hello,
>>>>
>>>> This is the 5th weekly update for my progress on the webvtt tests.
>>>> My next semester of university has started, so I'm transitioning from
>>>> full time to part time work.
>>>> This means progress should slow down slightly, but I'm hoping to make
>>>> up for it with a now deeper understanding of the codebase.
>>>> https://github.com/w3c/web-platform-tests/pull/5032 has been merged,
>>>> meaning file parsing tests are complete
>>>> I've created a pull request for updating the included vtt file parser
>>>> in the tests for a incoming spec change:
>>>> https://github.com/w3c/web-platform-tests/pull/5113
>>>> I'm also close to completing a tool that automatically categorizes
>>>> test results from a json result dump, along with categorizing all the
>>>> current tests.
>>>> I expect to be done with that tool by the end of the week.
>>>>
>>>> Thanks,
>>>> Benjamin Schaaf

Received on Monday, 13 March 2017 09:35:31 UTC