W3C home > Mailing lists > Public > public-rdf-shapes@w3.org > May 2017

Re: formal objection to early cutoff of test submissions

From: Peter F. Patel-Schneider <pfpschneider@gmail.com>
Date: Wed, 10 May 2017 10:26:26 -0700
To: Irene Polikoff <irene@topquadrant.com>
Cc: Holger Knublauch <holger@topquadrant.com>, public-rdf-shapes@w3.org
Message-ID: <2d7df66c-65a2-a57d-2a90-87b81d97a288@gmail.com>
As this problem is largely moot, I no longer deem it important enough to raise
a formal object over.

Peter F. Patel-Schneider
Nuance Communications

On 05/10/2017 08:45 AM, Irene Polikoff wrote:
> Peter,
> 
> The decision to accept tests submitted after the deadline, should the WG vote
> to include them, was minuted in
> the https://www.w3.org/2017/05/03-shapes-minutes.html.
> 
> Do you see this as a sufficient announcement and are, therefore, withdrawing
> your formal objection? If not, let us know what you would see as a sufficient
> announcement.
> 
> Regards,
> 
> Irene
> 
>> On May 5, 2017, at 3:39 PM, Peter F. Patel-Schneider <pfpschneider@gmail.com
>> <mailto:pfpschneider@gmail.com>> wrote:
>>
>>
>>
>> It appears the the working group has changed its policy and will now
>> continue to accept test suite submissions subject to a vote of the working
>> group.  If the policy on test suite submissions has changed and this change
>> in policy is announced, this formal objection is moot.
>>
>> Peter F. Patel-Schneider
>> Nuance Communications
>>
>>
>> On 05/02/2017 09:58 AM, Irene Polikoff wrote:
>>> To me, this sounds like a procedural objection:
>>>
>>>>> Cutting off the submission of tests well before the
>>>>> end of the candidate recommendation period hinders wide review of the
>>>>> candidate recommendation and goes against the purpose of the W3C process.
>>>
>>> I think the key phrase is "well before the end of the candidate recommendation
>>> period”.
>>>
>>> I believe the process has been followed:
>>>
>>>  * Obviously, submission of tests to be used as part of the recommendation
>>>    track process needs to be concluded before the end of the end of the
>>>    candidate recommendation period.
>>>      o This is because implementers need time to run all tests and make fixes
>>>        if necessary.
>>>      o Then the WG needs to review the results, draft the transition request
>>>        and decide to submit it.
>>>      o Then request to transition has to be sent and transition meeting has
>>>        to happen.
>>>      o All these activities have to occur before the candidate recommendation
>>>        period ends. Conservatively speaking, they take at least a month.
>>>  * Further, the WG issued a “last call” for tests asking any interested
>>>    parties to let us know if they need extra time and, if so, how much. We
>>>    promised to “work something out” if such parties tell us about their plans
>>>    for submitting
>>>    tests
>>> https://lists.w3.org/Archives/Public/public-rdf-shapes/2017Apr/0034.html
>>>      o The last call also said that additional tests can be submitted at any
>>>        time - as something for the implementers to test with, but, after the
>>>        deadline, they will not be used as part of the recommendation track
>>>        process.
>>>  * Peter was the only one who responded and his response was not definitive.
>>>      o I am saying this because he questioned the short timeline and said he
>>>        was working on additional test cases, but never identified the
>>>        deadline he could work towards.
>>>  * We are now a week beyond the date set in the “last call for tests” and the
>>>    WG has been “working out something" even after that date. This has been
>>>    hindered by the bugs in submitted tests. I would expect that requiring
>>>    bug-free test submissions does not violate the process. In any case, I
>>>    think we went over and above by accepting tests with bugs and fixing said
>>>    bugs.
>>>  * Obviously, we need to have a “stop date”. With the charter currently set
>>>    to end on June 1st, I don’t see how this date can be later than today.
>>>
>>>
>>> Overall, I believe that cutting the submission of the tests 5 weeks before the
>>> end of the period is not “well before the end”. In practice, however, WG
>>> allowed submission of tests to be used as part of the process for another week
>>> - cutting it off 4 weeks before the end which is probably the minimal time
>>> required to go through all the logistics that must happened before the period
>>> ends. Sandro, of course, as W3C staff member, will be able to tell us if there
>>> are any process issues.
>>>
>>> Regards,
>>>
>>> Irene Polikoff
>>>
>>>
>>>
>>>> On May 2, 2017, at 3:53 AM, Holger Knublauch <holger@topquadrant.com
>>>> <mailto:holger@topquadrant.com>
>>>> <mailto:holger@topquadrant.com>> wrote:
>>>>
>>>>
>>>>
>>>> On 2/05/2017 1:50, Peter F. Patel-Schneider wrote:
>>>>> This is a formal objection to the early cutoff of submissions to the working
>>>>> group's test suite.
>>>>>
>>>>> The candidate recommendation period in the W3C process is designed so that
>>>>> implementers can test out a potential W3C recommendation to see whether
>>>>> conforming implementations can be created.  Testing to confirm
>>>>> implementation behaviour is thus an important part of the candidate
>>>>> recommendation process.  Cutting off the submission of tests well before the
>>>>> end of the candidate recommendation period hinders wide review of the
>>>>> candidate recommendation and goes against the purpose of the W3C process.
>>>>>
>>>>>
>>>>> Two tests were submitted that examine the behaviour of SHACL implementations
>>>>> on property paths that contain information about two different kinds of
>>>>> SPARQL paths.  These tests came from recent implementation experience.  The
>>>>> aspect of SHACL that they test is not covered by any of the tests in the
>>>>> current working group's test suite so without them there is no confirmation
>>>>> that there are two independent implementations of SHACL property paths.
>>>>> These tests were not accepted.
>>>>
>>>> Since you seem to find this so very important I have now added these two
>>>> test cases to the suite. I still don't agree they are important, but I don't
>>>> think this is worth spending any more time on. I have notified the other
>>>> implementers about the new test(s) and hope they can re-run their tests.
>>>>
>>>>>
>>>>> Two tests were submitted that examine the behaviour of SHACL implementations
>>>>> of pre-binding.  Pre-binding underlies all of SHACL-SPARQL so it is vitally
>>>>> important for SHACL-SPARQL both that pre-binding have a suitable definition
>>>>> and that implementations correctly implement pre-binding.  The working group
>>>>> approved changes to the definition of pre-binding on 26 April 2017 and the
>>>>> tests were submitted on 28 April 2017.  These tests replaced previous tests
>>>>> to examine the behaviour of SHACL implementations of pre-binding that had
>>>>> become irrelevant because of the change to pre-binding.  These tests were
>>>>> not accepted.
>>>>
>>>> I have just made the use of VALUES in general ill-formed in SHACL-SPARQL,
>>>> making your submitted test cases outdated. I had never rejected them, see my
>>>> other email in this thread.
>>>>
>>>> So both of the examples that you have used to motivate your formal objection
>>>> are now handled.
>>>>
>>>> Holger
>>>>
>>>>
>>>
> 
Received on Wednesday, 10 May 2017 17:27:02 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:02:50 UTC