Re: Micropub test suite

*Discovery*
The discovery test does not have a number, but by doing the setup process
with the sign-in flow, you will pass this test. The process provides lots
of feedback during the sign-in flow if you do something wrong:

   1. https://media.aaronpk.com/Screen-Shot-2017-03-07-07-09-50.png
   2. https://aaronparecki.com/2017/02/08/19/image-1.png
   3. https://aaronparecki.com/2017/02/08/19/image-2.png
   4. https://media.aaronpk.com/Screen-Shot-2017-03-07-07-10-14.png

*Authentication*
Similarly, there is no test number, but by using any of the tests, you will
check off the boxes depending on how the request is made. The "scope" item
is checked off as soon as the authorization process begins.

*Syntax*
Each item in syntax has a corresponding test. I've updated the report
template to show the test number that corresponds with each checkbox.

*Creating Posts*
Each item in creating posts has a corresponding test, with the exception of
"recognizes HTTP 201 and 202..." which is checked off after you create a
post with any of the other tests and the client redirects to the post. In
the report template, each item now references the corresponding test number.

*Media Endpoint*
There is a test for the first item, but not the second.

*Updates, Deletes, Undeletes*
No tests for these yet.

*Querying*
There are tests for the two endpoint queries. Tests for querying post
information are still in progress.

All the tests are pretty picky about syntax. I've already had to make a
couple of fixes to Quill in order to pass the tests. Things like when you
send a Micropub JSON request, the values have to be arrays. Here's all the
error messages I return for the tests:
https://github.com/aaronpk/micropub.rocks/blob/master/app/ClientTests.php#L501

I spent today adding tests for querying and media endpoints, as well as
making micropub.rocks fill out the implementation report template for you.
Now as you work through the tests, it will check off all the boxes as you
go.


----
Aaron Parecki
aaronparecki.com
@aaronpk <http://twitter.com/aaronpk>


On Mon, Mar 6, 2017 at 8:37 PM, Sandro Hawke <sandro@w3.org> wrote:

> Excellent.   Can you characterize which features have automated tests and
> which do not?     Is there anything boundary testing about them, testing
> the kind of thing someone might get wrong, or middle-of-the-road
> probably-everyone-will-pass tests?
>
>      -- Sandro
>
>
>
> On 03/06/2017 07:50 PM, Aaron Parecki wrote:
>
> We now have a client test suite for creating posts! Also the good news is
> that the test suite passes itself, as demonstrated in the below video.
>
>
> This is live on the micropub.rocks website now! You can read more about
> the details here:
>
> https://aaronparecki.com/2017/03/06/14/day-76-micropub-rocks-client-tests
>
>
>
>
> ----
> Aaron Parecki
> aaronparecki.com
> @aaronpk <http://twitter.com/aaronpk>
>
>
> On Mon, Mar 6, 2017 at 8:11 AM, Sandro Hawke <sandro@w3.org> wrote:
>
>> Apparently I wasn't paying enough attention at this point of the meeting
>> last week.  Sorry for not catching this.
>>
>> To answer the question: I'm pretty sure we did not explicitly, with full
>> consideration, agree to relax our previous plan of having complete test
>> suites.  I see how that's implicit in the decision we made, and I recall
>> Aaron mentioning it, so maybe everyone else thought it through, but in the
>> mix of all the things going on during that meeting, I didn't put 2+2
>> together.
>>
>> I agree we should be consistent on this.   In general, I'd say every
>> constraint in the spec ought to have a few tests.   That's not a constraint
>> of W3C process though -- the WG is free to set a different bar for
>> interoperability and confirming implementations -- but we probably do need
>> to be rational and consistent in setting that bar.
>>
>> So, which way do we want to go on this?
>>
>> And Aaron, how much of a burden would it be to finish that test suite (or
>> can we recruit someone else to do it?)
>>
>>     -- Sandro
>>
>>
>>
>> On 03/06/2017 10:20 AM, Amy G wrote:
>>
>> Given the resolution about advancing Micropub to PR at the last meeting,
>> did the working group decide that we don't need actually need complete test
>> suites to progress to PR so long as there are reports? Does this also apply
>> to LDN, WebSub and ActivityPub?
>>
>> On 6 March 2017 at 23:05, Aaron Parecki <aaron@parecki.com> wrote:
>>
>>> Correct, I have a start to the client tests but I haven't launched it on
>>> the site yet. The implementation reports for clients are all self reported.
>>> On Mon, Mar 6, 2017 at 6:49 AM Amy G <amy@rhiaro.co.uk> wrote:
>>>
>>>> Hola,
>>>>
>>>> Just to confirm - there are no tests for Micropub clients right? You
>>>> can only test a server implementation with the current test suite?
>>>>
>>>> Amy
>>>>
>>>
>>
>>
>
>