Re: Call for Agenda: Testing Task Force Sync Call, 30Jan

> Bumblefudge <bumblefudge@learningproof.xyz> wrote:
> 
> Please reply-all to the list if you would like to put something on the agenda.


1. I can provide a brief update on the status of FediTest. There is some running code, but still experimental and a bit too early to show-and-tell.

2. I could also provide an overview over the responses to my survey on developers and their development setup and virtualization. There are 44 responses so far, some of it is unexpected (to me). (A number that surprised me!) Survey is here: https://apps.dazzlelabs.net/nextcloud/apps/forms/s/ed2WBPzrrWFcWKjT5a9MwMGp 

More importantly, echo’ing some of what Marcus said:

> On Jan 23, 2024, at 01:17, Marcus Rohrmoser <me.swicg@mro.name> wrote:
> 
> Yes, however I am unsure how to phrase it. 
> 
> 1. IMO the tests should benefit the netizens, operators and developers and therefore should be easy to operate and friendly to ad-hoc deploy (rather than 24/7) for unprevileged (non-root) netizens. Without vendor lock in. And tomorrow and the week after as well, without permanent underlying framework changes to be integrated (upwards compatibility. I know it's hard). So I advocate thinking from the end towards the means, not vice versa. What options are there aside php and cgis?
> 
> 2. Many ActivityPub processes are asynchronous in nature and are hard to follow and therefore test. IMO we should encourage a 'friendly feedback' policy to
> a. immediately report back the effect in a machine readable manner, evtl. with an url to track the progress,
> b, if asynchronous, then call back once there is a result,
> c. never silently discard requests.

3. I think it would be useful to spend some time on the requirements side of testing for the Fediverse.

We sometimes wave “testing!!!” around as some kind of magic wand, but as we can see from the various projects, I don’t think we quite agree on what testing should be done, and in particular Why. Also: Who needs it and what does it need to look like so they can get the maximum benefit out of a testing environment and the results of testing?

4. Another important subject would be: just where exactly do those tests come from? Who decides what is and isn’t a valid test? That’s particularly important because AS and AP are so flexible. Example:

* if anybody gets to do anything allowed by AS and AP, hopes of out-of-the-box interop as low, and testing tells developers who want real-world interoperability fairly little.

* if “passing all tests” is supposed to mean “will interoperate with 90% of the installed fediverse base”, then many tests have to be defined that do not have a root in a W3C or other standards document, but test for conventions deployed by the leading implementations.

(Personally I believe we need to have both, and tests needs to be organized in a very modular fashion based on the “authority” from which they are derived.)

So plenty to talk about from my perspective :-)

Cheers,



Johannes.

Johannes Ernst

Fediforum <https://fediforum.org/>
Dazzle Labs <https://dazzlelabs.net/>