RE: Data on WG participation / activity / progress

> . How much of this did you have to manually add to the data, and how much could you automatically find?)

It took a lot more manual labor than it should have.  Part of that might be my unfamiliarity with how to get information out of the working group database, and much of it was due to my antiquated programming skills that made it easier to hand-crank the data for N=47 than to do it in Python or whatever. But still, I came away with even more sympathy for the AC members who have been pressing the team for a “dashboard” for some time now.  I also came away wishing that all WGs would just move their docs and issues lists to GitHub.  It’s tedious to track down which have and which haven’t.

> There are some obvious issues with the metrics
>  - for groups using github (which is the case for some webapps work), mailing list activity probably isn't a useful measure
>  - I suspect translations of recs are a useful measure - although they are also skewed towards short documents
Good points, but as far as I can tell none of the WGs I flagged has significant discussion threads on GitHub., although your message motivated me to track down the NFC WG’s issues list in GitHub. It’s possible I’m missing some more, but as I mentioned, it would be good to get some help cleaning up the data (not to mention automating keeping it in sync with the WG DB).
As for translations, WGs that don’t manage to even create TRs don’t have anything to translate, right?

>- For groups doing a lot of work, knowing if some work is destined to fail is possibly as helpful as knowing if some particular group is destined to fail.

> Overall, I think we should be looking at actual specs, rather than Working Groups.

Any thoughts on how we could do either a quantitative or qualitative analysis of this question?  My intention in this initial effort was to identify a manageable number of WGs to investigate more closely to see if they are on the road to nowhere.  The supergroups clearly are not, but as you point out some of the sub-efforts may be.

> I think a key part of this is to look at what resources are going into something…

Agree.  It’s on my TO DO list to integrate data from the staff effort table.


From: chaals@yandex-team.ru [mailto:chaals@yandex-team.ru]
Sent: Sunday, January 11, 2015 3:17 PM
To: Michael Champion (MS OPEN TECH); public-success-fail@w3.org
Subject: Re: Data on WG participation / activity / progress

- public-w3process@, w3c-ac-forum@
Hi,

11.01.2015, 10:57, "Michael Champion (MS OPEN TECH)" <Michael.Champion@microsoft.com<mailto:Michael.Champion@microsoft.com>>:

[cross-posting to the Process CG and AC lists to encourage anyone interested to join the discussion in the public-success-fail@w3.org<mailto:public-success-fail@w3.org> mailing list and the task force wiki https://www.w3.org/wiki/AB/2014-2015_Priorities/w3c_work_success<https://www.w3.org/wiki/AB/2014-2015_Priorities/w3c_work_success#Mining_the_.2FTR_page.2C_participation_lists.2C_mailing_lists.2C_and_issues_lists_for_patterns> .]

As you may recall, the AB has created a public task force to try to identify W3C work that is not on the road to success.

I have made some progress on the planned task to mine data to identify WGs who have had the least activity and progress in recent years.  There is now a table mashing up data from the working groups database, and the mailing list archive. See WG Data Mining<https://www.w3.org/wiki/WG_Data_Mining> for links to the data and a description of the fields in the table.
Thanks... that is useful... (not just for the task at hand. How much of this did you have to manually add to the data, and how much could you automatically find?)

>From a simple analysis based on sorting the table by a) having an expired charter for multiple years, b) few messages in their mailing lists in the last half of 2014, c) few technical reports published 2013-2014, and zero CRs / PRs / Recommendations 2013-2015, the following WGs scored near the bottom on on multiple criteria:
·         Web Notifications
·         Forms
·         Near Field Communications
·         Voice Browser

See WG Data Mining Analysis<https://www.w3.org/wiki/WG_Data_Mining_Analysis> for more information.
There are some obvious issues with the metrics
- for groups using github (which is the case for some webapps work), mailing list activity probably isn't a useful measure
- I suspect translations of recs are a useful measure - although they are also skewed towards short documents
- For groups doing a lot of work, knowing if some work is destined to fail is possibly as helpful as knowing if some particular group is destined to fail.


Overall, I think we should be looking at actual specs, rather than Working Groups.

Things that have been in development for a long time are a worry, especially if they have consumed a lot of resources but don't seem to have anyone interested. I think a key part of this is to look at what resources are going into something…

That said, a qualitive analysis is IMHO critical. XHR has been worked on over a long time, and any argument that the technology is doomed to fail depends on what the spec says. I.e. it may be unfeasible to make a critical change to the point where any such work is basically doomed, while it is pretty clear that XHR is likely to be used on the Web for a little while yet.

cheers



Please contribute to the wiki and join the discussion. I have not done any qualitative research to investigate whether these WGs are actually on the road to failure, and some of this data collecting and analysis was done "by hand" and could have errors. Please consider joining the task force to help interpret/correct this data and analysis. I'm sure that more sophisticated data science could yield lots more interesting insights into the questions the task force is addressing, so feel free to contribute your expertise!










--
Charles McCathie Nevile - web standards - CTO Office, Yandex
chaals@yandex-team.ru<mailto:chaals@yandex-team.ru> - - - Find more at http://yandex.com