Re: Rebalancing How the Web is Built

Interesting proposal.

Two questions:

   1. Don't large W3C member firms often face the same frustrations with
   other large competitor firms? Is this really a size issue ultimately, or a
   general governance issue?
   2. Would your proposed approach have W3C funding implications, where
   some major donors find it less useful for constraining competition?

And one tweak for consideration:

RE: "Produce two implementations and a test suite."

I'd suggest three, and on different platforms.

Joseph Potvin
Operations Manager | Gestionnaire des opérations
The Opman Company | La compagnie Opman
jpotvin@opman.ca
Mobile: 819-593-5983

On Sun, Sep 11, 2016 at 11:39 AM, Manu Sporny <msporny@digitalbazaar.com>
wrote:

> Hi all,
>
> The mailing list traffic has been a bit anaemic since the Web Payments
> Interest Group and Working Group started over two years ago. There's a
> reason for that: a few of us involved with the work across the Web
> Payments Community Group, Interest Group, and Working Group became
> disillusioned regarding the process of doing work in this Community
> Group and transitioning it into a Working Group.
>
> Fundamentally, we believe the tables to be tilted in a way that is not
> in favor of Community Groups and thus having this community work on
> anything else without getting some traction in the Web Payments Working
> Group felt like we'd be complicit in creating an environment of false hope.
>
> The good news is that this has recently changed. Some of the work that
> this group has done is now transitioning to a Web Payments Working Group
> First Public Working Draft:
>
> Web Payments HTTP API Transitions to FPWD:
> https://lists.w3.org/Archives/Public/public-payments-wg/2016Sep/0047.html
>
> I've recently had some time to write down some thoughts on transitioning
> work from a Community Group to a Working Group where there are very
> powerful players involved. It's a proposal, but thought it would be
> worth it for this group to understand the dynamic we're under at W3C.
> We'll have follow-up posts on the Web Payments HTTP API over the next
> couple of weeks, but for now here's the blog post explaining the current
> dynamic of getting new work started at W3C:
>
> The World Wide Web Consortium members standardize technology for the
> next generation Web. It’s arguable that the way this process works does
> not provide a clear path to progress work and favors large
> organizations. This blog post explains why we ended up here and how we
> could make the process more fair and predictable.
>
> http://manu.sporny.org/2016/rebalancing/
>
> The contents of the blog post can be found below for W3C archival purposes:
>
> ---------------------
> Rebalancing How the Web is Built
>
> Since the early 1990s, the World Wide Web Consortium (W3C) has
> been the organization where most of the next generation Web has
> been standardized. It is where many of the important decisions
> around HTML, CSS, and Browser APIs are made. It is also where many
> emerging non-browser related technologies like Linked Data,
> Blockchain, Automotive Web, and Web of Things are incubated.
>
> The consortium has always tried to balance the needs of large
> corporations with the needs of the general public to varying
> degrees of success. More recently, a number of the smaller
> organizations at W3C have noted how arduous the process of
> starting new work at W3C has become while the [10]behavior of
> large organizations continues to cause heartburn. As a result,
> some of us are concerned that the process is going to further tilt
> the playing field toward large organizations and unnecessarily
> slow the rate of innovation at the heart of the Web.
>
> In this blog post I’m going to try to explain how creating new
> technology for the core of the Web works. I’ll also suggest some
> changes that W3C could make to make the process more predictable
> and hopefully more fair to all participants.
>
> >From There to Here
>
> For most of its existence, the W3C has had a process for
> developing next generation technology for the [11]Web Platform
> that goes something like this:
>
> 1. Wait for a group of people to identify a problem in the Web
> Platform and propose how adding specific new features would
> address the problem.
> 2. Create a Working Group to standardize the features in the Web
> Platform that would solve the identified problem.
> 3. Release an “official” technical specification under a patent
> and royalty-free license that is used by Web developers to
> varying degrees of success.
>
> It takes anywhere from 3-6 years for a technology to get through
> the W3C process. The formation and operation of a Working Group
> tends to be where the costs are fairly high in terms of W3C staff
> resources, which means the cost of failure is also relatively high
> for the organization as a whole. This has led to changes to the
> process that have raised the bar, in a negative way, on what is
> necessary to start a Working Group. The playing field has been
> tilted to the point that some of us are concerned that a handful
> of large organizations now have a tremendous amount of influence
> on where the Web is going while large coalitions of smaller
> organizations or the general public struggle to have their
> concerns addressed in the Web Platform.
>
> Recently, W3C has [12]undergone a reorganization to make it more
> responsive to the needs of the organizations and people that use
> the Web. The reorganization splits the organization into multiple
> functional groups: Strategy, Project Management, Architecture and
> Technology, Global Participation, Member Satisfaction, Business
> Development, etc. While the reorganization feels like a move in
> the right direction, it doesn’t seem to address how difficult it
> is to start and shepherd work through the W3C. Let’s analyze why
> that is.
>
> Why Shepherding Work Through the W3C is Onerous
>
> It is claimed that one of the primary bottlenecks at W3C when it
> comes to starting a new Working Group is allocating W3C staff
> member time to the project. The argument goes that there are just
> not enough W3C staffers for the workload of the organization. This
> is because W3C exists on a thin gruel of funding from membership
> fees (as well as other disparate sources) and hiring W3C staff to
> help support new work is financially burdensome. We have limited
> resources, so we must only pursue work that has the very highest
> likelihood of success. This means that parts of the Web Platform
> ecosystem develop far more slowly than they should.
>
> This dynamic has prompted a number of the larger organizations at
> W3C to push back on new charters citing this bottleneck. This is a
> misguided strategy. We shouldn’t be focused on eliminating as much
> risk as possible, we should be focused on reducing the cost of
> making mistakes.
>
> As a result of the focus on eliminating as much risk of failure as
> possible, some of these large organizations have started
> requesting new requirements for starting work. Some examples
> include requirements like complete technical specifications,
> “significant” deployments to customers, and large vendor support.
> These new requirements are intended to increase the likelihood of
> success and slow the rate of new work at W3C. While slowing the
> rate of new work may address the staffing issue, it also creates a
> situation where the W3C can’t keep up with the required amount of
> standardization needed for the Web Platform to stay relevant. It
> also creates a bias in favor of large organizations.
>
> We have seen these new requirements ignored when it suits a large
> organization to do so (e.g. Web Payments and Web Application
> Security). If a large organization can see how their company
> financially benefits from an added feature to the Web Platform
> they seem to have a much easier time getting work started at the
> W3C as the “starting new work” requirements are not applied to
> them in the same way as they are to the smaller organizations at
> W3C.
>
> While I won’t go into all of them here, there are more double
> standards, no pun intended, at play when transitioning work to a
> Working Group at W3C. This dynamic results in significant
> frustration from the smaller organizations at W3C when the goal
> posts for starting new work keep changing based on the current
> desires of the larger organizations.
>
> Regardless of whether or not this is a staffing issue, a
> technology maturity issue, or one of the other new requirements at
> play at W3C, one thing is certain. The list of requirements for
> transitioning work from an experimental technology to a Working
> Group at W3C is not clear and seems to favor larger organizations.
> It is a source of extreme frustration for those of us that are
> trying to help build the next generation Web and are NOT working
> for a multi-billion dollar multinational corporation.
>
> The W3C Working Group Formation Checklist
>
> One remedy for the aforementioned problems is to employ a simple
> but detailed checklist. This is one of the tools that is currently
> missing for W3C members.
>
> Some have argued that this lack of a clear checklist is by design.
> Some argue that the formation of every new Working Group is unique
> and requires discussion and debate. Having participated in
> starting multiple groups at W3C over the past decade, I disagree.
> The details differ, but the general topics that are debated remain
> fairly consistent. The data that you need to support the creation
> of a Working Group tend to be the same.
>
> Here is the checklist that I’ve put together over the years after
> much trial and error at W3C. The purpose of the checklist is to
> gather data so that your community can prove that it has done its
> due diligence to the W3C membership when requesting the creation
> of a Working Group.
>
> 1. Clearly identify and articulate a problem statement. Do this
> by sending out a survey to all organizations that you believe
> may benefit from a standard. You will need at least 35
> organizations to become actively involved. I typically end up
> having to contact close to 100 organizations to get this core
> group formed. Example: [13]Digital Offers Problem Statement
> Survey.
> 2. Create a W3C Community Group around a broad version of the
> problem statement and invite organizations to join the group.
> Organize weekly calls to coordinate activity. Example:
> [14]Credentials Community Group
> 3. Gather use cases that these organizations want to support.
> Specifically, gather use cases that are currently impossible
> to achieve. This should be in the form of another survey where
> you ask each organization about their most important use case.
> You will eventually process this list down to 3-5 core use
> cases. Example: [15]Verifiable Claims Use Cases Survey and
> [16]Verifiable Claims Use Cases.
> 4. Perform a gap analysis. Identify capabilities that are missing
> from the Web Platform and explain why those capabilities can
> address some of the use cases outlined in the previous step.
> 5. Produce a proposed architecture and technical specification
> that addresses part or all of the problem statement. If
> possible, get multiple implementations of the technical
> specification and deploy it at a few customer sites. Example:
> [17]Verifiable Claims Architecture Document
> 6. Create a draft charter for a Working Group that will take the
> technical specification through the W3C standardization
> process. The charter time frame should be for no more than 24
> months: 6 months to spin up, 12 months to finalize the specs,
> 6 months to complete interoperability testing. Example:
> [18]Verifiable Claims Working Group Draft Charter
> 7. Create an executive summary for W3C member companies.
> Summarize the work that the group has done. At this point you
> have far more content than W3C member company representatives
> have the time to read to determine if they want to join the
> initiative. Ease their decision making process by providing a
> summary. Example: [19]Verifiable Claims Executive Summary for
> W3C Members
> 8. Measure buy-in for the proposed Working Group Charter and
> technical deliverables. The best way to do this is via another
> survey that should be distributed to any organization that
> participated in step #1, any organization that has joined the
> work since then, and any W3C member organization that may be
> impacted by the work. Example: [20]Demonstration of Support
> for Verifiable Claims Working Group Charter Survey
>
> I don’t expect much in the list above to be controversial. There
> are templates that the W3C membership should create for the
> surveys and reports above. Having this W3C Working Group Formation
> Checklist available will help organizations navigate the often
> confusing W3C Process.
>
> Reducing Reliance on W3C Staff
>
> The checklist above makes the process of going from an
> experimental technology to a W3C Working Group more clear. The
> checklist, however, does not alleviate the W3C staffing shortage.
> In fact, if the way W3C staff is utilized does not also change, it
> might make the current situation worse.
>
> The W3C staff play a very important role in that they help
> organizations navigate how to get things done via the W3C Process.
> They help build consensus before and during a Working Group
> activity at W3C. This can be a double-edged sword. If there is a
> W3C staff member available to help, they can be a fantastic
> champion for a group’s work. If there isn’t, and you’ve never
> progressed work yourself through W3C, you will find yourself in
> the unfavorable position of not knowing how to proceed. Many of
> the staff members also have their own way of navigating the W3C
> process and many of them have to repeat themselves when new work
> is started. In short, there is a lot of engineering churn and
> tribal knowledge at W3C; this is true of most standardization
> bodies.
>
> The unfortunate truth is that building out the Web platform is
> currently being restricted due to the way that we start and
> progress new work at W3C. The W3C membership relies on the W3C
> staff to its detriment. The W3C staff are incredibly helpful, but
> there aren’t enough of them to support building out all of the new
> functionality needed for the Web, and this is ultimately bad for
> the Web.
>
> In order to address the staffing shortage, it is proposed that we
> shift much of their work from executing upon the proposed
> checklist, which is more or less what they do today, to verifying
> that the checklists are being processed correctly. This is
> effectively a quality control check on the work that groups are
> doing as they work their way through the proposed checklist. This
> offloads a significant chunk of W3C staff time to the groups that
> want to see their work get traction while giving those groups
> clear goals to achieve.
>
> Blocking Work at W3C
>
> W3C members currently have the ability to stop work that has
> ticked all of the necessary boxes in the checklist mentioned
> above. At present, large organizations tend to not become involved
> with work until a vote for the Working Group is circulated by W3C
> staff. Some members then suggest that they may vote against the
> work if their viewpoint isn’t taken into account even though they
> have not participated in the work to date. Some call this a part
> of the process, but it is uselessly frustrating to those that have
> spent years building consensus around a Working Group proposal
> only to have a large W3C member company respond with a “Why wasn’t
> I consulted” retort.
>
> So, now for the controversial bit:
>
> A W3C Community Group’s work should automatically transition to a
> W3C Working Group if a significant coalition of companies, say
> around 35 of them, have ticked all of the boxes in a predefined
> W3C Working Group Formation checklist.
>
> This checklist should include the creation of two interoperable
> implementations and a test suite. In other words, it should meet
> all of the minimum bars for a Working Group’s success per the W3C
> Process. Making this change achieves the following:
>
> 1. It clearly defines how experimental technology progresses to
> being part of the Web platform so that organizations can
> allocate resources appropriately.
> 2. It reduces the workload on W3C staff members, which is a very
> limited resource.
> 3. It prevents large organizations from blocking work at W3C
> while addressing their concerns around technological maturity.
>
> In the end, the checklist for starting work at W3C could look
> something like this:
>
> 1. Clearly identify and articulate a problem statement.
> 2. Create a W3C Community Group.
> 3. Gather and document use cases.
> 4. Perform a gap analysis.
> 5. Produce a proposed architecture and technical specification.
> 6. Produce two implementations and a test suite.
> 7. Create a draft charter for a Working Group.
> 8. Create an executive summary for W3C member companies.
> 9. Survey organizations and demonstrate that at least 35 of them
> are supportive of the work and technical direction.
>
> Each step above would have document templates that W3C members can
> use as a starting point. None of the steps above require W3C staff
> resources and if all steps are completed, the formation of a
> Working Group is the natural outcome and should not be blocked by
> the membership.
>
> This checklist approach may be seen as too constraining for some,
> and that’s why it’s voluntary. Some organizations may feel that
> they do not need to produce all of the work above to get a Working
> Group and for those organizations, they can choose to ignore the
> checklist. Initiatives not using the checklist should expect push
> back if they choose to not answer some of the questions that the
> checklist covers.
>
> Rebalancing How the Web is Built
>
> The current process for developing next generation standards for
> the Web is too unpredictable and too constrained by limited W3C
> resources. There are groups of small organizations that want to
> help create these next generation standards; we have to empower
> those groups with a clear path to standardization. We must not
> allow organizations to block work if the champions of the work
> have met the requirements in the checklist. Doing this will free
> up W3C staffing resources to ensure that the Web Platform is
> advancing at a natural and rapid rate.
>
> A W3C Working Group formation checklist would help make the
> process more predictable, require less W3C staff time to execute,
> and provide a smoother path from the inception of an idea, to
> implementation, to standardization of a technology for the Web
> platform. This would be great for the Web.
>
> References
>
> 10. https://manu.sporny.org/2016/browser-api-incubation-antipattern/
> 11. https://www.w3.org/wiki/Open_Web_Platform
> 12. https://www.w3.org/2016/08/2016-reorg.html
> 13.
> https://docs.google.com/forms/d/e/1FAIpQLScVDxYlBTWchzK3kThdRTn0
> nulipPy6Hb5qLmRDLOdBtioFfQ/viewform
> 14. https://www.w3.org/community/credentials/
> 15.
> https://docs.google.com/forms/d/e/1FAIpQLSdQJWx8eMRkoTwA2TIWbigT
> ipZ2lqZ_UPFzJbY8IvqmVAHr6A/viewform
> 16. https://w3c.github.io/webpayments-ig/VCTF/use-cases/
> 17. http://w3c.github.io/webpayments-ig/VCTF/architecture/
> 18. http://w3c.github.io/webpayments-ig/VCTF/charter/rc-2.html
> 19. http://w3c.github.io/webpayments-ig/VCTF/primer/
> 20. http://w3c.github.io/webpayments-ig/VCTF/support/
>
> -- manu
>
> --
> Manu Sporny (skype: msporny, twitter: manusporny, G+: +Manu Sporny)
> Founder/CEO - Digital Bazaar, Inc.
> blog: Rebalancing How the Web is Built
> http://manu.sporny.org/2016/rebalancing/
>
>