Re: Rebalancing How the Web is Built

FWIW IMHO, Melvin's work in this area is highly developed and has an array
of applications.

Particularly when paired with LDP / SoLiD like systems.

Time should be provided to explore his work and communicate the methods and
benefits to other who may not understand the consideration made by Melvin,
should they only provide the work a cursory review.

Tim.h.

On Wed., 14 Sep. 2016, 9:16 pm Melvin Carvalho, <melvincarvalho@gmail.com>
wrote:

> On 11 September 2016 at 17:39, Manu Sporny <msporny@digitalbazaar.com>
> wrote:
>
>> Hi all,
>>
>> The mailing list traffic has been a bit anaemic since the Web Payments
>> Interest Group and Working Group started over two years ago. There's a
>> reason for that: a few of us involved with the work across the Web
>> Payments Community Group, Interest Group, and Working Group became
>> disillusioned regarding the process of doing work in this Community
>> Group and transitioning it into a Working Group.
>>
>> Fundamentally, we believe the tables to be tilted in a way that is not
>> in favor of Community Groups and thus having this community work on
>> anything else without getting some traction in the Web Payments Working
>> Group felt like we'd be complicit in creating an environment of false
>> hope.
>>
>> The good news is that this has recently changed. Some of the work that
>> this group has done is now transitioning to a Web Payments Working Group
>> First Public Working Draft:
>>
>> Web Payments HTTP API Transitions to FPWD:
>> https://lists.w3.org/Archives/Public/public-payments-wg/2016Sep/0047.html
>
>
> Hi Manu
>
> Regarding work in the CG.
>
> I've done quite a bit of work in the last year on my linked data based
> system, webcredits.
>
> I like to work with running code, so rather than standardizing, then
> implementing I have gone the other way round.
>
> I crated a proof of concept and very rough spec.  And have been heavily
> testing it in various scenarios.
>
> I'm extremely happy with the functionality.  Webcredits is at version 0.1
> and I'd like to get it to v 0.2 which will be a general cleanup and
> creating a promises API on top of the async functionality.  It has basic
> signatures but that could possibly do with another round of coding.
>
> My main wish is to get payments working in the browser.  Looking at the
> work in the WG it seems to be encumbered by existing systems, namely web
> 2.0 browsers, which I think has extended the timeline of that work.
>
> I'd like to get to proof of concept of a new style of linked data
> frictionless payments in the browser, demonstrated via the solid platform.
>
> In addition to this I'd like to start working on a more polished spec in
> the CG, which might be within the charter of the WG too, tho im not holding
> out on that, nor letting the WG hold back innovation.
>
> I believe I can mock up actual payments in the browser via a shim quite
> quickly.  I am also increasingly aware we desperately need more (micro)
> payments browser that is payments ready, and data ready.  Something like
> Brave + Amaya redux.  It's not infeasible as a stretch goal that something
> like this could be incubated in the crypto currency community.
>
> What do you think are next steps?
>
>
>>
>>
>> I've recently had some time to write down some thoughts on transitioning
>> work from a Community Group to a Working Group where there are very
>> powerful players involved. It's a proposal, but thought it would be
>> worth it for this group to understand the dynamic we're under at W3C.
>> We'll have follow-up posts on the Web Payments HTTP API over the next
>> couple of weeks, but for now here's the blog post explaining the current
>> dynamic of getting new work started at W3C:
>>
>> The World Wide Web Consortium members standardize technology for the
>> next generation Web. It’s arguable that the way this process works does
>> not provide a clear path to progress work and favors large
>> organizations. This blog post explains why we ended up here and how we
>> could make the process more fair and predictable.
>>
>> http://manu.sporny.org/2016/rebalancing/
>>
>> The contents of the blog post can be found below for W3C archival
>> purposes:
>>
>> ---------------------
>> Rebalancing How the Web is Built
>>
>> Since the early 1990s, the World Wide Web Consortium (W3C) has
>> been the organization where most of the next generation Web has
>> been standardized. It is where many of the important decisions
>> around HTML, CSS, and Browser APIs are made. It is also where many
>> emerging non-browser related technologies like Linked Data,
>> Blockchain, Automotive Web, and Web of Things are incubated.
>>
>> The consortium has always tried to balance the needs of large
>> corporations with the needs of the general public to varying
>> degrees of success. More recently, a number of the smaller
>> organizations at W3C have noted how arduous the process of
>> starting new work at W3C has become while the [10]behavior of
>> large organizations continues to cause heartburn. As a result,
>> some of us are concerned that the process is going to further tilt
>> the playing field toward large organizations and unnecessarily
>> slow the rate of innovation at the heart of the Web.
>>
>> In this blog post I’m going to try to explain how creating new
>> technology for the core of the Web works. I’ll also suggest some
>> changes that W3C could make to make the process more predictable
>> and hopefully more fair to all participants.
>>
>> >From There to Here
>>
>> For most of its existence, the W3C has had a process for
>> developing next generation technology for the [11]Web Platform
>> that goes something like this:
>>
>> 1. Wait for a group of people to identify a problem in the Web
>> Platform and propose how adding specific new features would
>> address the problem.
>> 2. Create a Working Group to standardize the features in the Web
>> Platform that would solve the identified problem.
>> 3. Release an “official” technical specification under a patent
>> and royalty-free license that is used by Web developers to
>> varying degrees of success.
>>
>> It takes anywhere from 3-6 years for a technology to get through
>> the W3C process. The formation and operation of a Working Group
>> tends to be where the costs are fairly high in terms of W3C staff
>> resources, which means the cost of failure is also relatively high
>> for the organization as a whole. This has led to changes to the
>> process that have raised the bar, in a negative way, on what is
>> necessary to start a Working Group. The playing field has been
>> tilted to the point that some of us are concerned that a handful
>> of large organizations now have a tremendous amount of influence
>> on where the Web is going while large coalitions of smaller
>> organizations or the general public struggle to have their
>> concerns addressed in the Web Platform.
>>
>> Recently, W3C has [12]undergone a reorganization to make it more
>> responsive to the needs of the organizations and people that use
>> the Web. The reorganization splits the organization into multiple
>> functional groups: Strategy, Project Management, Architecture and
>> Technology, Global Participation, Member Satisfaction, Business
>> Development, etc. While the reorganization feels like a move in
>> the right direction, it doesn’t seem to address how difficult it
>> is to start and shepherd work through the W3C. Let’s analyze why
>> that is.
>>
>> Why Shepherding Work Through the W3C is Onerous
>>
>> It is claimed that one of the primary bottlenecks at W3C when it
>> comes to starting a new Working Group is allocating W3C staff
>> member time to the project. The argument goes that there are just
>> not enough W3C staffers for the workload of the organization. This
>> is because W3C exists on a thin gruel of funding from membership
>> fees (as well as other disparate sources) and hiring W3C staff to
>> help support new work is financially burdensome. We have limited
>> resources, so we must only pursue work that has the very highest
>> likelihood of success. This means that parts of the Web Platform
>> ecosystem develop far more slowly than they should.
>>
>> This dynamic has prompted a number of the larger organizations at
>> W3C to push back on new charters citing this bottleneck. This is a
>> misguided strategy. We shouldn’t be focused on eliminating as much
>> risk as possible, we should be focused on reducing the cost of
>> making mistakes.
>>
>> As a result of the focus on eliminating as much risk of failure as
>> possible, some of these large organizations have started
>> requesting new requirements for starting work. Some examples
>> include requirements like complete technical specifications,
>> “significant” deployments to customers, and large vendor support.
>> These new requirements are intended to increase the likelihood of
>> success and slow the rate of new work at W3C. While slowing the
>> rate of new work may address the staffing issue, it also creates a
>> situation where the W3C can’t keep up with the required amount of
>> standardization needed for the Web Platform to stay relevant. It
>> also creates a bias in favor of large organizations.
>>
>> We have seen these new requirements ignored when it suits a large
>> organization to do so (e.g. Web Payments and Web Application
>> Security). If a large organization can see how their company
>> financially benefits from an added feature to the Web Platform
>> they seem to have a much easier time getting work started at the
>> W3C as the “starting new work” requirements are not applied to
>> them in the same way as they are to the smaller organizations at
>> W3C.
>>
>> While I won’t go into all of them here, there are more double
>> standards, no pun intended, at play when transitioning work to a
>> Working Group at W3C. This dynamic results in significant
>> frustration from the smaller organizations at W3C when the goal
>> posts for starting new work keep changing based on the current
>> desires of the larger organizations.
>>
>> Regardless of whether or not this is a staffing issue, a
>> technology maturity issue, or one of the other new requirements at
>> play at W3C, one thing is certain. The list of requirements for
>> transitioning work from an experimental technology to a Working
>> Group at W3C is not clear and seems to favor larger organizations.
>> It is a source of extreme frustration for those of us that are
>> trying to help build the next generation Web and are NOT working
>> for a multi-billion dollar multinational corporation.
>>
>> The W3C Working Group Formation Checklist
>>
>> One remedy for the aforementioned problems is to employ a simple
>> but detailed checklist. This is one of the tools that is currently
>> missing for W3C members.
>>
>> Some have argued that this lack of a clear checklist is by design.
>> Some argue that the formation of every new Working Group is unique
>> and requires discussion and debate. Having participated in
>> starting multiple groups at W3C over the past decade, I disagree.
>> The details differ, but the general topics that are debated remain
>> fairly consistent. The data that you need to support the creation
>> of a Working Group tend to be the same.
>>
>> Here is the checklist that I’ve put together over the years after
>> much trial and error at W3C. The purpose of the checklist is to
>> gather data so that your community can prove that it has done its
>> due diligence to the W3C membership when requesting the creation
>> of a Working Group.
>>
>> 1. Clearly identify and articulate a problem statement. Do this
>> by sending out a survey to all organizations that you believe
>> may benefit from a standard. You will need at least 35
>> organizations to become actively involved. I typically end up
>> having to contact close to 100 organizations to get this core
>> group formed. Example: [13]Digital Offers Problem Statement
>> Survey.
>> 2. Create a W3C Community Group around a broad version of the
>> problem statement and invite organizations to join the group.
>> Organize weekly calls to coordinate activity. Example:
>> [14]Credentials Community Group
>> 3. Gather use cases that these organizations want to support.
>> Specifically, gather use cases that are currently impossible
>> to achieve. This should be in the form of another survey where
>> you ask each organization about their most important use case.
>> You will eventually process this list down to 3-5 core use
>> cases. Example: [15]Verifiable Claims Use Cases Survey and
>> [16]Verifiable Claims Use Cases.
>> 4. Perform a gap analysis. Identify capabilities that are missing
>> from the Web Platform and explain why those capabilities can
>> address some of the use cases outlined in the previous step.
>> 5. Produce a proposed architecture and technical specification
>> that addresses part or all of the problem statement. If
>> possible, get multiple implementations of the technical
>> specification and deploy it at a few customer sites. Example:
>> [17]Verifiable Claims Architecture Document
>> 6. Create a draft charter for a Working Group that will take the
>> technical specification through the W3C standardization
>> process. The charter time frame should be for no more than 24
>> months: 6 months to spin up, 12 months to finalize the specs,
>> 6 months to complete interoperability testing. Example:
>> [18]Verifiable Claims Working Group Draft Charter
>> 7. Create an executive summary for W3C member companies.
>> Summarize the work that the group has done. At this point you
>> have far more content than W3C member company representatives
>> have the time to read to determine if they want to join the
>> initiative. Ease their decision making process by providing a
>> summary. Example: [19]Verifiable Claims Executive Summary for
>> W3C Members
>> 8. Measure buy-in for the proposed Working Group Charter and
>> technical deliverables. The best way to do this is via another
>> survey that should be distributed to any organization that
>> participated in step #1, any organization that has joined the
>> work since then, and any W3C member organization that may be
>> impacted by the work. Example: [20]Demonstration of Support
>> for Verifiable Claims Working Group Charter Survey
>>
>> I don’t expect much in the list above to be controversial. There
>> are templates that the W3C membership should create for the
>> surveys and reports above. Having this W3C Working Group Formation
>> Checklist available will help organizations navigate the often
>> confusing W3C Process.
>>
>> Reducing Reliance on W3C Staff
>>
>> The checklist above makes the process of going from an
>> experimental technology to a W3C Working Group more clear. The
>> checklist, however, does not alleviate the W3C staffing shortage.
>> In fact, if the way W3C staff is utilized does not also change, it
>> might make the current situation worse.
>>
>> The W3C staff play a very important role in that they help
>> organizations navigate how to get things done via the W3C Process.
>> They help build consensus before and during a Working Group
>> activity at W3C. This can be a double-edged sword. If there is a
>> W3C staff member available to help, they can be a fantastic
>> champion for a group’s work. If there isn’t, and you’ve never
>> progressed work yourself through W3C, you will find yourself in
>> the unfavorable position of not knowing how to proceed. Many of
>> the staff members also have their own way of navigating the W3C
>> process and many of them have to repeat themselves when new work
>> is started. In short, there is a lot of engineering churn and
>> tribal knowledge at W3C; this is true of most standardization
>> bodies.
>>
>> The unfortunate truth is that building out the Web platform is
>> currently being restricted due to the way that we start and
>> progress new work at W3C. The W3C membership relies on the W3C
>> staff to its detriment. The W3C staff are incredibly helpful, but
>> there aren’t enough of them to support building out all of the new
>> functionality needed for the Web, and this is ultimately bad for
>> the Web.
>>
>> In order to address the staffing shortage, it is proposed that we
>> shift much of their work from executing upon the proposed
>> checklist, which is more or less what they do today, to verifying
>> that the checklists are being processed correctly. This is
>> effectively a quality control check on the work that groups are
>> doing as they work their way through the proposed checklist. This
>> offloads a significant chunk of W3C staff time to the groups that
>> want to see their work get traction while giving those groups
>> clear goals to achieve.
>>
>> Blocking Work at W3C
>>
>> W3C members currently have the ability to stop work that has
>> ticked all of the necessary boxes in the checklist mentioned
>> above. At present, large organizations tend to not become involved
>> with work until a vote for the Working Group is circulated by W3C
>> staff. Some members then suggest that they may vote against the
>> work if their viewpoint isn’t taken into account even though they
>> have not participated in the work to date. Some call this a part
>> of the process, but it is uselessly frustrating to those that have
>> spent years building consensus around a Working Group proposal
>> only to have a large W3C member company respond with a “Why wasn’t
>> I consulted” retort.
>>
>> So, now for the controversial bit:
>>
>> A W3C Community Group’s work should automatically transition to a
>> W3C Working Group if a significant coalition of companies, say
>> around 35 of them, have ticked all of the boxes in a predefined
>> W3C Working Group Formation checklist.
>>
>> This checklist should include the creation of two interoperable
>> implementations and a test suite. In other words, it should meet
>> all of the minimum bars for a Working Group’s success per the W3C
>> Process. Making this change achieves the following:
>>
>> 1. It clearly defines how experimental technology progresses to
>> being part of the Web platform so that organizations can
>> allocate resources appropriately.
>> 2. It reduces the workload on W3C staff members, which is a very
>> limited resource.
>> 3. It prevents large organizations from blocking work at W3C
>> while addressing their concerns around technological maturity.
>>
>> In the end, the checklist for starting work at W3C could look
>> something like this:
>>
>> 1. Clearly identify and articulate a problem statement.
>> 2. Create a W3C Community Group.
>> 3. Gather and document use cases.
>> 4. Perform a gap analysis.
>> 5. Produce a proposed architecture and technical specification.
>> 6. Produce two implementations and a test suite.
>> 7. Create a draft charter for a Working Group.
>> 8. Create an executive summary for W3C member companies.
>> 9. Survey organizations and demonstrate that at least 35 of them
>> are supportive of the work and technical direction.
>>
>> Each step above would have document templates that W3C members can
>> use as a starting point. None of the steps above require W3C staff
>> resources and if all steps are completed, the formation of a
>> Working Group is the natural outcome and should not be blocked by
>> the membership.
>>
>> This checklist approach may be seen as too constraining for some,
>> and that’s why it’s voluntary. Some organizations may feel that
>> they do not need to produce all of the work above to get a Working
>> Group and for those organizations, they can choose to ignore the
>> checklist. Initiatives not using the checklist should expect push
>> back if they choose to not answer some of the questions that the
>> checklist covers.
>>
>> Rebalancing How the Web is Built
>>
>> The current process for developing next generation standards for
>> the Web is too unpredictable and too constrained by limited W3C
>> resources. There are groups of small organizations that want to
>> help create these next generation standards; we have to empower
>> those groups with a clear path to standardization. We must not
>> allow organizations to block work if the champions of the work
>> have met the requirements in the checklist. Doing this will free
>> up W3C staffing resources to ensure that the Web Platform is
>> advancing at a natural and rapid rate.
>>
>> A W3C Working Group formation checklist would help make the
>> process more predictable, require less W3C staff time to execute,
>> and provide a smoother path from the inception of an idea, to
>> implementation, to standardization of a technology for the Web
>> platform. This would be great for the Web.
>>
>> References
>>
>> 10. https://manu.sporny.org/2016/browser-api-incubation-antipattern/
>> 11. https://www.w3.org/wiki/Open_Web_Platform
>> 12. https://www.w3.org/2016/08/2016-reorg.html
>> 13.
>>
>> https://docs.google.com/forms/d/e/1FAIpQLScVDxYlBTWchzK3kThdRTn0nulipPy6Hb5qLmRDLOdBtioFfQ/viewform
>> 14. https://www.w3.org/community/credentials/
>> 15.
>>
>> https://docs.google.com/forms/d/e/1FAIpQLSdQJWx8eMRkoTwA2TIWbigTipZ2lqZ_UPFzJbY8IvqmVAHr6A/viewform
>> 16. https://w3c.github.io/webpayments-ig/VCTF/use-cases/
>> 17. http://w3c.github.io/webpayments-ig/VCTF/architecture/
>> 18. http://w3c.github.io/webpayments-ig/VCTF/charter/rc-2.html
>> 19. http://w3c.github.io/webpayments-ig/VCTF/primer/
>> 20. http://w3c.github.io/webpayments-ig/VCTF/support/
>>
>> -- manu
>>
>> --
>> Manu Sporny (skype: msporny, twitter: manusporny, G+: +Manu Sporny)
>> Founder/CEO - Digital Bazaar, Inc.
>> blog: Rebalancing How the Web is Built
>> http://manu.sporny.org/2016/rebalancing/
>>
>>