Re: Rethinking Apps Payments API, Re: Payment App API: updated flow description

Marcos,

Thanks for your response. Don't worry I am not taking anything personally.
I personally agree with a lot of what you have said and disagree with some
of the decisions taken by the group in the past that you consider mistakes.
That is they way things unfold sometimes in a WG, you won't always agree
with the final decision of the group.

This is me with my chair hat on, as opposed to editor of the current spec.
I don't want us reopening old issues unless there is a compelling reason to
do so and I don't think simply proposing a whole new spec without directly
addressing the previous decisions first is the way to do this.

It's useful to maybe provide you with some context here.

This spec was largely ignored by the vendors for the first year that the WG
existed and the split between payment request and payment apps was by
design of the editors of Payment Request such that a vendor could implement
Payment Request and not Payment Apps. This was a very contentious issue.

Put differently, the goal of the PR editors was a payment request API that
a website could invoke with no knowledge of how the request would be
processed (by the browser just returning a PAN, a web-based app interacting
with some third-party web-service or even a native mobile app processing
the request) and getting a standardized response. I would go so far as to
say the PR spec editors didn't, until very recently, even acknowledge that
payment apps would ever even exist.

So, as a compromise, the entire architecture was designed on the following
assumptions:

1. The interaction pattern was a simple request response (no chattiness
between website and payment app allowed)
2. All data required to process the payment must be passed in the request
up front.

I would love us to throw those assumptions away as it would fundamentally
change what we can do with Payment Apps and Payment Request but I think we
need to be sure that the editors and implementors of Payment Request are
also happy with what the implications of this on their spec would be.

WRT to the TAG the WG has tried on numerous occasions to engage the TAG on
some of these big decisions but have had little feedback or help to date so
we rely on subject matter experts like Jake and yourself to step in when
things have matured and you can spare the bandwidth.

The WG, and it's deliverables, have matured somewhat since those early days
so perhaps there is scope to put these issue to the TAG again?

My only ask is that before you invest significant time in a new spec that
you engage the current editors on the big issues, as I think you'll find
them keen to solve the same problems as you but they also have over a year
of background that perhaps you don't.

Adrian

On Mon, Jan 23, 2017 at 4:48 PM, <marcos@marcosc.com> wrote:

> Hi Adrian,
> Apologies in advance - these discussion is going to seem heated (and OMG,
> so much text!). I know I'm advocating for a radical departure - but even if
> it doesn't happen, I'm hoping we will end up in a better place.
>
> Again, this is not a personal attack on anyone - just a disagreement over
> architectural choices (specially as they affect us, Mozilla, as a potential
> implementation). With that said...
>
> (I'm also typing this on my phone, so it's probably full of typos :))
>
> On 23 Jan 2017, at 10:06 pm, Adrian Hope-Bailie <adrian@ripple.com> wrote:
>
> Hi Marcos,
>
> This is an interesting development, although it's not clear to me what it
> is trying to solve with the current API or why a complete re-write is
> better than working through the issues that remain with the task force that
> has been working on them for over a year.
>
>
> I started actually trying to fix the current spec, but I it easier to just
> start from scratch.
>
> I found too many issues and things I couldn't reconcile :(
>
>
> * It addresses all the same use cases/requirements.
>>
>
> So no different from the current spec?
>
>
> The current spec does not address the use cases. The purpose of the new
> spec is to.
>
>
>
>> * It integrates with PaymentRequest - allowing implementation of
>> `PaymentResponse` in Service Workers.
>>
>
> The current API design is intentional. The WG decided that passing the
> entire Payment Request to the payment app was a bad idea so this is a
> regression.
>
>
> https://github.com/w3c/webpayments-payment-apps-api/issues/2
>
>
> I think that's a  fundamental design flaw. In fact, we should send
> PaymentRequest in, IMO. Because an issue is closed, doesn't mean the
> solution is correct.
>
> I don't want to have lengthy design discussions; I think we should
> immediately raise this as a TAG issue.
>
> If the TAG disagrees, then I'm happy to back down.
>
>
>
>> * It solves the canHandle() issue.
>>
>
> It's not clear what you mean by the "canHandle()" issue. It would be
> usefgul if you provided a link to some issue(s) so we can understand what
> you're solving.
>
>
> As discussed below, the spec makes a flawed assumption about what ES
> functions are and how they work.
>
> If my reading of the spec is correct, then these functions are somehow
> supposed to survive a browser tab being destroyed (and the js environment
> disappearing). Is that correct?
>
> If so, we should immediately take this to the TAG or public-script-coord
> as that would be creating a new function type.
>
> Again, if the spec is introducing a new primitive, then that's likely a
> serious design flaw: specially if it can be solved in some other way (e.g.,
> using a SW event, as I propose).
>
>
> If you are referring to Issue #92 then I think @ianjacobs has done a
> pretty good job of summarizing the goals of canHandle() here and I'm not
> sure then what the issue is: https://github.com/w3c/web
> payments-payment-apps-api/issues/92#issuecomment-274097944
>
>
> I discussed this with Ian over the phone - and it's actually what prompted
> me to attempt to solve the problem differently.
>
>
>
>> * It solves the identity problem (it's just origin).
>>
>
> Again, it's not clear what you mean by "identity problem".
>
>
> Sorry, I should have been more clear: there are no app types on the web -
> just features. Hence there can't be "payment apps", only web applications
> whose origin has been granted the ability to use a power feature.
>
> Hence: the identity problem is not a problem if we accept that this API is
> just another (powerful) feature of the web.
>
> Also this is a rather naive solution because Service Workers are not
> identified by "just origin" they are identified by their scope URL. So are
> you proposing that Payment Apps are special kind of Service Worker for
> which only 1 can be registered per origin?
>
>
> No - the use of the feature is bound to the SW registration. As per any
> other feature that relies on SWs.
>
> What I was trying to say, with respect to identity, is that there are no
> "payment apps": just website that use this feature.
>
>
>
>> * It solves the "manifest" problem (use it if it's there!) - and doesn't
>> require trashing everything any time there is an update.
>>
>
> Again, not sure what you mean by "manifest problem". Manifests, in the
> context of payment apps, serve a simple purpose (similar to the purpose of
> app manifest) which is to provide icons, labels and descriptions for an app
> so that a browser can present a user with UI for selecting an app from a
> list.
>
>
> I argue that this is not the concern of the specification and should be
> removed. I removed it in my proposal.
>
>
> Unfortunately the spec has used the term "manifest" despite the
> implementation being inconsitent with "app manifest" (the current proposal
> is to pass the "manifest" as a Javascript object into the registration flow
> of the Service Worker as opposed to storing it as a JSON file) so it has
> caused some confusion.
>
>
> Clearly - which again is why I removed that in my proposal. That's handled
> by HTML and Web Manifest. Not this spec's concern.
>
>
> It's important to note however that the manifest is only required to
> support the recommended apps functionality so if we drop recommended apps,
> which is gaining support, then there is no "manifest problem" because there
> is no manifest.
>
>
> Exactly, but the whole API in the current spec is:
>
> .getManifest() and .setManifest()
>
> So you still have "the manifest problem" in that the front-end of this API
> only had two method calls that focus on a manifest object. You will need
> something new, which I propose in my API.
>
>
>
>> * it provides a secure way to open a payment flow window.
>>
>
> It's not clear to me how the proposal is different from what is being
> discussed in the Payment Apps TF (https://github.com/w3c/
> webpayments-payment-apps-api/issues/73). The current spec will provide
> the same functionality but the group is taking the time to figure out the
> best way to do this while being cognizant of the different form factors and
> UX that this involves.
>
>
> UX outside the scope of standardization. All we can standardize is that
> the browser needs to open a payment flow - let browser implement the rest.
>
> That's what Im proposing. We got this.
>
>
>
>> * it doesn't magically fill in stuff (e.g., shipping address) - giving
>> the payment handler more control over the response, and providing the
>> ability to create much richer payment handling web applications.
>>
>
> Again, not passing the "information request" aspects of a Payment Request
> to the Payment App was a decision taken by the group so the current design
> is intentional.
>
>
> Again - I think this is an architect mistake and think we should take this
> to the TAG immediately.
>
> It's completely going against the extensible web manifesto. We should
> absolutely be letting developers set that information.
>
>
> Only the mediator/browser interacts with the web page to provide user
> details. Payment Apps are exclusively for handling the payment.
>
> https://github.com/w3c/webpayments-payment-apps-api/issues/13
>
>
> Again, this is a serious design flaw IMO. We should take this to the TAG.
>
>
>
>>
>> The above is just the API shape (no algorithms yet), but it shows how it
>> is expected to work with the code examples.
>>
>
> Unfortunately the devil is in the details and as I have said above you are
> regressing on a number of design decisions that the WG has already taken so
> if your goal is to simply tweak the API shape I'd suggest doing so by
> suggesting changes to the existing spec.
>
>
> The spec is not even FPWD - so it's completely unfair to say I'm
> "regressing" anything. Specs are there for comment, criticism, and revision
> (this group should know that, given it's history). It's definitely not
> uncommon for multiple proposals to come in or for work to restart.
>
> It's healthy to have competing proposals - specially early on.
>
>
>
>> ## The long version
>> Firstly, I must acknowledge that the current API (and surrounding
>> discussions) have done a great job at formulating the use cases and
>> requirements for this API. Thus it serves as an important foundation to
>> build on.
>>
>> However, the current Payment API does not meet the use cases (listed
>> below),
>
>
> It would be helpful if you could explain why you think this is the case
>
>
>> it is impossible to implement,
>
>
> Again, some detail on why you think this is the case because we already
> have working implementations so "impossible" seems like a strong word here.
>
>
>> it's obsessed with the idea of "manifests",
>
>
> I suspect you have hopped onto the back of a very recent discussion which
> is attempting to solve for a requirement that many of the merchants in the
> group feel strongly about; recommended apps.
>
> There is a desire to allow merchants to recommend apps to their customers
> as a way to bootstrap the ecosystem. To do this it is necessary for the
> browser to present the user with a list of one or more apps that are not
> yet installed therefor the browser must source the necessary meta-data (an
> icon, label etc) about the app from somewhere. One of the proposals is a
> manifest.
>
>
>> and has become bogged down in issues that are clearly outside it's scope:
>> like, trying to change the web security model, issues of application
>> identity, etc).
>
>
>> These are tremendously important problems, but they are outside the scope
>> of this work. Those are issues that we need to address universally in the
>> web platform, but they are not issues that this working group can solve in
>> isolation.
>>
>
> Can you point at some issues where this is happening. On the contrary I
> think the group is at pains to try and adhere to the existing models and
> re-use what is available as much as possible.
>
>
>>
>> ## Let's start with use cases and requirements
>>
>> The fundamental *use cases* are:
>>
>> 1. Register an web app, with user consent, to handle requests for payment
>> of particular types (e.g., credit cards, a proprietary payment method).
>> 2. Allow the end-user to manage payment methods (adding, deleting,
>> modifying, etc.) - and for these methods to be available as payment options
>> upon request for payment. This includes providing both visual and
>> accessible representations of those payment methods (i.e., icons and text).
>> 3. Allow merchants to query if those payment methods are available at
>> point of sale.
>>
>> The *requirements** are thus:
>>
>> R1. Registration: Provide a means for a web application to get user
>> consent to act as a payment handler (a "powerful feature", amongst other
>> powerful features).
>>
>> R2. Payment Method Management: Where there are multiple payment methods,
>> provide a means to allow the end-user to manage payment methods.
>>
>> R3. Payment Handling: handle "request" for payment, and facilitate
>> communication between a `PaymentRequest`, the end-user (through a payment
>> UI), and a `PaymentResponse` on a third party site.
>>
>> R4. UI to make payment: provide a means for a web application to spawn a
>> window-like context in which to perform a payment method.
>>
>> R5. Can handle: Provide a means to check if a payment handler can handle
>> a payment.
>>
>> Now we can look at how the current API is deficient in addressing these
>> use cases - and how the new API can address the requirements.
>>
>> ## R1 - Registration
>> The current API conflates web app manifest and payment manifest, etc. In
>> relation to other powerful features afforded by the web platforms there is
>> nothing special about the payments response API that requires it to have a
>> manifest.
>>
>
> I think you are getting a bit obsessed with the manifest topic.
>
>
> The API is literally just .get/setManifest()! I honestly don't know how
> else to view it?
>
>
> Nobody is proposing that the API have a manifest but the current spec
> proposes that when a payment app is registered it provides some information
> about itself to the PaymentAppManager (icon, label etc) which are often
> stored in an app manifest.
>
> As I have said to you numerous times the goals is to try and re-use some
> of what is already defined for the Web platform (eg: the data structures in
> app manifest for describing icons) so that this WG doesn't try to reinvent
> everything.
>
> If you read issue #48 you'll see that there is not a final decision on how
> a payment app (during it's registration flow) will provide the icons, label
> and other meta data it must provide to the browser. The current proposal is
> a Javascript object passed to the registration function. This is called a
> manifest which is clearly creating some confusion, although it must be said
> "app manifest" didn't invent that word, it's a pretty well understood term
> for application meta-data so using it here is appropriate.
>
>
> Im arguing that the current manifest thing and related objects- as well as
> the API (.set/getManifest()) is unsuitable for managing payment methods.
>
> What I propose is more flexible.
>
>
>> How Payment Requests API solves this:
>> Having a web manifest is not a requirement for "payment handlers".
>> However, a web application can include a web manifest.
>>
>> To get permission to act as a payment handler, the new API allows the
>> developer to ask for permission via the a call to:
>> paymentAppManager.requestPermission()
>>
>> This allows the browser to intervene and negotiate with the user if the
>> application can act as a payment manger.
>>
>
> Hooking handlePayment into the permissions framework is a good idea but
> not a new one. The existing registration algorithm already proposes that
> the user be asked to grant permission to the app.
>
> We could certainly use some help to ensure this language is improved to
> make it clear that this is intended to be part of the existing browser
> permissions framework.
>
>
> Or, we can use the new spec.
>
>
>
>>
>> ## R2 - Payment Method Management
>>
>> The current API only has two methods for handling payment methods - and
>> those are conflated with "manifests":
>>
>>  * setManifest()
>>  * getManifest()
>>
>> This makes it impossible to manage payment methods without continuously
>> trashing the existing payment methods and "the manifest". This makes it
>> difficult to get at current payment methods, and modify and work with them
>> individually.
>
>
>> How the new API solves this:
>> The new API introduces a `PaymentMethodMap`, which allows for
>> asynchronous atomic management of payment methods (add, delete, iterate,
>> etc). Exposed on window and workers.
>>
>
> What this ignores is that each payment method is a new permission that the
> user should consent to granting the app.
>
> If I have given an app permission to make a certain type of payment then I
> don't expect it to be able to make other kinds of payments because it
> silently added supported payment methods in the background.
>
>
> This is another design flaw. You can still have atomic management and give
> users control over the payment methods.
>
>
> Again, while you may not think the API is very developer friendly it's
> design is intentional. It may also not be perfect but there are some design
> considerations that you are ignoring.
>
>
> I'm not ignoring them - I'm disagreeing with them and trying to fix them
> by providing alternatives.
>
>
>> ## R3. Payment Handling
>> The existing API fails to capture all the possible events sent from a
>> PaymentRequest object.
>
>
> As discussed above this is by design. Those events are only supposed to be
> for the sake of the website and browser.
>
>
> This is again a serious design and architectural flaw. I urge us to
> discuss with the TAG immediately.
>
>
>
>> It then goes on to introduce new primitives instead of just using
>> PaymentResponse. As such it fails to integrate fully with `PaymentRequest`,
>> and fails to allow bi-directional communication between PaymentRequest and
>> PaymentResponse.
>>
>
> As discussed above this is by design.
>
>
> This is again a serious design and architectural flaw. I urge us to
> discuss with the TAG immediately.
>
>
>
> At the time of designing Payment Request we made the decision we wanted
> payment apps to be implementable in any form (web-based, native etc) and
> for the interface between the website and browser to be the same.
>
>
> Then why are they different?
>
>
> This is a compromise but was made intentionally. The result is that there
> is a bidirectional interaction between the website and browser initially
> and then a single request passed beyond the browser and single response
> that comes back.
>
>
> It may be that we need to reconsider this approach but I'd want to hear
> from implementors that are planning to support native payment apps (like
> Android) if this is something they'd support.
>
>
> I'm an implementer representing a little browser called Firefox. I have
> serious concern about the design.
>
> I've also spoken to the other browser vendors, and they share many of my
> concerns.
>
> Mozilla thinks the API is REALLY important, so we'd like to see the best
> API possible - even if that means "regressing" a few old closed issues.
>
> There are issues with background processing on low spec devices to
> consider here, it's not all simple.
>
>
> I'm well aware if those. It's a real risk till we are able to prototype
> it.
>
>
>
>>
>> The new API solves this by giving PaymentResponse a constructor -
>> allowing ev.respondWith() to actually respond with a PaymentResponse and by
>> allowing the "onpaymentrequest" event to catch the various events resulting
>> from calls to PaymentResponse - for instance:
>>
>
> As discussed this is also by design.
>
> Again, I think this a design flaw.
>
>
> The group made the decision that the payment app should not be passed the
> complete payment request but rather a modified trimmed down version with
> only the data relevant to it.
>
>
> Again, I think this is a poor decision of the part of the group and we
> need to reconsider.
>
>
>
>>
>>  * .onabort() can now be handled.
>>  * Response.updateShippingAddress() - can notify the PaymentRequest with
>> a new address, and await a response from the PaymentRequest to check if
>> it's ok.
>>  * Response.updateShippingOption() - as above, but with a new shipping
>> option id.
>>
>>
> Sourcing shipping and email data from the app has never been a goal of
> this WG.
>
>
> As above. It should absolutely be.
>
>
>
>> ## R4. UI to make payment
>> The current API is hand-wavy about using clients.openWindow() in a
>> service worker to perform the payment flow. This solution is inadequate (or
>> simply not any better) when compared to what browser do today.
>>
>
> I wouldn't call it hand-wavy, I'd say it is still being discussed. It's
> likely the final solution will be just as you describe but we're getting
> there by considering all the implications, form factors etc
>
>
> This is not the concern of the group. It's browser UI: leave that to us
> (browser vendors).
>
>
>
>>
>> The new API introduces a the ability to open a payment-specific payment
>> flow window via `PaymentRequestEvent.openClientWindow()`. This allows
>> the browser to bring up an appropriate payment window for the purpose of
>> negotiating a request for payment.
>>
>> ## R5. Can handle payment
>> The current API tries to define `canHandle` the the manifest - by
>> defining a new type of function that appears to be transferrable across
>> threads. That is: if the payment app is closed, the function is somehow
>> magically supposed to still be active (!). This is impossible to implement,
>> because functions are dynamic and associated with their global environment
>> (and contain closures). Hence, it's not something that can be implemented -
>> not without changing how JavaScript works, which is just not going to
>> happen.
>>
>> The new API addresses this by adding a "canhandlepayment" event on the
>> service worker instead. Registered handlers can then be woken to check if
>> the payment method is supported - by performing whatever checks are
>> necessary.
>>
>
> I am no expert on the internals of browsers or their threading models and
> how they allocate processing to JS or sandbox and GC these processes.
>
> This sounds like a good suggested change, perhaps you could propose it
> against the existing API?
>
>
> Again, I could, but the current spec has so many serious issues that I
> feel it would be quicker for us to take the good bits (of which there are
> many!) from the old spec and transplant them to the new one.
>
> Thanks again for hearing me out. Again, I personally hate these kind of
> debate emails - they are eeek and polarizing, and for that I'm sorry (and
> please don't take anything I say personally or with offence, I have pretty
> bad interpersonal skills - specially over email). It's also why I want us
> to resolve some of this through the TAG: they can act as a neutral mediator
> whose opinion we all respect.
>
> Kind regards,
> Marcos
>
>
>> ===
>>
>> Obviously, people will have a lot of questions. Let's work through those!
>>
>> There is lots of stuff we can take directly from the existing API too -
>> we should totally do that where it makes sense.
>>
>
> I think you have designed the new API lacking a lot of context and
> background discussion and design decisions. There are some good ideas but I
> don't think starting again is a a good idea at all.
>
> If you are able to join the weekly Payment App TF calls it would be great
> to look at some of the ideas in here as possible additions or edits to the
> existing spec.
>
> If the group feels we should re-open some previous discussion like
> allowing a bi-directional channel between apps and the calling website or
> allowing apps to participate in providing the non-payment related
> (shipping, email etc) data then I'd like to hear proposals to do that and
> why before we jump straight to a whole new technical proposal.
>
>
>> Kind regards,
>> Marcos
>>
>>
>>
>>
>>
>