Re: Rethinking Apps Payments API, Re: Payment App API: updated flow description

There are a lot of words here, but when I look closely, I see a very 
small number of issues here that Marcos is raising. On close inspection, 
it seems to me that these can be fixed in the current spec with fairly 
minor changes. Rather than trying to discuss all of these in this email 
thread, I think we'll make more rapid progress if we break them down 
into separate issues. To that end, I've filed a number of issues on the 
Payment App spec:

 1. Add explicit call to request user permission:
    https://github.com/w3c/webpayments-payment-apps-api/issues/94
 2. Replace setManifest()/getManifest() with
    set()/get()/keys()/has()/delete():
    https://github.com/w3c/webpayments-payment-apps-api/issues/95
 3. Handling of filtering (currently, canHandle()):
    https://github.com/w3c/webpayments-payment-apps-api/issues/96
 4. Different approaches to opening windows:
    https://github.com/w3c/webpayments-payment-apps-api/issues/97
 5. Limit of one-payment-app-per-origin:
    https://github.com/w3c/webpayments-payment-apps-api/issues/98

The one flipping-over-the-apple-cart suggestion in here is an implied 
proposal to remove the vast majority of the browser behavior (e.g. 
shipping address collection), and to simply pass the raw payment request 
object to payment apps unchanged. I believe this reads more squarely on 
the Payment Request spec, since it has several baked-in notions of how 
this is supposed to work. That said, it's not clear how Marcos' proposal 
intends for these things to hang together (e.g., payment matching, 
filtering), so I think we need some level of clarification before I can 
even file an issue that captures it sensible. Marcos: I'm sure you 
understand your objection here better than I do; can you please file an 
issue that captures the core issue?

Thanks.

/a


On 1/23/17 08:48, marcos@marcosc.com wrote:
> Hi Adrian,
> Apologies in advance - these discussion is going to seem heated (and 
> OMG, so much text!). I know I'm advocating for a radical departure - 
> but even if it doesn't happen, I'm hoping we will end up in a better 
> place.
>
> Again, this is not a personal attack on anyone - just a disagreement 
> over architectural choices (specially as they affect us, Mozilla, as a 
> potential implementation). With that said...
>
> (I'm also typing this on my phone, so it's probably full of typos :))
>
> On 23 Jan 2017, at 10:06 pm, Adrian Hope-Bailie <adrian@ripple.com 
> <mailto:adrian@ripple.com>> wrote:
>
>> Hi Marcos,
>>
>> This is an interesting development, although it's not clear to me 
>> what it is trying to solve with the current API or why a complete 
>> re-write is better than working through the issues that remain with 
>> the task force that has been working on them for over a year.
>
> I started actually trying to fix the current spec, but I it easier to 
> just start from scratch.
>
> I found too many issues and things I couldn't reconcile :(
>
>>
>>     * It addresses all the same use cases/requirements.
>>
>>
>> So no different from the current spec?
>
> The current spec does not address the use cases. The purpose of the 
> new spec is to.
>
>>     * It integrates with PaymentRequest - allowing implementation of
>>     `PaymentResponse` in Service Workers.
>>
>>
>> The current API design is intentional. The WG decided that passing 
>> the entire Payment Request to the payment app was a bad idea so this 
>> is a regression.
>>
>> https://github.com/w3c/webpayments-payment-apps-api/issues/2
>
> I think that's a  fundamental design flaw. In fact, we should send 
> PaymentRequest in, IMO. Because an issue is closed, doesn't mean the 
> solution is correct.
>
> I don't want to have lengthy design discussions; I think we should 
> immediately raise this as a TAG issue.
>
> If the TAG disagrees, then I'm happy to back down.
>
>>
>>     * It solves the canHandle() issue.
>>
>>
>> It's not clear what you mean by the "canHandle()" issue. It would be 
>> usefgul if you provided a link to some issue(s) so we can understand 
>> what you're solving.
>
> As discussed below, the spec makes a flawed assumption about what ES 
> functions are and how they work.
>
> If my reading of the spec is correct, then these functions are somehow 
> supposed to survive a browser tab being destroyed (and the js 
> environment disappearing). Is that correct?
>
> If so, we should immediately take this to the TAG or 
> public-script-coord as that would be creating a new function type.
>
> Again, if the spec is introducing a new primitive, then that's likely 
> a serious design flaw: specially if it can be solved in some other way 
> (e.g., using a SW event, as I propose).
>
>>
>> If you are referring to Issue #92 then I think @ianjacobs has done a 
>> pretty good job of summarizing the goals of canHandle() here and I'm 
>> not sure then what the issue is: 
>> https://github.com/w3c/webpayments-payment-apps-api/issues/92#issuecomment-274097944 
>> <https://github.com/w3c/webpayments-payment-apps-api/issues/92#issuecomment-274097944>
>
> I discussed this with Ian over the phone - and it's actually what 
> prompted me to attempt to solve the problem differently.
>
>>     * It solves the identity problem (it's just origin).
>>
>>
>> Again, it's not clear what you mean by "identity problem".
>
> Sorry, I should have been more clear: there are no app types on the 
> web - just features. Hence there can't be "payment apps", only web 
> applications whose origin has been granted the ability to use a power 
> feature.
>
> Hence: the identity problem is not a problem if we accept that this 
> API is just another (powerful) feature of the web.
>
>> Also this is a rather naive solution because Service Workers are not 
>> identified by "just origin" they are identified by their scope URL. 
>> So are you proposing that Payment Apps are special kind of Service 
>> Worker for which only 1 can be registered per origin?
>
> No - the use of the feature is bound to the SW registration. As per 
> any other feature that relies on SWs.
>
> What I was trying to say, with respect to identity, is that there are 
> no "payment apps": just website that use this feature.
>
>>     * It solves the "manifest" problem (use it if it's there!) - and
>>     doesn't require trashing everything any time there is an update.
>>
>>
>> Again, not sure what you mean by "manifest problem". Manifests, in 
>> the context of payment apps, serve a simple purpose (similar to the 
>> purpose of app manifest) which is to provide icons, labels and 
>> descriptions for an app so that a browser can present a user with UI 
>> for selecting an app from a list.
>
> I argue that this is not the concern of the specification and should 
> be removed. I removed it in my proposal.
>
>>
>> Unfortunately the spec has used the term "manifest" despite the 
>> implementation being inconsitent with "app manifest" (the current 
>> proposal is to pass the "manifest" as a Javascript object into the 
>> registration flow of the Service Worker as opposed to storing it as a 
>> JSON file) so it has caused some confusion.
>
> Clearly - which again is why I removed that in my proposal. That's 
> handled by HTML and Web Manifest. Not this spec's concern.
>
>>
>> It's important to note however that the manifest is only required to 
>> support the recommended apps functionality so if we drop recommended 
>> apps, which is gaining support, then there is no "manifest problem" 
>> because there is no manifest.
>
> Exactly, but the whole API in the current spec is:
>
> .getManifest() and .setManifest()
>
> So you still have "the manifest problem" in that the front-end of this 
> API only had two method calls that focus on a manifest object. You 
> will need something new, which I propose in my API.
>
>>     * it provides a secure way to open a payment flow window.
>>
>>
>> It's not clear to me how the proposal is different from what is being 
>> discussed in the Payment Apps TF 
>> (https://github.com/w3c/webpayments-payment-apps-api/issues/73). The 
>> current spec will provide the same functionality but the group is 
>> taking the time to figure out the best way to do this while being 
>> cognizant of the different form factors and UX that this involves.
>
> UX outside the scope of standardization. All we can standardize is 
> that the browser needs to open a payment flow - let browser implement 
> the rest.
>
> That's what Im proposing. We got this.
>
>>     * it doesn't magically fill in stuff (e.g., shipping address) -
>>     giving the payment handler more control over the response, and
>>     providing the ability to create much richer payment handling web
>>     applications.
>>
>>
>> Again, not passing the "information request" aspects of a Payment 
>> Request to the Payment App was a decision taken by the group so the 
>> current design is intentional.
>
> Again - I think this is an architect mistake and think we should take 
> this to the TAG immediately.
>
> It's completely going against the extensible web manifesto. We should 
> absolutely be letting developers set that information.
>
>>
>> Only the mediator/browser interacts with the web page to provide user 
>> details. Payment Apps are exclusively for handling the payment.
>>
>> https://github.com/w3c/webpayments-payment-apps-api/issues/13
>
> Again, this is a serious design flaw IMO. We should take this to the TAG.
>
>>
>>     The above is just the API shape (no algorithms yet), but it shows
>>     how it is expected to work with the code examples.
>>
>>
>> Unfortunately the devil is in the details and as I have said above 
>> you are regressing on a number of design decisions that the WG has 
>> already taken so if your goal is to simply tweak the API shape I'd 
>> suggest doing so by suggesting changes to the existing spec.
>
> The spec is not even FPWD - so it's completely unfair to say I'm 
> "regressing" anything. Specs are there for comment, criticism, and 
> revision (this group should know that, given it's history). It's 
> definitely not uncommon for multiple proposals to come in or for work 
> to restart.
>
> It's healthy to have competing proposals - specially early on.
>
>
>>
>>     ## The long version
>>     Firstly, I must acknowledge that the current API (and surrounding
>>     discussions) have done a great job at formulating the use cases
>>     and requirements for this API. Thus it serves as an important
>>     foundation to build on.
>>
>>     However, the current Payment API does not meet the use cases
>>     (listed below), 
>>
>>
>> It would be helpful if you could explain why you think this is the case
>>
>>     it is impossible to implement, 
>>
>>
>> Again, some detail on why you think this is the case because we 
>> already have working implementations so "impossible" seems like a 
>> strong word here.
>>
>>     it's obsessed with the idea of "manifests", 
>>
>>
>> I suspect you have hopped onto the back of a very recent discussion 
>> which is attempting to solve for a requirement that many of the 
>> merchants in the group feel strongly about; recommended apps.
>>
>> There is a desire to allow merchants to recommend apps to their 
>> customers as a way to bootstrap the ecosystem. To do this it is 
>> necessary for the browser to present the user with a list of one or 
>> more apps that are not yet installed therefor the browser must source 
>> the necessary meta-data (an icon, label etc) about the app from 
>> somewhere. One of the proposals is a manifest.
>>
>>     and has become bogged down in issues that are clearly outside
>>     it's scope: like, trying to change the web security model, issues
>>     of application identity, etc). 
>>
>>
>>     These are tremendously important problems, but they are outside
>>     the scope of this work. Those are issues that we need to address
>>     universally in the web platform, but they are not issues that
>>     this working group can solve in isolation.
>>
>>
>> Can you point at some issues where this is happening. On the contrary 
>> I think the group is at pains to try and adhere to the existing 
>> models and re-use what is available as much as possible.
>>
>>
>>     ## Let's start with use cases and requirements
>>
>>     The fundamental *use cases* are:
>>
>>     1. Register an web app, with user consent, to handle requests for
>>     payment of particular types (e.g., credit cards, a proprietary
>>     payment method).
>>     2. Allow the end-user to manage payment methods (adding,
>>     deleting, modifying, etc.) - and for these methods to be
>>     available as payment options upon request for payment. This
>>     includes providing both visual and accessible representations of
>>     those payment methods (i.e., icons and text).
>>     3. Allow merchants to query if those payment methods are
>>     available at point of sale.
>>
>>     The *requirements** are thus:
>>
>>     R1. Registration: Provide a means for a web application to get
>>     user consent to act as a payment handler (a "powerful feature",
>>     amongst other powerful features).
>>
>>     R2. Payment Method Management: Where there are multiple payment
>>     methods, provide a means to allow the end-user to manage payment
>>     methods.
>>
>>     R3. Payment Handling: handle "request" for payment, and
>>     facilitate communication between a `PaymentRequest`, the end-user
>>     (through a payment UI), and a `PaymentResponse` on a third party
>>     site.
>>
>>     R4. UI to make payment: provide a means for a web application to
>>     spawn a window-like context in which to perform a payment method.
>>
>>     R5. Can handle: Provide a means to check if a payment handler can
>>     handle a payment.
>>
>>     Now we can look at how the current API is deficient in addressing
>>     these use cases - and how the new API can address the requirements.
>>
>>     ## R1 - Registration
>>     The current API conflates web app manifest and payment manifest,
>>     etc. In relation to other powerful features afforded by the web
>>     platforms there is nothing special about the payments response
>>     API that requires it to have a manifest.
>>
>>
>> I think you are getting a bit obsessed with the manifest topic.
>
> The API is literally just .get/setManifest()! I honestly don't know 
> how else to view it?
>
>
>> Nobody is proposing that the API have a manifest but the current spec 
>> proposes that when a payment app is registered it provides some 
>> information about itself to the PaymentAppManager (icon, label etc) 
>> which are often stored in an app manifest.
>>
>> As I have said to you numerous times the goals is to try and re-use 
>> some of what is already defined for the Web platform (eg: the data 
>> structures in app manifest for describing icons) so that this WG 
>> doesn't try to reinvent everything.
>>
>> If you read issue #48 you'll see that there is not a final decision 
>> on how a payment app (during it's registration flow) will provide the 
>> icons, label and other meta data it must provide to the browser. The 
>> current proposal is a Javascript object passed to the registration 
>> function. This is called a manifest which is clearly creating some 
>> confusion, although it must be said "app manifest" didn't invent that 
>> word, it's a pretty well understood term for application meta-data so 
>> using it here is appropriate.
>
> Im arguing that the current manifest thing and related objects- as 
> well as the API (.set/getManifest()) is unsuitable for managing 
> payment methods.
>
> What I propose is more flexible.
>
>>
>>     How Payment Requests API solves this:
>>     Having a web manifest is not a requirement for "payment
>>     handlers". However, a web application can include a web manifest.
>>
>>     To get permission to act as a payment handler, the new API allows
>>     the developer to ask for permission via the a call to:
>>     paymentAppManager.requestPermission()
>>
>>     This allows the browser to intervene and negotiate with the user
>>     if the application can act as a payment manger.
>>
>>
>> Hooking handlePayment into the permissions framework is a good idea 
>> but not a new one. The existing registration algorithm already 
>> proposes that the user be asked to grant permission to the app.
>>
>> We could certainly use some help to ensure this language is improved 
>> to make it clear that this is intended to be part of the existing 
>> browser permissions framework.
>
> Or, we can use the new spec.
>
>>
>>     ## R2 - Payment Method Management
>>
>>     The current API only has two methods for handling payment methods
>>     - and those are conflated with "manifests":
>>
>>      * setManifest()
>>      * getManifest()
>>
>>     This makes it impossible to manage payment methods without
>>     continuously trashing the existing payment methods and "the
>>     manifest". This makes it difficult to get at current payment
>>     methods, and modify and work with them individually. 
>>
>>
>>     How the new API solves this:
>>     The new API introduces a `PaymentMethodMap`, which allows for
>>     asynchronous atomic management of payment methods (add, delete,
>>     iterate, etc). Exposed on window and workers.
>>
>>
>> What this ignores is that each payment method is a new permission 
>> that the user should consent to granting the app.
>> If I have given an app permission to make a certain type of payment 
>> then I don't expect it to be able to make other kinds of payments 
>> because it silently added supported payment methods in the background.
>
> This is another design flaw. You can still have atomic management and 
> give users control over the payment methods.
>
>>
>> Again, while you may not think the API is very developer friendly 
>> it's design is intentional. It may also not be perfect but there are 
>> some design considerations that you are ignoring.
>
> I'm not ignoring them - I'm disagreeing with them and trying to fix 
> them by providing alternatives.
>
>>
>>     ## R3. Payment Handling
>>     The existing API fails to capture all the possible events sent
>>     from a PaymentRequest object. 
>>
>>
>> As discussed above this is by design. Those events are only supposed 
>> to be for the sake of the website and browser.
>
> This is again a serious design and architectural flaw. I urge us to 
> discuss with the TAG immediately.
>
>>     It then goes on to introduce new primitives instead of just using
>>     PaymentResponse. As such it fails to integrate fully with
>>     `PaymentRequest`, and fails to allow bi-directional communication
>>     between PaymentRequest and PaymentResponse.
>>
>>
>> As discussed above this is by design.
>
> This is again a serious design and architectural flaw. I urge us to 
> discuss with the TAG immediately.
>
>
>>
>> At the time of designing Payment Request we made the decision we 
>> wanted payment apps to be implementable in any form (web-based, 
>> native etc) and for the interface between the website and browser to 
>> be the same.
>
> Then why are they different?
>
>>
>> This is a compromise but was made intentionally. The result is that 
>> there is a bidirectional interaction between the website and browser 
>> initially and then a single request passed beyond the browser and 
>> single response that comes back.
>
>> It may be that we need to reconsider this approach but I'd want to 
>> hear from implementors that are planning to support native payment 
>> apps (like Android) if this is something they'd support.
>
> I'm an implementer representing a little browser called Firefox. I 
> have serious concern about the design.
>
> I've also spoken to the other browser vendors, and they share many of 
> my concerns.
>
> Mozilla thinks the API is REALLY important, so we'd like to see the 
> best API possible - even if that means "regressing" a few old closed 
> issues.
>
>> There are issues with background processing on low spec devices to 
>> consider here, it's not all simple.
>
> I'm well aware if those. It's a real risk till we are able to 
> prototype it.
>
>>
>>
>>     The new API solves this by giving PaymentResponse a constructor -
>>     allowing ev.respondWith() to actually respond with a
>>     PaymentResponse and by allowing the "onpaymentrequest" event to
>>     catch the various events resulting from calls to PaymentResponse
>>     - for instance:
>>
>>
>> As discussed this is also by design.
>>
> Again, I think this a design flaw.
>
>
>> The group made the decision that the payment app should not be passed 
>> the complete payment request but rather a modified trimmed down 
>> version with only the data relevant to it.
>
> Again, I think this is a poor decision of the part of the group and we 
> need to reconsider.
>
>>
>>      * .onabort() can now be handled.
>>      * Response.updateShippingAddress() - can notify the
>>     PaymentRequest with a new address, and await a response from the
>>     PaymentRequest to check if it's ok.
>>      * Response.updateShippingOption() - as above, but with a new
>>     shipping option id.
>>
>>
>> Sourcing shipping and email data from the app has never been a goal 
>> of this WG.
>
> As above. It should absolutely be.
>
>>     ## R4. UI to make payment
>>     The current API is hand-wavy about using clients.openWindow() in
>>     a service worker to perform the payment flow. This solution is
>>     inadequate (or simply not any better) when compared to what
>>     browser do today.
>>
>>
>> I wouldn't call it hand-wavy, I'd say it is still being discussed. 
>> It's likely the final solution will be just as you describe but we're 
>> getting there by considering all the implications, form factors etc
>
> This is not the concern of the group. It's browser UI: leave that to 
> us (browser vendors).
>
>>
>>     The new API introduces a the ability to open a payment-specific
>>     payment flow window via `PaymentRequestEvent.openClientWindow()`.
>>     This allows the browser to bring up an appropriate payment window
>>     for the purpose of negotiating a request for payment.
>>
>>     ## R5. Can handle payment
>>     The current API tries to define `canHandle` the the manifest - by
>>     defining a new type of function that appears to be transferrable
>>     across threads. That is: if the payment app is closed, the
>>     function is somehow magically supposed to still be active (!).
>>     This is impossible to implement, because functions are dynamic
>>     and associated with their global environment (and contain
>>     closures). Hence, it's not something that can be implemented -
>>     not without changing how JavaScript works, which is just not
>>     going to happen.
>>
>>     The new API addresses this by adding a "canhandlepayment" event
>>     on the service worker instead. Registered handlers can then be
>>     woken to check if the payment method is supported - by performing
>>     whatever checks are necessary.
>>
>>
>> I am no expert on the internals of browsers or their threading models 
>> and how they allocate processing to JS or sandbox and GC these processes.
>>
>> This sounds like a good suggested change, perhaps you could propose 
>> it against the existing API?
>
> Again, I could, but the current spec has so many serious issues that I 
> feel it would be quicker for us to take the good bits (of which there 
> are many!) from the old spec and transplant them to the new one.
>
> Thanks again for hearing me out. Again, I personally hate these kind 
> of debate emails - they are eeek and polarizing, and for that I'm 
> sorry (and please don't take anything I say personally or with 
> offence, I have pretty bad interpersonal skills - specially over 
> email). It's also why I want us to resolve some of this through the 
> TAG: they can act as a neutral mediator whose opinion we all respect.
>
> Kind regards,
> Marcos
>
>>
>>     ===
>>
>>     Obviously, people will have a lot of questions. Let's work
>>     through those!
>>
>>     There is lots of stuff we can take directly from the existing API
>>     too - we should totally do that where it makes sense.
>>
>>
>> I think you have designed the new API lacking a lot of context and 
>> background discussion and design decisions. There are some good ideas 
>> but I don't think starting again is a a good idea at all.
>>
>> If you are able to join the weekly Payment App TF calls it would be 
>> great to look at some of the ideas in here as possible additions or 
>> edits to the existing spec.
>>
>> If the group feels we should re-open some previous discussion like 
>> allowing a bi-directional channel between apps and the calling 
>> website or allowing apps to participate in providing the non-payment 
>> related (shipping, email etc) data then I'd like to hear proposals to 
>> do that and why before we jump straight to a whole new technical 
>> proposal.
>>
>>
>>     Kind regards,
>>     Marcos
>>
>>
>>
>>
>>


-- 
Adam Roach
Principal Engineer, Mozilla