Re: HTTP Resumable Uploads on Apple Platforms

Hi Jonathan,

I played around with the example code using SwiftNIO and wanted to provide
some feedback. First of all, it follows a very interesting and different
approach than other resumable upload servers. In the past, I have mostly
seen resumable upload servers implemented as additional services, which are
separate from the main backend services. Once a resumable upload is
complete, the main service will be notified about the complete upload using
some custom communication channel. However, your example takes a different
approach by wrapping the main backend logic allowing every request to be
turned into a resumable upload, while being completely transparent to the
main backend logic. It is going to be interesting to see what the benefits
and drawbacks of these two approaches will be over time.

You can find my example code in
https://github.com/tus/draft-example/tree/main/servers/swift-nio, where I
try to collect different implementations of the resumable upload draft.
It's my first exposure to SwiftNIO, so it might be quite rough around the
edges. Nevertheless, there are a few points worth mentioning:

   1. If the offsets in the Upload Appending Produce do not match the
   server's expectation, a 409 Conflict is returned and the entire resumable
   upload is cancelled. Subsequent requests to the upload URL result in a 404
   Not Found. While this is not prohibited by the draft, I am wondering if
   it's better to keep the upload around and give the client a second chance
   to resume the upload.
   2. The decision whether an upload is complete seems to depend on the
   value of Content-Length and Upload-Incomplete. For example, an Upload
   Creation Procedure with Upload-Incomplete: ?1, Content-Length: X, and a
   request body of X bytes creates an upload that is marked as incomplete in
   responses for the Offset Retrieving Procedure, but which cannot be extended
   using the Upload Appending Procedure. All subsequent PATCH requests are
   rejected with 409 Conflict responses because the server considers the
   upload to have a maximum length of X bytes (taken from the first request's
   Content-Length), which is already satisfied through the first request. The
   server seems to ignore the Upload-Incomplete header here, which is the main
   indicator for an upload's completion state (as far as I understood the
   specification). I could also be wrong here, but we should improve the
   draft's wording to make it clearer when an upload is considered complete. I
   also included a brief curl exchange below to demonstrate what I meant here.
   3. The sample code from
   https://developer.apple.com/documentation/foundation/urlsession/building_a_resumable_upload_server_with_swiftnio
   requires the so far unreleased Swift 5.9 version to run. Is 5.9 necessary
   or could 5.8 also suffice, which is the latest released version?
   4. Are there any plans to continue maintaining NIOResumableUpload as a
   library or was it intended as a one-off example? If you to don't have such
   plans, maybe we can host it in the tus GitHub organization and make it more
   accessible for other developers.

Thank you again for releasing the code and features!

Best regards
Marius

~/workspace/tus/draft-example (main) $ curl -i -X POST 127.0.0.1:8080 -H
> 'upload-draft-interop-version: 3' -H 'upload-incomplete: ?1' -d "hello "
> HTTP/1.1 104 Upload Resumption Supported
> Upload-Draft-Interop-Version: 3
> Location:
> http://127.0.0.1:8080/resumable_upload/14803916209729849474-1337128731644763746
>
> HTTP/1.1 201 Created
> Upload-Draft-Interop-Version: 3
> Location:
> http://127.0.0.1:8080/resumable_upload/14803916209729849474-1337128731644763746
> Upload-Incomplete: ?1
> Upload-Offset: 6
> transfer-encoding: chunked
>
> ~/workspace/tus/draft-example (main) $ curl -i --head
> http://127.0.0.1:8080/resumable_upload/14803916209729849474-1337128731644763746
> -H 'upload-d
> raft-interop-version: 3'
> HTTP/1.1 204 No Content
> Upload-Draft-Interop-Version: 3
> Upload-Incomplete: ?1
> Upload-Offset: 6
> Cache-Control: no-store
>
> ~/workspace/tus/draft-example (main) $ curl -i -X PATCH
> http://127.0.0.1:8080/resumable_upload/14803916209729849474-1337128731644763746
> -H 'upload
> -draft-interop-version: 3' -H 'upload-offset: 6' -H 'upload-incomplete: ?
> 0' -d "world"
> HTTP/1.1 409 Conflict
> Upload-Draft-Interop-Version: 3
> Upload-Incomplete: ?1
> Upload-Offset: 6
> Content-Length: 0
>

On Fri, Jun 9, 2023 at 1:13 AM Jonathan Flat <jflat@apple.com> wrote:

> Hi all,
>
> New on all Apple platforms, URLSession supports HTTP resumable uploads
> using the protocol in draft-ietf-httpbis-resumable-upload-01
> <https://www.ietf.org/archive/id/draft-ietf-httpbis-resumable-upload-01.txt>.
> The WWDC23 session “Build robust and resumable file transfers
> <https://developer.apple.com/videos/play/wwdc2023/10006/>" describes the
> new API and is accompanied by a sample code project “Building a resumable
> upload server with SwiftNIO
> <https://developer.apple.com/documentation/foundation/urlsession/building_a_resumable_upload_server_with_swiftnio>".
> Excited to hear your thoughts and feedback, and looking forward to more
> implementations!
>
> Best,
> Jonathan
> jflat@apple.com
>