Network Working Group F. Maurer Internet-Draft September 22, 2014 Expires: March 26, 2015 The Federated Object Sharing Protocol fosp-00 Abstract This document describes the Federated Object Sharing Protocol (FOSP). FOSP is a protocol that allows sharing arbitrary data, setting access rights on the shared data and receiving notifications on changes. Furthermore, all features of FOSP can be used in a federated network, e.g. between multiple independed providers. It was designed to be the most simple solution that combines all these features. Status of this Memo This document is an Internet-Draft and is NOT offered in accordance with Section 10 of RFC 2026, and the author does not provide the IETF with any rights other than to publish as an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on March 26, 2015. Maurer Expires March 26, 2015 [Page 1] Internet-Draft FOSP September 2014 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Purpose . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2. Conventions . . . . . . . . . . . . . . . . . . . . . . . 3 1.3. Requirements . . . . . . . . . . . . . . . . . . . . . . . 3 1.4. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 2. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1. User and providers . . . . . . . . . . . . . . . . . . . . 5 2.2. Network topology . . . . . . . . . . . . . . . . . . . . . 5 2.3. Messages . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.4. Trees, Objects and Attachments . . . . . . . . . . . . . . 6 3. Network Topology . . . . . . . . . . . . . . . . . . . . . . . 7 4. Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.1. Types . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.1.1. Request . . . . . . . . . . . . . . . . . . . . . . . 8 4.1.2. Response . . . . . . . . . . . . . . . . . . . . . . . 9 4.1.3. Notification . . . . . . . . . . . . . . . . . . . . . 9 4.2. Format . . . . . . . . . . . . . . . . . . . . . . . . . . 10 4.3. Serialization . . . . . . . . . . . . . . . . . . . . . . 12 5. Data structure . . . . . . . . . . . . . . . . . . . . . . . . 13 5.1. Objects . . . . . . . . . . . . . . . . . . . . . . . . . 13 5.2. Trees . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5.3. Attachments . . . . . . . . . . . . . . . . . . . . . . . 13 5.4. Standard Fields . . . . . . . . . . . . . . . . . . . . . 13 5.5. Standart Objects . . . . . . . . . . . . . . . . . . . . . 16 6. Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 6.1. Access control . . . . . . . . . . . . . . . . . . . . . . 17 6.2. Subscriptions . . . . . . . . . . . . . . . . . . . . . . 17 6.3. Attachments . . . . . . . . . . . . . . . . . . . . . . . 18 6.4. Authentication and Registration . . . . . . . . . . . . . 18 6.5. Message forwarding . . . . . . . . . . . . . . . . . . . . 19 7. Connection Initiation . . . . . . . . . . . . . . . . . . . . 20 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 22 Maurer Expires March 26, 2015 [Page 2] Internet-Draft FOSP September 2014 1. Introduction 1.1. Purpose The Federated Object Storage Protocol (FOSP) is an application-level protocol for exchanging structured and unstructured data generated by users of different providers. It is designed to fullfill the needs of online social networks by combining necessary features into one simple protocol. FOSP aims to provide three core features: 1. Store data online and support standart operations on it. 2. Enforce access control on the data, for useres from different hosts, without central authentication. 3. Notify users when data is added, removed or has changed. Furthermore, some non-functional requirements are imposed. The protocol needs to be data agnostic and should support structured, unstructured and metadata. It also must be as simple as possible, e.g. it must be easy to implement and it should be possible to write easy to deploy applications for it. 1.2. Conventions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [refs.rfc2119]. 1.3. Requirements FOSP builds on existing protocols and data formats. The transport protocol for FOSP messages currently is the WebSocket protocol, tough other options may be added in the future. Objects are serialized into the JavaScript Object Notation (JSON). Besides these technical dependencies, FOSP is inspired by WebDAV and XMPP and has similarities to LDAP. 1.4. Terminology This document uses a number of terminologies to refer to concepts found in FOSP. Maurer Expires March 26, 2015 [Page 3] Internet-Draft FOSP September 2014 provider An entity that provides storage space on the Internet for the data of users. It is identified by a DNS domain name. server A FOSP server stores the data of users of a certain provider. Multiple servers can be deployed to deliver the data of a provider to implement fault tolerance and load balancing. client A FOSP client is a program a user uses to communicate with a FOSP server. It facilitates accessing the data of the user that is stored on the server. message A message is the basic unit of communication in FOSP. Messages come in three different flavours. request A request is a message sent from a client to a server. It is used to retrieve or alter data. response A response is a message sent from a server to a client. It is always sent as an answer to a request and contains the status of the request and possibly data. notification A notification is a message that is sent by servers when changes happen to an object. Maurer Expires March 26, 2015 [Page 4] Internet-Draft FOSP September 2014 2. Overview 2.1. User and providers In the FOSP world, data is created by users. Users store their content on the server of providers and can share it by setting the appropriate access rights for other users. Each user is registered with one provider. The user has an unique name at the provider that togther with the domain name of the provider forms his or her identifier, similar to an Email address. However users can still share their content with friends that are registered with different providers. The FOSP protocol is mostly a request/response protocol. A user connects to the server of his or her provider, using a client and then both can exchange messages. Request messages are sent from the client to the server to manipulate or retrieve data. Servers respond by sending a response message. 2.2. Network topology The network topology is similar to the SMTP or XMPP network. A provider operates one or more server that handle the domain of the provider. A user can use a client to connect to a server of a provide where he or she is registered and request data. The server can connect to a server of a different provider to forward request it can not process itself. 2.3. Messages Server and clients communicate by sending messages over a persisten connection. There are for now three different kind of messages that serve different purposes. Requests are sent from the client to the server to retrieve, alter or delete data. The server either processes the request or if it cannot, forwards it to a server of a different provider that can process the request. After processing a request, the server sents a response. If the request has been forwarded, the response will take the same path back to the client, through the server that forwarded the request. Whenever an object or attachment has been modified, notifications can be sent to users who have subscribed for these events. These notifications might also be forwarded by the server. Maurer Expires March 26, 2015 [Page 5] Internet-Draft FOSP September 2014 2.4. Trees, Objects and Attachments The information units that can be stored on the server have a well defined structure and we call them objects. An object is basically an dictionary or hashmap with a selection of predefined keys. Most of the key-value pairs have a special meaning, for example there is a key value pair that stores the access conrol list. They adhere to the JSON specification. All objects are ordered into trees. For each user, one such tree exists and the root object of the tree is named like the user. Each object can thus be uniquely identified by its path, e.g. alice@wonderlant.lit/social/me. To store data that can not be expressed in JSON, each object can have an attachment. The attachement hast the same name and path as the object. It is modified using special requests. Maurer Expires March 26, 2015 [Page 6] Internet-Draft FOSP September 2014 3. Network Topology In the FOSP network, agents are either clients or servers. Each server belongs to one provider that is identified by a domain name. For load balancing purposes, more than one server per domain might be used. To keep the examples simple, we nevertheless assume that there is one server per provider and use the term provider and server interchangeable. A user connects, using a client, to the server of their provider. We refer to this server as the home server of the user. To manipulate or access data, the user sends requests. A request can act upon a resource. If the resource is not managed by the server the user is connected to, the server will relay the request to the responsible server. Hence, FOSP servers may open connections to other FOSP servers. +--------------+ +--------------+ |alice@ | |queen@ | |wonderland.lit| |wonderland.lit| +------------+-+ +-+------------+ | | | | | | +--v--------v--+ |wonderland.lit| | | +---+------^---+ | | | | +---v------+---+ |realworld.lit | | | +--^-----------+ | | +-----------+--+ | sister@ | | realworld.lit| +--------------+ Figure 1 Maurer Expires March 26, 2015 [Page 7] Internet-Draft FOSP September 2014 4. Messages Messages are the basic unit of communication in the FOSP network. There are three different types of messages which serve different purposes. After being serialized, each message is transported over a WebSocket connection inside of one WebSocket text message or binary message. 4.1. Types Each message has a specific type, which determines the kind of purpose this message serves. The type is implicitly given in the way that the content of the message determines it's type. For now there are only three types of messages needed to support all functionality. The types are requests, responses and notifications. Should there be the need for additional types of messages, a new one can be added in later versions of the protocol. All messages can have headers and a body, similar to HTTP messages and are distinguished by their main attributes. Requests and notifications usually act on, or are emitted from an object or an attachemnt. The identifier of the object or attachemnt is then part of the request. If a request does not act upon an object, the identifier is replaced with an asterisk. Furthermore, each request and response carries an sequence number that makes it possible to to assign one response to one request. 4.1.1. Request Requests are sent from clients to servers to authenticate or manipulate objects. A request consists of ... a request type optionally an identifier of a resource that it manipulates a sequence number The request type is one of the following ... CONNECT used to initiate a connection REGISTER used to create a new FOSP user AUTHENTICATE used to authenticate Maurer Expires March 26, 2015 [Page 8] Internet-Draft FOSP September 2014 SELECT retrieves an object CREATE adds a new object UPDATE changes an existing object DELETE removes an object and all its children LIST returns a list of children of an object READ download the attached file of an object WRITE upload a file as an attachment to an object The content of the body depends on the type of request. For example, the body of a SELECT or DELETE request is empty and the body of a CREATE request contains the new object. In general the body is a JSON object, except for the WRITE request that has the byte representation of the file as the body. 4.1.2. Response Responses are sent from servers to clients when a request has been processed. A response has ... a type, either "SUCCEEDED" or "FAILED". a status code, an integer greater zero. a sequence number which must match the sequence number of the request it responds to. The body of a response depends on the request that is answered. For a CREATE or DELETE request it would be empty, for a SELECT request it would be the object that was requested. In case of a READ request it would contain the file. 4.1.3. Notification Notifications are sent from servers to clients when objects change and the client has subscribed to those changes. A notification consists of ... an event which is one of "CREATED", "UPDATED" or "DELETED". a resource identifier. The notifications are sent when a request triggers an event, for Maurer Expires March 26, 2015 [Page 9] Internet-Draft FOSP September 2014 example CREATE triggers a CREATED event. If the event equals DELETED then the body of the notification must be empty. Otherwise it should contain the new version of the object. 4.2. Format **WARNING: To simplify the definition for the moment, only ASCII characters are used. This WILL change as the protocol itself mandates the use of Unicode and the UTF-8 encoding and international user and resource names MUST be supported** Also the header definition will be subject to change after determining acceptable characters. For the resource identifier, we will probably refer to the IRI definition in the future. Maurer Expires March 26, 2015 [Page 10] Internet-Draft FOSP September 2014 message = request / response / notification request = request-type SP ( resource-id / "*" ) SP sequence-number CRLF headers [ CRLF body ] request-type = "CONNECT" / "AUTHENTICATE" / "REGISTER" / "CREATE" / "SELECT" / "UPDATE" / "DELETE" / "READ" / "WRITE" response = response-type SP response-status SP sequence-number CRLF headers [ CRLF body ] response-type = "SUCCEDED" / "FAILED" notification = event-type SP resource-id CRLF headers [ CRLF body ] event-type = "CREATED" / "UPDATED" / "DELETED" headers = *( header CRLF ) header = header-key ":" SP header-value header-key = ALPHA *( ALPHANUM / "-" ) ( ALPHANUM ) header-value = 1*( ALPHANUM / "/" / "-" / "<" / ">" / "=" / "+" / "_" / ";" / "!" / "~" / "*" / "." ) resource-id = user-name "@" domain [ path ] user-name = ALPHA *( ALPHANUM / "_" / "-" / "+" / "." ) ( ALPHANUM ) domain = domain-part *( "." domain-part ) [ "." ] domain-part = ALPHANUM / ( ALPHANUM *( ALPHANUM / "-" / "_" ) ALPHANUM ) path = "/" / 1*( "/" path-part ) path-part = 1*( ALPHANUM / "\" / "." / "+" / "*" / "-" / "_" / ";" / ":" / "!" / "~" / "=" / "<" / ">" ) ALPHANUM = ALPHA / DIGIT Figure 2 Maurer Expires March 26, 2015 [Page 11] Internet-Draft FOSP September 2014 4.3. Serialization Messages are serialized into a blob of bytes. Most of the message is text in UTF-8 encoding, only the body is either a valid JSON object, UTF-8 encoded, or not interpreted at all. The second case occurs when an attached file is up- or downloaded. The beginning and end, e. g. the length of a message must be determined from lenght of the WebSocket message. Maurer Expires March 26, 2015 [Page 12] Internet-Draft FOSP September 2014 5. Data structure 5.1. Objects We refer to the basic unit of data, that can be manipulated in the FOSP network, as an object. An object consists of key-value pairs. The constraints for keys and values are the ones described by the JSON specification. Each object has by default specific key value pairs that have a special meaning, for example, the key "owner" contains the identifier of the user who created the object. When transfering objects in messages, the object is serialized according to the JSON specification. 5.2. Trees In FOSP, all objects are part of a tree of objects. Therefore, all objects have a parent object, except for the root object of a tree. Consequently, all objects can have child objects. For each user there exists one tree of objects and the root node of this tree is named like the identifier of the user. The tree of the user is stored on the servers that are responsible for the domain where the user is registered. 5.3. Attachments As files, like pictures and documents, are also shared via social networks, FOSP supports saving binary files. For each object, one file can be saved as an attachment. This way, files can be addressed with the same schema and the objects they are attached to can provide the meta-data. If an object has a belonging file, it is extended by a new attributes named "attachment". In this attribute, the size, file name, and mime-type of the file is saved. The attached file is read and written using special requests. 5.4. Standard Fields Most attributes of an object have a well defined meaning. The servers have to ensure that the content of these attributes adheres to the specification and that users do not make changes that are not allowed. Figure 3 shows an example of an objects with all the fields described here. Currently we define the following attributes and their values: The "owner" field is set by the server on creation of the object and contains the identifier of the user who created the object as a string. It determines the ownership of this object and is used when enforcing access rights. Maurer Expires March 26, 2015 [Page 13] Internet-Draft FOSP September 2014 The "btime" (birth time) field is set by the server to the date of creation of the object. It is saved as a string, formatted according to the ISO 8601 standard and must always be in the UTC time zone. Users should not be able to change its content. It was added because it's an information that is usually of interest to users. Similar the "mtime" (modify time) is set to the current date each time the object is altered and is not to be set by the users directly. It is saved just like "btime". In the future it will be used to allow client side caching similar to ETags in HTTP. We recommend that users should not be allowed to add or alter arbitrary fields except for the "data" and "type" field. The "data" field is where the user should save the payload and may contain any valid value. The "type" field should contain a string that describes the content type of the "data" field, similar to MIME types. Furthermore, there are the three fields "acl", "subscriptions" and "attachment" that contain more complex objects. The "acl" field saves information about access rights. It is read by the server to enforce access control. The acl object can have the following fields: "owner", "users", "groups", "other" The value of the "owner" and "other" field is a set of rights A set of rights is an array of strings where each string identifies a certain right. For example, the string "read-acl" stands for the right to read the "acl" field. The value of the "users" and "groups" field is an object. In these objects, each key identifies a user or a group and the related value is a set of rights. The "subscriptions" field saves information about subscriptions. It is read by the server to determine which users have to be notified on changes to an object. It contains an object and each key is the identifier of a user. The value is an object with the fields: "events" and "depth" The value of "events" is an array of strings which identify an event, the value of "depth" is an integer between -1 and infinity Maurer Expires March 26, 2015 [Page 14] Internet-Draft FOSP September 2014 The "attachment" field is only present if this object has an attachment. It contains an object with three fields. The "name" field contains a string with the file name. The "size" field contains the number of bytes of the attached file. The "mime-type" field contains a string with the mime-type of the attached file. { btime: "2007-03-01T13:00:00Z", mtime: "2008-05-11T15:30:00Z", owner: "alice@wonderland.lit", acl: { owner: ["read-data", "write-data", "read-acl", "write-acl", "read-subscriptions", "write-subscriptions", "read-children", "write-children", "delete-children"], users: { alice@wonderland.lit: [ "read-data", "not-write-data", "read-acl", "write-acl", "read-subscriptions", "not-write-subscriptions" ] } }, subscriptions: { users: { alice@wonderland.lit: { events: [ "created", "updated" ], depth: 1 } } }, type: "text/plain", data: "Just plain text" } Figure 3 Maurer Expires March 26, 2015 [Page 15] Internet-Draft FOSP September 2014 5.5. Standart Objects Some objects in the tree of a user will be used by the server to obtain configurations. For example alice@wonderland. lit/config/ groups will contain the mapping from users to groups, that is valid for the tree alice@wonderland. lit. Maurer Expires March 26, 2015 [Page 16] Internet-Draft FOSP September 2014 6. Policies Besides implementing the technical specification of the protocol, the server has to enforce a set of policies. These policies describe how the server interprets the objects to enforces access control and send notifications for subscriptions. They also define constraints on certain attributes of objects and constraints on whether or not a message should be forwarded. 6.1. Access control As stated in Section 5.4, each object can save access control information. A valid entry in the access control list consists of the identifier of a user or a group of users and a list of rights. Manipulation of different parts of the object can require different rights, for example, altering access control information requires the "acl-write" right. Because the objects are part of a tree, we can use inheritance to set access rights for a whole subtree by setting the appropriate access control information on the root of the subtree. This way the rights on an object are the sum of the rights on this object and all it's ancestors. To still be able to have less rights on a child object, rights can be prefix with "not-" to explicitly remove a right, for example "not-acl-write". The server must enforce access control for an object in the following way. 1. Check if the right in question is set on the current object 2. If it is set positive (e.g. not prefixed with "not-") grant access 3. If it is set negative (e.g. prefixed with "not-") deny access 4. If is is not set, go to the parent object and repeat from step 1. 5. If the current object is already the root object, deny access 6.2. Subscriptions Similar to the access control information, subscriptions can be set on objects so that a user will be notified when changes occur. A subscription consists of an identifier of a user, a list of events to subscribe to and a "depth". The depth is used to subscribe to events from all child objects to a certain depth. When a change happens on an object, the server must use the following algorithm to determine which users must be notified. 1. Read the "subscriptions" from the current object. Maurer Expires March 26, 2015 [Page 17] Internet-Draft FOSP September 2014 2. For each subscription: If the event that occurred is in the list of subscribed events and the distance of the current object to the object where the event occurred is smaller or equal to the "depth" of the subscription or the "depth" is equal to "-1", add the user of this subscription to the list of users that should be notified. The distance between two objects is 0 if they are the same object, 1 if one is the parent of the other and so on. 3. Go to the parent of the current object and repeat from step 1. unless the current object is the root object An important aspect to consider when sending notifications that include the new version of the object, is that every user who will be notified might be allowed to only see different parts of the object. Therefore, the server has to calculate the view of the object per user to prevent leaking of data a user might not be allowed to see. 6.3. Attachments As explained in Section 5.3, each object can have a file as attachment. However, attachments will likely not be stored together with objects, but in a storage that is more suitable for files. Depending on how the server implements operations on attachments, it is possible that there exists an attachment for an object in the storage but the object itself is missing the "attachment" field. In any case the behavior of the server should be consistent. Hence, if there is no attachment field in the object then the attachment should be deleted and an attachment should only be readable if there is an attachment field in the object. 6.4. Authentication and Registration When a client connects to a server, it has to provide credentials to the server, so that the user of the client can be authenticated. These are send in the body of the AUTHENTICATE request and can simply be an object containing the name and the password of the user. The schema of authentication is not limited thought as the body can contain arbitrary structured data. Server to server authentication is done using the DNS. When a server receives a connection from another server it verifies that the domain of the connecting server resolves to the same IP address the connection comes from. The REGISTER request should be supported by all servers, but can be disabled to prevent automatic creation of accounts by autonomous programs. It is also not required that the REGISTER command is the only way to create an acount on a server. For example, there could be sign up forms on websites or a server of a company could be connected to a directory service and fetch the login informations Maurer Expires March 26, 2015 [Page 18] Internet-Draft FOSP September 2014 from there. 6.5. Message forwarding A server does not need to accept forwarded requests from other servers, if it shouldn't be part of the federated network. This allows FOSP to be used in cooperate environments or other closed environments where federation with the outside world is not allowed. In any case, a server must never accept a forwarded request if the user of the request is not on the domain of the server that forwarded the request. For example, if server A, that is authoritative for domain "example. net", authenticates to server B and then forwards a request of user "alice@wonderland. lit", server B must close the connection. Maurer Expires March 26, 2015 [Page 19] Internet-Draft FOSP September 2014 7. Connection Initiation First, the agent that wants to open a new connection has to find out which server to connect to and on which port. The server responsible for a certain domain is the server with the IP address that is assigned to the DNS A record of the domain name and the port is 1337 for insecure connections and 1338 for secure connections (e.g. ws:// or wss://). In the future, a better approach of discovering the server will be necessary and probably done by using DNS SRV records. After opening a connection to a server, the initiator first sends a CONNECT request that has to be answered with a SUCCEEDED response if the connection is accepted. The body of the request contains information about the supported version of the protocol and could in the future be used to negotiate more pa- rameters of the connection. The response, if successful, contains the version the server supports and could also be used to advertise additional services provided by the server. If the connection is opened by a client to a server, a new user can then be registered with the REGISTER request. Finally, the initiator has to authenticate to the server he connects to. The specific mechanism of authentication depends on whether a client connects to a server or a server connects to another server. Maurer Expires March 26, 2015 [Page 20] Internet-Draft FOSP September 2014 8. References [refs.rfc2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, BCP 14, March 1997. Maurer Expires March 26, 2015 [Page 21] Internet-Draft FOSP September 2014 Author's Address Felix K. Maurer Email: felix.maurer@student.kit.edu Maurer Expires March 26, 2015 [Page 22]