This document describes a proposal for a protocol for Free Digital Territories, FDT, allowing autonomous or semi-autonomous communities to form networks and exchange ideas and services based on their own infrastructure which is required to function with or without access to the public Internet.
Status of this memo
This memo proposes a standard for supporting free digital territories with Free Software, as defined by the Free Software Foundation and the GNU project. Anyone is free to implement the protocols defined in this memo under the license conditions they see fit, but the memo itself always supposes that territories are completely free to share software and solutions. All protocols described in this memo are implementable in software and is not subject to patenting. The current status of this memo is unfinished. The distribution of this memo is unlimited.
The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY” and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.
1. Introduction
The concept of Free Digital Territories will be defined below. Historically, this work is motivated by the needs encountered by the Mocambos network in Brazil, which is a network of traditional Afro-Brazilian communities, so-called “quilombos”. The foundations of the work was laid in the early 2000s [1]. A precursor of or prototype for the protocol described in this memo is implemented by the Mocambos network’s Baobáxia project. This memo aims to formalize and improve the protocol in order to allow it to co-exist with many other applications for free digital territories
1.1 What is Free Digital Territories?
The concept of free digital territories is linked to the notion of “free territories”, autonomous or semi-autonomous communities, often geographically well-defined, sometimes very rural and remote, capable of running their own infrastructure and capable of and interested in collaborating with other free territories in a wider network. Traditional examples of such free territories are indigenous villages and quilombos (rural Afro-Brazilian communities). Modern examples are inner-city squats, community-organized cultural centres and eco-villages.
For a free territory, a top priority will often be to carve out a physical space for themselves. In the case of traditional communities, this is often reflected in a very basic struggle for land on which to build and grow things.
For a free digital territory, the priority is likewise to carve out a digital space owned and controlled by the community and in which the community’s lives, ideas and initiatives can grow. Just as free territories are assumed to build on access to physical land or physical spaces, free digital territories build on digital land and digital spaces, services and storage facilities operated by the communities themselves.
1.2 Basic Requirements for the Free Digital Territories Protocol
Free Digital Territories will be organized in networks, and these networks will be viewed as consisting of nodes. Each of these nodes is a repository for data related to services available in the network.
1.2.1. Eventually connected
Nodes in a network MUST be able to share data and request services even without a direct network connection between them. They MAY be installed on the Internet, i.e. with a direct connection between them, and in this case they MAY be able to share continuously and directly. Data MUST be able to traverse several nodes while travelling the network, and responses e.g. for service requests MUST be able to propagate back to the origin of the request, possibly by another route.
1.2.2 Identity
A common identity system SHOULD be provided in order to share data even if different ways could be implemented. This identification system SHALL follow the normal Internet domain addressing scheme. So the user Vince’s user account in the Dpadua node blog system that is linked to the Mocambos network will follow the usual syntax vince@dpadua.mocambos.net.
Following requisite 1.2.1, identity SHOULD be managed by a decentralized authentication system, where user credentials are communicated to other nodes in the network in the same way as all other kinds of data.
1.2.3 Data Layer
Each node is autonomous and uses a common data addressing system as a URI, following the unix path system. So the blog article “Hello World” on the Dpadua node can be acessed on path “/fdt/mocambos/dpadua/blog/hello-word” in all nodes onto which it is syncronized in the Mocambos network.
The structure of this URI MUST be: <top level path> / <network> / <node> / <application directory> / <content>
Of these items, the top level path WILL contain the URI scheme, domain authority and MAY include an additional path to where the FDT is mounted in the domain authority’s URI scheme. It is RECOMMENDED that the nodes of an FDT are installed on dedicated servers and its networks appear directly after the authority (see example in note [1]).
The network indicates the network of FDTs that the content exists in - a network being defined as a collection of nodes that may be synchronized with each other. In the example, the network is “mocambos”, representing the Mocambos network.
The node is the node in which the content originates - in the current example, “dpadua” [2].
In each node, the data are organized in application directories. These are structured according to the needs of the application which uses them and are not governed by this RFC. However, the naming of application directories and their assignment to applications MUST be agreed upon between all nodes in the network.
2. The FDT Protocol
2.1 Separation between metadata and content
The applications alone SHALL decide the format in which their content is be represented and the way it should be stored in each application directory. However, there MUST be a mechanism for separating the metadata and the content of each object in order to synchronize the metadata between nodes without including the actual contents. Each application MAY use this separation according to its needs. This separation MUST be used for all binary formats - for text-based formats (e.g., reStructuredText or HTML), this separation is OPTIONAL. For non-binary formats, the inclusion of metadata is OPTIONAL and application-dependent.
If information about an object (as opposed to just its contents) is to be displayed by itself on a public URI, it is RECOMMENDED that metadata is used as described in section 2.2 regardless of whether the object is binary or not.
2.2 Metadata format
The metadata for each content object MUST, if present, allow for the following information:
Author
Creation date
Format
Last modification date
License
Media file name (path)
Note
Node of origin
Network
Type
URL
UUID
The application MAY decide that one or more of these fields should be optional. However, if a unique ID for the object ID is included, this MUST be a UUID. The metadata MUST be in a human-readable text-based format such as YAML or JSON.
2.3 Object creation
2.4 Object changes
2.5 Synchronization of metadata and non-binary content
2.6 Tags
2.7 Transportation of binary content
2.8 Service requests
2.9 Version history
2.10 Deletion
3. A sample implementation (using git and git-annex)
We can share our experience with the free software Baobáxia, that can manage a network of multimedia libraries. Each library is a computer with Baobáxia installed, a local node we call “mucua”, where data is stored in filesystem like a git and git-annex repository.
We separate metadata and contents (2.1) using git to store metadata (like 2.2) in simple json text format, and git-annex to store and manage the data content. When a new content is published (2.3) on a mucua we create the metadata json file on a directory structure organized by date like:
/data/bbx/mocambos/dpadua/images/18/02/14/working-on-rfc-2df4c.json
where the image binary is saved in the same path like:
/data/bbx/mocambos/dpadua/images/18/02/14/working-on-rfc-2df4c.jpg
and then with the “git-annex add” command, became a symbolic link stored in git history, and the binary data is stored in .git/annex/ directory, managed in git-annex way (see https://git-annex.branchable.com/).
This process is realized by a django application that also mantains its object data layer saved in a local sqlite database. We trust data on filesystem/git/git-annex as the primary data source, so we can always erase and regenerate the sqlite database.
4. Interoperability between different implementations
REFERENCES
[1] See, e.g., this blog post account of an early meeting: http://baobaxia.mocambos.net/#mocambos/dpadua/media/14fe5d93-3b0d-48d6-a7c9-4ec8a242feee
[2] The node Dpadua is located in Mercado Sul, Brasilia and is named in honor of the late Brazilian developer and activist Daniel Pádua (1980-2009) who participated in the creation of the protocol described in this document. See Daniel Pádua – Wikipédia, a enciclopédia livre for more information.
