The recently announced Data Transfer Project (DTP) will be a boon to bandwidth-strapped users who want or need to upload and download large quantities of data over slow, low or high-cost bandwidth connections.
That's according to the white paper published jointly by the four tech giants who initiated the project - Microsoft, Google, Facebook and Twitter. They chose to make it an open source initiative so as to encourage participation of as many providers as possible. So far, however, other tech heavyweights such as Amazon, have not joined.
Enabling connections
The goal of the project is to enable a connection between any two public facing product interfaces, allowing data to be transferred directly between them without it having to be downloaded first and then re-uploaded to the new provider.
"Given our understanding of the current (data portability) ecosystem, particularly the infrastructure constraints faced by people around the globe, it is clear that the future of portability will need to be more inclusive, flexible, and open. We believe users should be able to seamlessly and securely transfer their data directly from one provider to another," the white paper states.
The white paper authors point out that while companies like Google and Microsoft had added the ability to download your data via Takeout and Microsoft Export, this is not enough to overcome a range of shortcomings, of which bandwidth constraints is one.
Shifting the bandwidth burden
Other problems faced by anyone trying to transfer data between different providers often have to deal with incompatible formats which makes it hard for users to re-import data they have exported; and discoverability - knowing where else you could take data you have exported.
DTP aims to solve those problems by shifting the burden of bandwidth and figuring out compatibility from the user to the service provider. However, seamless data transfer will only be possible between subscribing or contributing providers.
In a blog post about DTP from Brian Willard, a Google software engineer, and Greg Fair, a Google product manager, the pair explained that the organisations involved in the project are developing tools that can concert any of their proprietary APIs to and from a small set of standardised data formats that can be used by anyone.
"This makes it possible to transfer data between any two providers using existing industry-standard infrastructure and authorisation mechanisms, such as OAuth. So far, we have developed adapters for seven different service providers across five different types of consumer data; we think this demonstrates the viability of this approach to scale to a large number of use cases," they wrote.
Integrity, security, privacy
Key to the design of the DTP is the integrity, security and privacy of the data being transferred. This is being addressed in several ways, taking into account the fact that because the data is being transferred between at least two providers, responsibility for security has to be a shared function.
Services will have to first agree to allow data transfer between them, and they will also require that users authenticate each of their accounts independently. In addition, all credentials and user data will be encrypted throughout the process.
Share