This presentation introduces "up-spin," a nascent global namespace system designed to address the challenges of personal data management in the modern digital era, where users often lack true ownership and control over their digital assets. up-spin aims to unify existing data sources (e.g., social media feeds, cloud storage, local files) under a single, user-centric framework, emphasizing personal privacy and secure sharing. It operates as a layer above current systems, not replacing them, and features end-to-end encryption by default, encrypted data at rest, and a transparent, plain-text access control model. The system comprises a minimalist Go-centric architecture with key, storage, and directory servers, alongside client-side tools like up-spin FS for UNIX integration. Practical examples demonstrate its potential for managing media libraries, live camera feeds, and even entire $HOME directories securely across multiple devices, all while retaining user control and simplifying sharing.
Summarizing "up-spin: A New Approach to Data Ownership"
0:03 Problem Statement: The speaker highlights the paradox of modern digital data: users often "rent" rather than "own" their digital media (e.g., from Google Play, iTunes), struggle with data management, and lack control over personal data tied to online accounts.
1:15 Historical Context: The speaker traces data sharing from the community-like experience of 1970s time-sharing systems and 1980s/90s distributed file systems to the current "cloud" model (2010s) where data is often inaccessible or controlled by third parties.
3:28 Introducing up-spin: A new, very young, Go-centric system is introduced as a solution. It's designed as a global namespace, not a file system, to simplify naming and management of diverse data across different platforms.
3:40 Global Namespace & Naming: up-spin is a "forest" of data trees, not a single tree. Each user has a "private root" named by their user ID (e.g., argorg@gog.org/some/stuff), allowing users to name and organize their data universally, including data external to up-spin.
4:28 Secure Federated Protocols: The system uses secure federated protocols, similar to email, where individual services can connect into a cohesive global space.
4:50 Flexible Sharing Model: up-spin offers a simple, fine-grained sharing model. Users can share specific items individually without making them globally readable.
5:05 Layer Above Existing Services: up-spin aims to connect and unify existing services (social web, file systems) rather than replace them, acting as an umbrella for diverse data. It is specifically designed for personal use (individuals, families, friends), not corporations.
5:47 What up-spin is NOT: It's not a new file system (though it can act like one), not a new cloud platform (like AWS or GCS), and not Dropbox (which copies everything everywhere, impractical for petabytes of data). It's most similar to Keybase but differs in sharing and target audience.
9:49 End-to-End Encryption & Security: Data stored in up-spin is end-to-end encrypted by default and encrypted at rest. Storage servers never see clear text or private keys, ensuring genuine privacy. Access control cooperates with encryption, allowing only authorized users' keys to decrypt specific files.
8:18 Transparent Access Control: Sharing is managed by plain text .access files dropped into directories, defining rights (read, write, delete, list) for specific up-spin users or named groups. Only the owner can modify these files, ensuring clarity and control.
11:15 Go-centric Architecture: The system uses a Go-centric model with three primary server types:
11:32 Key Server: A single global server (key.up-spin.io) stores user data (username, public key, network address of directory/store servers). It's crucial for users to live in a single namespace.
12:07 Storage Server: Many distributed servers that store encrypted data, typically content-addressed using hashes.
13:58 Directory Server: Many distributed servers that manage file paths and references, often implemented as a Merkle tree storing records in a storage server.
14:26 Minimal API: The entire system has a total of only 10 methods across the three server types. Directory servers do not handle data directly; they provide references to the storage server.
15:22 Client Library & Tools: A local client library offers traditional file-like APIs. up-spin FS, a FUSE-based file system, allows users to mount the entire up-spin global namespace (including shared data from others) onto their UNIX systems.
16:34 Use Case: Media Libraries: The speaker's Lightroom catalog and iTunes library are stored in up-spin on a Google Cloud instance. This provides a single, encrypted, network-accessible library visible across all his computers, eliminating manual syncing. Sharing specific photos (e.g., raw files) is as simple as adding an entry to an .access file and sharing the up-spin name.
18:44 Use Case: Streaming Cameras: Security or nursery cameras can stream data into up-spin. The data is encrypted and only accessible by authorized family members (defined via a group file), providing secure streaming without relying on external services.
20:01 Use Case: "$HOME" in up-spin: The ultimate goal is for the user's $HOME directory to reside entirely in up-spin. This would allow users to log onto any machine and have their complete, current environment instantly available, eliminating the need to carry disks or manually sync.
21:13 Fine-Grained Services: Chris Foster has built a service on up-spin for even more fine-grained sharing, like tax returns with an accountant, leveraging up-spin's API.
21:46 Status & Future: The project is in its early days. Protocols and a reference implementation are mostly complete (in Go). It works with GCP, AWS, and UNIX, with Windows support needing a filesystem plugin. It's open source and seeking community contributions for development and documentation.
22:59 Mascot: The project's mascot is Augie, a skeptical character designed by Renee, symbolizing vigilance against data theft.
23:47 Q&A - Piracy: The speaker acknowledges the piracy concern but emphasizes that up-spin protects user data from being "broken into" by the law.
24:55 Q&A - Concurrency: up-spin leverages Go's scaling features. The watch API on the directory server uses channels for handling concurrency and events across the network.
25:39 Q&A - Sensitive Information: up-spin is designed for sensitive information like tax records, allowing secure sharing with specific individuals (e.g., an accountant) through its access control model.
26:04 Q&A - Private Chat: While not specifically designed for it, up-spin's architecture could support private chat functionalities similar to Keybase.
26:17 Q&A - Transport Protocol: Currently uses a custom, simple HTTP-based protocol (after performance issues with gRPC) but plans to revert to gRPC once its performance improves.
26:47 Q&A - Comparison to IPFS: up-spin differs from IPFS and similar systems by targeting individuals over corporations, separating directory and storage servers (allowing for resilience like multi-cloud mirroring), and its unique, transparent sharing model through plain text access files.
28:51 Q&A - Sharing Reverse Lookup: No, outsiders cannot reverse-lookup who is sharing files because the .access files are also protected by up-spin's encryption and access rules.
29:40 Q&A - Integration with External Data (e.g., Twitter): The vision is for external services like Twitter to implement up-spin's trivial API, allowing users to name and access elements of their Twitter feeds within the up-spin namespace.
30:13 Q&A - Key Server Ownership: Currently, the development team runs the single global key server but expresses discomfort with this long-term. They plan to transition ownership to another party once a suitable candidate is found, as the key server's API is very simple.