Abstract:

The provided documents detail the experimental Upspin project, a framework designed for secure, uniform, and global naming and sharing of data, prioritizing security and uniformity over performance. It outlines the project's architecture, including core client functionalities, access control mechanisms, caching strategies, and various server implementations (Key, Directory, Store) accessible via HTTP/HTTPS. A significant announcement (dated 2025-02-11 in README.md) indicates the planned deprecation and shutdown of Upspin's central infrastructure, such as the keyserver, by May 2025, due to its limited success in fostering sharing communities, despite offering attractive features like end-to-end encryption and snapshotting. The project's tooling, contribution guidelines, and Go-based code structure (Go 1.20, Apache License 2.0) are also presented.

Summarizing the Upspin Project

• Project Overview (README.md):

  • Upspin is an experimental framework aiming to build a secure, uniform, and global system for naming and sharing files and other data.
  • It is described as a "global name system," not a traditional file system, focusing on protocols and reference implementations.
  • Primary goals are uniformity and security; performance is not a primary objective.
  • Currently, it has "rough edges" and is "not yet suitable for non-technical users."

• Imminent Infrastructure Shutdown (Key Takeaway) (README.md, 2025-02-11 announcement):

  • Upspin's central infrastructure, including its keyserver, will be gradually turned down.
  • March 4: Keyserver will be turned off for one week as an alert.
  • May 6: Keyserver will be permanently turned off.
  • The decision stems from the project being "less successful at fostering communities of sharing than we hoped."
  • Users facing significant pain from this shutdown are advised to contact grosse@gmail.com.

• Key Attractive Features (README.md):

  • Strong end-to-end encryption.
  • Convenience of upspinfs (FUSE interface) for integrating with existing applications.
  • Automatic nightly snapshots, inspired by Plan 9.

• Proposed Improvements (Backlog) (README.md):

  • Switching from a central keyserver to an SSH-like authorized_keys file system for clients and directory servers.
  • Revised API for Block unpacking, enabling parallel reads.
  • A clearer model for permissions on Access and Group files.
  • Post-quantum-cryptographic packing to enhance future security.

• Core Command-Line Tools (README.binary):

  • upspin: Main command for managing files, users, and servers.
  • upspin-ui: Web interface for the name space and user signup/deployment to Google Cloud Platform.
  • cacheserver: Implements directory and storage caching, supporting write-back (default) or write-through modes. It uses local disk for storage and can be configured with a maximum cache size.
  • upspinfs: A FUSE (Filesystem in Userspace) interface for Upspin, available on Linux and macOS.
  • upspin-audit: Provides subcommands for auditing storage consumption.

• Client Functionality (client/client.go):

  • The upspin.Client interface offers core operations such as Put (to store data, including PutSequenced for version control), Get (to retrieve data), Lookup (to find directory entries), MakeDirectory, Delete, and Glob (for pattern matching).
  • It also supports Create and Open for file I/O (client/file/file.go), PutLink (symbolic links), PutDuplicate (copying references), Rename, and SetTime.

• Access Control and Permissions (access/access.go):

  • Access to files is managed through special Access and Group files.
  • Defined Right types include Read, Write, List, Create, Delete, AllRights (wildcard), and AnyRight.
  • The AllUsers constant (all@upspin.io) grants universal access, but has restrictions on its usage with other users in Access files for Read or * rights.

• Server Architecture and Communication (bind/bind.go, cmd/*server/main.go):

  • Upspin's architecture separates functionality into KeyServer, DirServer, and StoreServer interfaces.
  • bind/bind.go provides a global binding switch and a caching mechanism for these services.
  • Server implementations (e.g., keyserver, dirserver, storeserver) are wrapped and exposed via HTTP/HTTPS (cloud/https/https.go).

• Storage Backends (cloud/storage/storage.go, cloud/storage/disk/disk.go):

  • A generic Storage interface defines low-level operations (LinkBase, Download, Put, Delete, List).
  • The disk package provides a concrete implementation for storing data on a local disk, including a mechanism (guaranteeNewEncoding) to manage and migrate between different path encoding schemes.

• Additional Tools and Details (go.mod, AUTHORS, LICENSE, etc.):

  • The project is developed in Go (version 1.20) and licensed under Apache License Version 2.0.
  • cache/lru.go implements a Least-Recently Used cache for efficient data management.
  • cloud/mail/sendgrid/sendgrid.go provides an email sending functionality using the SendGrid API.
  • cmd/upbox/main.go is a tool for building and running ephemeral Upspin server clusters for testing and development purposes.
  • The upspin command-line tool includes subcommands like countersign (to update signatures after key changes) and cp (for copying files between local and Upspin, and within Upspin).