DSNP Specification

Welcome to the Decentralized Social Networking Protocol (DSNP) specification! Here you can find the detailed specification documentation for DSNP, official implementation specifications, associated specifications, as well as a roadmap and proposals for future extensions.

Goals & Purpose

The free communication of users on the Internet faces a variety of problems in the modern day. These challenges include censorship from state and corporate actors, the amplification of misinformation through viral content, and an ever-shrinking collection of near monopolies with absolute power over social interaction in the twenty-first century. Through the DSNP, we hope to mitigate and ideally solve these challenges in the way social interaction operates online.

How to Read This Specification

There are three core specifications currently.

NameVersionDescription
DSNP1.1.0The implementation-agnostic DSNP specification
DSNP Implementations-Implementation-specific specifications
Activity Content1.1.0A specification for DSNP-referenced content (subset of W3C Activity Streams)

Each specification is divided into several modules. Each module describes a mostly self-contained aspect of the specification.

Versioning

DSNP specification versions follow Semantic Versioning 2.0 for releases.

Contributions

Development occurs on GitHub. All interactions must follow the Code of Conduct and Contribution guidelines.

Draft Specifications

Proposals for some future extensions can be found in the Draft Specifications section and on GitHub; however, these proposals should be regarded with caution as they are largely incomplete and may be subject to change or deletion without notice.

NameStatusDescription
ArchivistDRAFTLong-term DSNP Announcement Storage

Draft Status Definitions

NameDescription
DraftOpen for comment and major changes.
ProposedReady for formal review. It should be done except for minor changes.
TentativeThis is the accepted plan. The specification should not change unless there are blocking issues.

Learning More

In addition to this document, more resources regarding our project can be found at DSNP.org, including our blog, forum, code repository, and SDK.

Roadmap

Last Updated: 2021-09-08

Introduction

This document represents Roadmap for DSNP and related specifications. Roadmap may not reflect current priorities and is subject to change.

Completed

WorkStatus
WhitepaperDone (2020 Q4)
Spec OutlineDone (2021 Q1)
Batch Announce First DraftDone (2021 Q1)
Spec Live!Done (2021 Q1)
Identity Contract & DelegationDone (2021 Q2)
Batch Announce File FormatDone (2021 Q3)
Identity FactoryDone (2021 Q2)
Graph Handle RegistryDone (2021 Q2)
Batch Announce Filter SystemDone (2021 Q3)
Graph DataDone (2021 Q3)
Announcement/Publishing RevisionDone (2021 Q3)
Stabilize 1.0Done (2021 Q3)

Future

Below is a tentative roadmap to what protocol features are being developed next! Items are ordered by current priority.

Chain Migration

Status: In progress

The cost to use Ethereum is too high. Migrate the DSNP architecture to another chain.

Private Graph

Status: Whitepaper

Keep some graph connections private with permissioned access.

Direct Messaging

Status: Whitepaper

One-to-one messaging with metadata privacy.

Public Friends

Status: Not Started

Graph connections are currently limited to one-way follow relationships. Build support for other forms of relationships that require all parties to opt in.

Social Account Recovery

Status: Not Started

Use the social graph to leverage safe account recovery and protect against private key losses.

Private Friends

Status: Not Started

Expand the private graph to other relationships while maintaining metadata protection.

Verified Attributes

Status: Not Started

Some users and applications need social identities with additional layers of validation. How can these be created while preserving privacy and supporting an open system?

Distributed Content Moderation

Status: Not Started

Content moderation is hard because it means different things to different people. Make customizing content moderation easier through distributing the work and results.

Namespace and Multichain Support

Status: Not Started

Identity and content from other systems could be used without needing to understand the originating system. Support DSNP across multiple chains to increase user choice and avoid locking data to one chain or network.

Content Bridges

Status: Not Started

Open social network bridges would allow more content and users to be accessible to all via DSNP.

Service-Node Protocols

Status: Not Started

Sub-specifications targeted at ecosystem services to promote interoperability.

Private Group Messaging

Status: Not Started

Secure many-to-many conversations. Group chat style with metadata privacy.

Private Posts to Friends

Status: Not Started

Secure one-to-many posts that still allow for commenting.

Third-Party Identity Bridges

Status: Not Started

Create ways for users to be able to bring their own identities or connect a DSNP Identity with outside identities.

Archivists

Status: Draft

Long-term storage for announcements and perhaps expanding to content storage.

Peer-to-Peer Publishing

Status: Not Started

Build a peer-to-peer system for announcement publishing for faster routing and message delivery.

DSNP Specification

Version 1.1.0

DSNP is a social networking protocol designed to run on a blockchain. It specifies a set of social primitives along with requirements for interoperability. Go to Implementations for specifics on how DSNP is implemented on a specific blockchain.

Releases

Implementations MUST specify the version(s) of the DSNP specification with which they are compatible.

VersionDescriptionRelease DateChangelog
1.1.0DIP-148, DIP-149, DIP-150, DIP-165, DIP-1802022-05-06Changelog
1.0.0Initial Release2021-09-09Changelog

Identity

This specification is intended to cover the concept of identity within the protocol and how we represent it.

Identifiers

DSNP Identifiers form the basis for pseudo-anonymous streams of content. While some users may choose to link or expose their real-world identity, DSNP implementations MUST NOT require such data exposure for account creation. The social graph is formed using this identifier so that a user's connections maintain integrity regardless of changes in any user's client choices or access changes.

Pseudo Anonymity

  • An identifier MUST default to being disconnected from a real-world identity.
  • An identifier MUST be unique to the implementation.

Ownership

A user's ownership of their identity is expressed via ownership and control of their pseudo-anonymous identifier(s). Control entails the power to announce content associated with the identifier and the ability to delegate permission to others to announce content on the user's behalf.

Delegation

  • An owner MUST be able to delegate permission to announce on their behalf to other parties.
  • A user MUST be able to revoke delegated permissions.
  • Announcements from a delegate MUST be able to be verified as to which delegate made the specific announcement.
  • Delegation revocation MUST NOT be retroactive.

Non-normative

Retroactive Revocation of Delegation

There are many times when someone would desire retroactive revocation of delegation. If a key was found to have been compromised at an earlier time for example. However, retroactive revocation is much more difficult from a caching and performance perspective. Instead reverting any undesirable Announcements such as through Tombstones allows a user to choose the specific events that need reverting and notify the network of that change.

To this end, any Batch Publications can always be validated from the perspective of the time they were published instead of needing re-validation at future read times. The validity of a Batch is thus immutable.

  1. Collect all the DSNP Ids used in a Batch.
  2. Validate that each DSNP Id had delegated to the publisher of the Batch at the time of publishing.
  3. Validate that any failures from step 2 were from DSNP Ids that revoked delegation within the acceptance window of prior blocks.
  4. Batch can be recorded as valid or invalid.

Identifiers

The DSNP Identifiers form the basis for pseudo-anonymous streams of content. Graph connections are formed through the DSNP User Id.

DSNP User Id

  • 64-bit Unsigned Integer
  • MUST be serialized as decimal
  • MUST be unique per implementation

DSNP Content Hash

DSNP Protocol Scheme

  • MUST always be the string dsnp://

DSNP User URI

DSNP User URI consists of two parts: the scheme and the user id. It is used to identify a user via a URI.

Example

dsnp://1311768467294899700
partvalue
Schemedsnp://
User Id1311768467294899700

DSNP Content URI

DSNP Content URI consists of three parts: the scheme, the user id, and the content hash. It is used to uniquely identify an announcements from a given user with content.

Any Announcement Types with a fromId and contentHash have a DSNP Content URI.

Example

dsnp://78187493520/0x1234567890abcdef0123456789abcdef0123456789abcdef0123456789abcdef
partvalue
Schemedsnp://
User Id78187493520
Content Hash0x1234567890abcdef0123456789abcdef0123456789abcdef0123456789abcdef

Graph

This specification describes the social network graph and how it is represented in the protocol.

In this context a social graph means a graph that represents social relations between entities.

The DSNP graph represents nodes as DSNP User Ids.

Public Graph

The public graph is anchored in state transitions recorded in Graph Change Announcements. Playing all state transitions in order will generate the current state of the public graph.

Non-Normative

Private Graph

Future. See the DSNP Whitepaper for foundational ideas.

Following

A "follow" is the act of publicly following a DSNP User Identity, which results in adding this DSNP User Id to a user's social graph.

An "unfollow" is the act of publicly unfollowing a DSNP User Identity which results in the removal of this DSNP User Id from a user's social graph.

Friendship

There is currently no concept of "friendship" within the DSNP network. Friendship requires a mutual acknowledgment between two different DSNP User Identities. Friendship could be thought of as "mutual following" - where two DSNP User Identities are following each other.

Batch Publications

A Batch Publication is an Apache Parquet file with a collection of Announcements.

Implementation Requirements

Discoverable

Implementations MUST have publicly discoverable Batch Publications.

Validity

Implementations MUST be able to validate the Parquet file contents. Validity MUST be immutable.

Historical

Implementations MUST retain proof of existence of a Batch Publication.

Transparent Chain of Delegation

All Announcements in a Batch file MUST be able to be proven to be from or have a chain of delegation to the publisher of the Batch.

File Requirements

Batch files are stored and transferred in Apache Parquet format.

  • Batch files MUST match the spec for a single Announcement Type.
  • Batch files MUST have Bloom filters set in accordance with the Announcement Type Spec.
  • Batch files MUST have NO MORE THAN 128*1024 rows.

Bloom Filter

Calculation for filter bits is different and nearly a factor of 10 lower than for a standard Bloom filter: 128*1024 rows with a 0.001 false-positive rate results in around 29,000 bits for a Split Block Bloom filter.

Bloom filters are ONLY added to some fields. See also Announcement Types.

Columns with Bloom Filters

ColumnParquet Type
contentHashBYTE_ARRAY
emojiBYTE_ARRAY
fromIdBYTE_ARRAY
inReplyToBYTE_ARRAY
objectIdBYTE_ARRAY

Non-Normative

Design Requirements

Batch files need to be quickly and easily searchable. Minimal storage size and fast, simple querying are preferred over guarantees of no false positives or advanced data manipulation and column relationships. The files are parseable by client applications, web views, or browsers running pure JavaScript without a need to convert the format.

Applications need to know if a given Batch file has any information they are interested in without downloading the file first.

Why Parquet?

  1. Parquet is a column-oriented format. Since DSNP Batch Message data will have a very small column-to-row ratio compared to a typical web application database, it makes sense to prefer a column-oriented format.
  2. Parquet format has been field-tested under extreme network conditions. It has broad support in cloud storage solutions, with libraries in multiple languages.
  3. Bloom filters are already supported in the Parquet specification, which allows for fast and accurate searching (with caveats for proper configuration).
  4. Amazon S3 support: We anticipate that some Batch Announcers, and possibly Archivists, will store Batch files on Amazon S3. Amazon Athena also supports storage in Parquet, and its API supports SQL-like queries.
  5. Parquet also allows references to the same column across files, which could enable multi-file querying in the future.
  6. Parquet supports compression formats such as Brötli, which itself is already a browser standard and has a demonstrated improvement in compression speed and file size over older formats.
  7. Parquet files can be transferred directly to clients, which can parse the files in the app or browser. No conversion to some serialization format is necessary. This eliminates an entire class of bugs and makes both fetching and querying faster.
  8. Parquet uses schemas, which additionally reduces file size.

Rejected Alternatives

  1. Cassandra, RocksDB, CouchDB, MongoDB, and HBASE were rejected since DSNP data needs neither a database for storage nor the overhead of one. Each of these was designed for use cases ranging from somewhat to drastically different than the DSNP network.
  2. JSON, BSON, and SQLite, while used for storage sometimes, are intended for serialization. They are schemaless, which results in redundant information and therefore a larger size than formats with schemas. They also don't support Bloom filters; instead, indexing would be required, or new batches would need to be downloaded entirely. The exception is SQLite, which does support more advanced queries; however, it was designed for in-memory storage.

Batch Validity and Order

Batch validity is immutable and usually in part based on the validation of the delegation of authors listed inside the batch to the publisher. Due to the nature of distributed systems, it is possible that a race condition occurs such that a user's delegation revocation presents before a Batch that contains a message from that user via the revoked delegate. While those individual messages should be considered invalid, testing a historical window of some time is suggested before considering the entire batch invalid. This is analogous to the idea of a confirmation time, but only applies to the past instead of the future.

Announcements Overview

Announcements are content or references to content that communicate new user activity to the rest of the network. Announcements are associated with an Identifier that can be validated as the creator of the announcement. Depending on the implementation, Announcements may be published directly to the network, included in Batch Publication Files, or some combination of those two.

Announcement Validation

There is no guarantee that, at time of creation, a given Announcement will be from the fromId claimed in the Announcement. The reader MUST perform a validation of the Announcement at read time to ensure authenticity. Implementations MUST provide a way to validate that the identifier associated with a given Announcement is authentic.

Announcement Types

Each Announcement has an enumerated type for use when separating out a stream of Announcements.

ValueNameDescriptionDSNP Content URITombstone Allowed
0Tombstonean invalidation of previously announced contentnono
1Graph Changesocial graph changesnono
2Broadcasta public postYESYES
3Replya public response to a BroadcastYESYES
4Reactiona public visual reply to a Broadcastnono
5Profilea profileYESno
6Updatean update to contentYESno

Duplicate Handling

Duplicate Announcements may occur due to the nature of asynchronous communication. In the case of duplicates, the first Announcement should be considered the ONLY valid Announcement. Additional duplicate Announcements MUST be rejected or ignored.

Ordering Announcements

  1. Order Batch Publications by implementation order.
  2. Order Announcements in a Batch Publication File by row appearance order.

Reverting an Announcement

Announcements may not be deleted, but some may be marked as invalid by using a Tombstone Announcement, or updated by using an Update Announcement. For example, if a user creates a Reaction Announcement, they may remove that reaction by creating a Tombstone Announcement.

Non-Normative

Duplicate Announcements

Due to the distributed nature of DSNP, duplicate Announcements are possible from time to time. These should be discarded and ignored.

Replay Attacks

Implementations restrict replay attacks usually through testing that the chain transaction sender is authorized, often through delegation, to publish an Announcement.

Announcement Ordering and Activity Content Published Timestamp

Activity Content has a published field that contains a user-generated timestamp. User-generated timestamps cannot be validated, but may be used to indicate ordering other than the network order for Announcements, which are not time dependent.

Announcement Reference Ordering

Some Announcements contain references to other Announcements via the inReplyTo field. Due to the distributed nature, the canonical order can have an Announcement that refers to another later in the order. For display purposes, these messages should be considered to have occurred after the reference.

DSNP v1.0 Announcement Signatures

In DSNP v1.0, Announcements had individual signatures. That produced Batch Publications that were generic and disconnected from the user. They could be submitted to the chain via anyone not just delegates or users.

In DSNP v1.1, Announcement signatures were removed in favor of the implementation being responsible for the connection between the on-chain signature and the user. The expected and EVM implementation is that the implementation chain requires that the transaction that produces a Batch be performed by the user or delegate directly. This created batches that are delegate specific, but allows for faster testing of the validity of individual Announcements in a Batch.

For more information see DIP-145.

Tombstone Announcement

A Tombstone Announcement is a way to note that a previously announced content is invalid and the related Announcement should be considered reverted. It is NOT possible to revert a tombstone.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (0)enumdecimalINT32no
fromIdid of the user creating the Announcement and the Tombstoned Announcement64-bit unsigned integerdecimalUINT_64YES
targetAnnouncementTypetarget tombstoned Announcement typeenumdecimalINT32no
targetContentHashtarget contentHash of the original Announcement to tombstone32 byteshexadecimalBYTE_ARRAYYES

Field Requirements

announcementType

  • MUST be fixed to 0

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation

targetAnnouncementType

  • MUST be the Announcement Type of the target Announcement
  • MUST ONLY be a Tombstone allowed Announcement Type

Tombstone Allowed Announcement Types

ValueName
2Broadcast
3Reply

targetContentHash

  • MUST match a contentHash of previous Announcement with the same fromId as the Tombstone Announcement

Graph Change Announcement

A Graph Change Announcement is for publishing relationship state changes for a user.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (1)enumdecimalINT32no
changeTypeType of relationship changeenumdecimalINT32no
fromIdid of the user creating the relationship64-bit unsigned integerdecimalUINT_64YES
objectIdid of the target of the relationship64-bit unsigned integerdecimalUINT_64YES

Field Requirements

announcementType

  • MUST be fixed to 1

changeType

  • MUST be one of the Change Type Enum

Change Type Enum

Different change types have different meanings.

ValueNameDescription
0UnfollowRemove a Follow relationship
1FollowCreate a Follow relationship

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation

objectId

Non-Normative

Graph Retrieval, Ordering and Reading

Each Graph Change event represents a state transition for the graph. The state of the graph at any time is given by taking the state of the graph at a previous time and applying all Graph Change events not previously applied in the order specified above. Once those Graph Change events are retrieved, they can be ordered to reflect the current graph state (i.e. Charlie has followed Bob, then he unfollowed him, and then followed him again. The graph state reflects that Charlie is following Bob).

To retrieve the graph, do the following:

  1. Retrieve the events with announcementType matching the enum for "Graph Change"
  2. Filter the events to a particular DSNP User Id to retrieve information about the respective graph.
  3. Order the retrieved data by Announcement Ordering.

Broadcast Announcement

A Broadcast Announcement is a way to send a public message to everyone.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (2)enumdecimalINT32no
contentHashkeccak-256 hash of content stored at URL32 byteshexadecimalBYTE_ARRAYYES
fromIdid of the user creating the announcement64-bit unsigned integerdecimalUINT_64YES
urlcontent URLUTF-8UTF-8UTF8no

Field Requirements

announcementType

  • MUST be fixed to 2

contentHash

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation

url

  • MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
  • Resource MUST be one of the supported Activity Content Types
  • MUST use one of the supported URL Schemes

Supported URL Schemes

SchemeDescriptionReferenceDSNP Version Added
HTTPSHypertext Transfer Protocol SecureRFC28181.0

Reply Announcement

A Reply Announcement is the same as a Broadcast Announcement, but includes an inReplyTo field for noting it as a reply to a given DSNP Content URI.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (3)enumdecimalINT32no
contentHashkeccak-256 hash of content stored at URL32 byteshexadecimalBYTE_ARRAYYES
fromIdid of the user creating the Announcement64-bit unsigned integerdecimalUINT_64YES
inReplyToTarget DSNP Content URIUTF-8UTF-8UTF8YES
urlcontent URLUTF-8UTF-8UTF8no

Field Requirements

announcementType

  • MUST be fixed to 3

contentHash

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation

inReplyTo

url

  • MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
  • Resource MUST be one of the supported Activity Content Types
  • MUST use one of the supported URL Schemes

Supported URL Schemes

SchemeDescriptionReferenceDSNP Version Added
HTTPSHypertext Transfer Protocol SecureRFC28181.0

Reaction Announcement

A Reaction Announcement is for publishing emoji reactions to anything with a DSNP Content URI.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (4)enumdecimalINT32no
emojithe encoded reactionUTF-8UTF-8UTF8YES
applyhow to apply the reaction8-bit unsigned integerdecimalUINT_8no
fromIdid of the user creating the relationship64-bit unsigned integerdecimalUINT_64YES
inReplyToTarget DSNP Content URIUTF-8UTF-8UTF8YES

Field Requirements

announcementType

  • MUST be fixed to 4

emoji

  • Emoji fields must not be empty
  • Emoji fields must consist only of Unicode points from U+2000 to U+2BFF, from U+E000 to U+FFFF, or from U+1F000 to U+10FFFF

Examples

All of the following should be considered valid emoji:

"😀", "🤌🏼", "👩🏻‍🎤", "🧑🏿‍🏫", "🏳️‍🌈", "🏳️‍⚧️", "⚛︎", "🃑", "♻︎"

None of the following should be considered valid:

"F", ":custom-emoji:", "<custom-emoji>", "ᚱ", "ᘐ", "״"

apply

  • MUST be a UINT_8 Indicates whether the emoji should be applied and if so, its "strength".

Potential uses:

  • a single reaction
  • ratings
  • a range of responses, e.g. "strongly disagree" --> "strongly agree" = 1 --> 5 stars.
  • recommendation engines

Apply Enums

ValueNameDescription
0retractRemove the referenced emoji
napplyApply the referenced emoji N times

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation

inReplyTo

Non-Normative

Likes

Generic "likes" should default to the "❤️" or Unicode U+FE0F as the emoji in the reaction.

Profile Announcement

A Profile Announcement is a constrained version of a Broadcast Announcement. The reference content MUST be of profile type.

Fields

FieldDescriptionSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (5)decimalINT32no
contentHashkeccak-256 hash of content stored at URLhexadecimalBYTE_ARRAYYES
fromIdid of the user creating the AnnouncementdecimalUINT_64YES
urlprofile content URLUTF-8UTF8no

Field Requirements

announcementType

  • MUST be fixed to 5

contentHash

  • MUST be 32 bytes in length
  • MUST be the keccak-256 hash of the bytes of the reference at the URL

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation

url

  • MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
  • Resource MUST be a valid Profile Activity Content Type
  • MUST use one of the supported URL Schemes

Supported URL Schemes

SchemeDescriptionReferenceDSNP Version Added
HTTPSHypertext Transfer Protocol SecureRFC28181.0

Non-Normative

Most Recent Profile

When displaying a DSNP user's profile, the most recent profile should be considered the complete and correct version. Previous Profile Announcements from the same fromId may be disregarded.

Update Announcement

An Update Announcement is a way to note intent to update previously announced content. If the original Broadcast/Reply is Tombstoned, subsequent Updates should be ignored.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (6)enumdecimalINT32no
fromIdid of the user creating the announcement64 bit unsigned integerdecimalUINT_64YES
contentHashkeccak-256 hash of updated content32 byteshexadecimalBYTE_ARRAYYES
urlupdated content URLUTF-8UTF-8UTF8no
targetContentHashkeccak-256 hash of target content32 byteshexadecimalBYTE_ARRAYYES

Field Requirements

announcementType

  • MUST be fixed to 6

fromId

contentHash

url

  • MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
  • Resource MUST one of the supported Activity Content Types
  • MUST use one of the supported URL Schemes

targetContentHash

  • MUST be the contentHash of an allowed Announcement type with the same fromId as the Update Announcement

Update Allowed Announcement Types

ValueName
2Broadcast
3Reply

Serializations

Serialization is how the value should be stringified for signing and for transfer between systems. Most serializations use outside standards, but some require additional clarifications, provided here.

decimal

Used to represent integers. Strings are used to avoid issues with different implementations of numbers.

  • MUST use 0-9 representation
  • MUST NOT have spaces or separators
  • MUST be a string
InvalidWhyValid
0x123Must be decimal"291"
291Must be a string"291"
291nBigInt(291) serialization appends an n"291"

hexadecimal

Used to represent bytes.

  • MUST use 0-9,a-f representation
  • MUST be lowercase
  • MUST be prefixed with a 0x
  • MUST NOT have spaces or separators
  • MUST have two characters per byte in addition to the 0x characters
BytesInvalidValid
20x1230x0123
2123h0x0123
20x0ABC0x0abc
80xabc0x0000000000000abc
320x3e34c4325f4461b9355027b314f3eb56d31af549f7da7bd9ef1ce951651e0x00003e34c4325f4461b9355027b314f3eb56d31af549f7da7bd9ef1ce951651e

DSNP Implementations

Ethereum/EVM Compatible DSNP Implementation

Version 1.1.0

DSNP on Ethereum is designed using smart contracts and log messages. Smart contracts are used for identity and delegation. Log messages are used for announcement publishing via batches.

Solidity

All included Solidity interfaces are targeting the Solidity language version 0.8.x. Other versions may be available in the official contracts code repository.

Contracts and Interfaces

Official DSNP interfaces, contracts, and deployment information may be found in GitHub.

Libraries

NameLanguage(s)
LibertyDSNP/sdk-tsJavaScript/TypeScript

Releases

VersionDescriptionDSNP CompatibilityRelease DateChangelog
1.1.0DIP-150, DIP-1801.0.x, 1.1.x2022-05-06Changelog
1.0.0Initial Release1.0.x, 1.1.x2021-09-09Changelog

Identity

Purpose

  1. Provide the interface for a DSNP identity.
  2. Specify the delegation model and related interface.
  3. Specify the ownership model and related interface.
  4. Provide the list of EIPs that must be supported by a DSNP-compatible identity.

Details

Ownership

Ownership of an identity on Ethereum is determined by the address of the signer of the content.

Creation of any new identity MUST be authorized by the owner's address. The official Identity Factory is provided for initial creation of a new identity.

Creation of an identity on behalf of someone is described in the Identity Factory.

Permissioned Owners

Ownership is managed through using permissions. While at least one owner is required, additional public keys may be considered to have ownership and may remove or add other owners.

Delegation

Delegation allows adding additional public keys, in addition to the owner public key(s). These delegated keys are allowed to perform certain actions on behalf of the owner based on the delegate's "role".

Interface

/**
 * @dev DSNP Identity Interface for managing delegates
 */
interface IDelegation {

    struct DelegateAdd {
        uint32 nonce;
        address delegateAddr;
        Role role;
    }

    struct DelegateRemove {
        uint32 nonce;
        address delegateAddr;
    }

    /**
     * @dev Enumerated Permissions
     *      Roles have different permissions
     *      APPEND ONLY
     */
    enum Permission {
        /**
         * @dev 0x0 NONE reserved for no permissions
         */
        NONE,
        /**
         * @dev 0x1 Announce any DSNP message
         */
        ANNOUNCE,
        /**
         * @dev 0x2 Add new delegate
         */
        OWNERSHIP_TRANSFER,
        /**
         * @dev 0x3 Add new delegates
         */
        DELEGATE_ADD,
        /**
         * @dev 0x4 Remove delegates
         */
        DELEGATE_REMOVE
    }

    /**
     * @dev Enumerated Roles
     *      Roles have different permissions
     *      APPEND ONLY
     */
    enum Role {
        /**
         * @dev 0x0 NONE reserved for no permissions
         */
        NONE,
        /**
         * @dev 0x1 OWNER:
         *      - Permission.*
         */
        OWNER,
        /**
         * @dev 0x2 ANNOUNCER:
         *      - Permission.ANNOUNCE
         */
        ANNOUNCER
    }

    /**
     * @dev Log for addition of a new delegate
     * @param delegate Address delegated
     * @param role Permission Role
     */
    event DSNPAddDelegate(address delegate, Role role);

    /**
     * @dev Log for removal of a delegate
     * @param delegate Address revoked
     */
    event DSNPRemoveDelegate(address delegate);

    /**
     * @dev Add or change permissions for delegate
     * @param newDelegate Address to delegate new permissions to
     * @param role Role for the delegate
     *
     * MUST be called by owner or other delegate with permissions
     * MUST consider newDelegate to be valid from the beginning to time
     * MUST emit DSNPAddDelegate
     */
    function delegate(address newDelegate, Role role) external;

    /**
     * @dev Add or change permissions for delegate by EIP-712 signature
     * @param v EIP-155 calculated Signature v value
     * @param r ECDSA Signature r value
     * @param s ECDSA Signature s value
     * @param change Change data containing new delegate address, role, and nonce
     *
     * MUST be signed by owner or other delegate with permissions (implementation specific)
     * MUST consider newDelegate to be valid from the beginning to time
     * MUST emit DSNPAddDelegate
     */
    function delegateByEIP712Sig(
        uint8 v,
        bytes32 r,
        bytes32 s,
        DelegateAdd calldata change
    ) external;

    /**
     * @dev Remove Delegate
     * @param addr Address to remove all permissions from
     *
     * MUST be called by the delegate, owner, or other delegate with permissions
     * MUST store the block.number as the endBlock for response in isAuthorizedToAnnounce (exclusive)
     *
     * MUST emit DSNPRemoveDelegate
     */
    function delegateRemove(address addr) external;

    /**
     * @dev Remove Delegate By EIP-712 Signature
     * @param v EIP-155 calculated Signature v value
     * @param r ECDSA Signature r value
     * @param s ECDSA Signature s value
     * @param change Change data containing new delegate address and nonce
     *
     * MUST be signed by the delegate, owner, or other delegate with permissions
     * MUST store the block.number as the endBlock for response in isAuthorizedToAnnounce (exclusive)
     * MUST emit DSNPRemoveDelegate
     */
    function delegateRemoveByEIP712Sig(
        uint8 v,
        bytes32 r,
        bytes32 s,
        DelegateRemove calldata change
    ) external;

    /**
     * @dev Checks to see if address is authorized with the given permission
     * @param addr Address that is used to test
     * @param permission Level of permission check. See Permission for details
     * @param blockNumber Check for authorization at a particular block number, 0x0 reserved for endless permissions
     * @return boolean
     *
     * @dev Immutable assuming operating on confirmed blocks
     */
    function isAuthorizedTo(
        address addr,
        Permission permission,
        uint256 blockNumber
    ) external view returns (bool);

    /**
     * @dev Get a delegate's nonce
     * @param addr The delegate's address to get the nonce for
     *
     * @return nonce value for delegate
     */
    function getNonceForDelegate(address addr) external view returns (uint32);
}

Upgrade Interface

It is an open question if we want to require a standard set of upgrade interfaces.

Additional Required Interfaces

EIP 165

EIP 165 provides standard interface detection. It is required to support optional interfaces and upgrade expansion.

Additional Optional Interfaces

EIP 173

EIP 173 provides methods that confirm ownership and provide methods to transfer ownership. Implementations that choose to use this interface will need to consider that transferring ownership should revoke all existing delegations, or at a minimum all delegates at Role.OWNER.

EIP 897

EIP 897 is for DSNP identity contracts that are proxies such as those produced by the default Identity Factory.

EIP 1271

EIP 1271 allows the identity contract to validate a signature. To support delegation, this interface allows a contract to validate against the signature of the owner.

Unlike IDelegation.isAuthorizedToAnnounce which supports permission levels and end block number, isValidSignature will only respond to currently permissioned addresses at the owner level.

REMEMBER: Implementation MUST ONLY return true for active owner level.

Identity Requirements

InterfaceRequired
IDelegationRequired
EIP 165Required
EIP 897Proxy Contracts Only
EIP 173Optional
EIP 1271Optional

Rejected Solutions

  • Centralized smart contract maintaining identifiers and owners
    • Top Rejection Reasons:
      • A centralized contract must either be controlled by the foundation, a DAO, or non-upgradable
      • Does not allow for complete flexibility in ownership
  • Using a single public key as an identifier
    • Top Rejection Reasons:
      • Does not allow for flexibility in ownership
      • Completely dependent on off-chain software and processing for verification of delegation
  • Using a single owner model
    • A single owner (as well as the existing EIP 173 to manage it) was considered as opposed to the multi-owner role system
    • Top Rejection Reasons:
      • Does not allow for flexibility in ownership
      • Single-owner increases the likelihood of users using choosing less secure key management practices
      • Multi-owner pairs better with a permissioned system
  • Using a simple two-tier permission model
    • Owner level and "everything" else
    • Top Rejection Reasons:
      • Limits options for future specific roles such as social recovery
      • Cost savings were minimal when paired with a role based system

Identifiers

The identity registry is the source of building and maintaining a unique DSNP User Id.

Creating and Retrieving a DSNP User Id

  1. Create an entry in the Identity Registry contract.
  2. Use the DSNPRegistryUpdate log event or the resolveRegistration function to retrieve the uint64 id
  3. That id value is the DSNP User Id and can be used to generate the DSNP User URI.

Identity Factory

The least expensive way to create a new identity is through an identity factory. Official contracts will provide one or more of these standard interfaces to easily generate an identity with different upgrade paths.

Remember: Using a factory or even a proxy is just an optimization and NOT required. Any contract that matches the DSNP Identity interfaces is valid.

Purpose

  1. Describe how an Identity Factory can create an identity.
  2. Describe how an Identity Factory can allow someone else to pay for the creation an identity.
  3. Restrict the creation of identities without owner permission.

Assumptions

Proxy Contracts

While it is not required, most of the DSNP Identity compatible contracts are proxy contracts. Proxy contracts are often created through a factory contract. Here are the interfaces to be a DSNP-compatible identity factory.

What is a Proxy Contract?

Proxy contracts are used to limit the gas for deploying many contracts that all have the same logic, but need different state. The state is maintained at the "proxy" contract while the logic to alter the state is able to be in one "logic" contract.

Remember: A Logic Contract has 100% control over the state of a smart contract. While a logic contract cannot have state that effects the execution of a proxy contract, a logic contract's code can be written in such a way that allows for others to take control of a contract. Never use logic contracts that you do not trust!

What are the different types of Proxy Contracts?

OpenZeppelin has a great set of standard and audited proxy contracts.

While there may not be an identity factory interface for each type, the documentation from OpenZeppelin gives good detail on the differences between the types.

Can I switch from one type to another?

Switching types is possible, but difficult. See Identity Registry for more information on changing an identity contract to another type.

Data storage and EIP 1967

Due to the state management system that Ethereum uses, it can easily cause issues for upgradable contracts. EIP 1967 provides for ways to safely use state that will not collide. Implementations of upgradable proxies MUST use EIP 1967 style data storage.

Logic Contract Constraints

Contracts that are used as the logic for the proxy are not able to use constructors for initialization. Proxy contracts however can have constructors and additionally the factory can be used to call methods once the proxy is created. Remember that setting up the initial authorization state of a contract MUST be done in a single transaction to prevent others sniping the contract.

Factory

An identity factory will give easy methods to allow for the creation of proxy contracts that function as DSNP Identities. Official implementation contract addresses will be published once deployed.

Clone Interface

Clones follow EIP 1167 for a non-upgradable identity contract.

/**
 * @dev DSNP Identity Factory Interface for creating identities via [EIP 1167](https://eips.ethereum.org/EIPS/eip-1167)
 */
interface IIdentityCloneFactory {

    /**
     * @dev event to log the created proxy contract address
     */
    event ProxyCreated(address addr);

    /**
     * @dev Creates a new identity with the message sender as the owner
     * @dev [EIP 1167](https://eips.ethereum.org/EIPS/eip-1167) Proxy
     * @param logic The address to use for the logic contract
     *
     * @dev This MUST emit ProxyCreated with the address of the new proxy contract
     * @return The address of the newly created proxy contract
     */
    function createCloneProxy(address logic) public returns (address);

    /**
     * @dev Creates a new identity with the ecrecover address as the owner
     * @dev [EIP 1167](https://eips.ethereum.org/EIPS/eip-1167) Proxy
     * @param logic The address to use for the logic contract
     * @param owner The initial owner's address of the new contract
     *
     * @dev This MUST emit ProxyCreated with the address of the new proxy contract
     * @return The address of the newly created proxy contract
     */
    function createCloneProxyWithOwner(address logic, address owner) external returns (address);
}

Upgradable Proxy Interface

Upgradable Proxies can be upgraded by the owner or permissioned delegates.

/**
 * @dev DSNP Identity Factory Interface for creating upgradable identities
 */
interface IIdentityUpgradableFactory {

    /**
     * @dev event to log the created proxy contract address
     */
    event ProxyCreated(address addr);

    /**
     * @dev Logs updates to the suggested logic contract
     * @dev MUST BE emitted when the contract changes the suggested logic address
     * @param newLogic The new address
     */
    event LogicUpdated(address newLogic);

    /**
     * @dev This may be upgradable by the owner of the factory
     *
     * @return The current logic contract suggested by this factory
     */
    function getLogic() external view returns (address);

    /**
     * @dev Creates a new identity with the message sender as the owner
     *      and will be pointed at the default logic address.
     *
     * @dev This MUST emit ProxyCreated with the address of the new proxy contract
     * @return The address of the newly created proxy contract
     */
    function createUpgradableProxy() external returns (address);

    /**
     * @dev Creates a new identity with the message sender as the owner
     * @param logic The address to use for the logic contract
     *
     * @dev This MUST emit ProxyCreated with the address of the new proxy contract
     * @return The address of the newly created proxy contract
     */
    function createUpgradableProxy(address logic) external returns (address);

    /**
     * @dev Creates a new identity with the ecrecover address as the owner
     * @param logic The logic address to use for identity creation
     * @param owner The initial owner's address of the new contract
     *
     * @dev This MUST emit ProxyCreated with the address of the new proxy contract
     * @return The address of the newly created proxy contract
     */
    function createUpgradableProxyWithOwner(address logic, address owner) external returns (address);
}

Beacon Factory Interface

Beacon Proxies will use the beacon's logic address and will be upgraded when the beacon's logic address is changed. This is the suggested factory for use on Betanet to remain up to date.

/**
 * @dev DSNP Identity Factory Interface for creating beacon following identities
 */
interface IIdentityBeaconFactory {

    /**
     * @dev event to log the created proxy contract address
     */
    event ProxyCreated(address addr);

    /**
     * @dev This MUST NOT be upgradable by the owner of the factory
     *
     * @return The current beacon contract suggested by this factory
     */
    function getBeacon() external view returns (address);

    /**
     * @dev Creates a new identity with the message sender as the owner
     *      Uses the beacon defined by getBeacon()
     *
     * @dev This MUST emit ProxyCreated with the address of the new proxy contract
     * @return The address of the newly created proxy contract
     */
    function createBeaconProxy() external returns (address);

    /**
     * @dev Creates a new identity with the message sender as the owner
     * @param beacon The beacon address to use for logic contract resolution
     *
     * @dev This MUST emit ProxyCreated with the address of the new proxy contract
     * @return The address of the newly created proxy contract
     */
    function createBeaconProxy(address beacon) external returns (address);

    /**
     * @dev Creates a new identity with the ecrecover address as the owner
     * @param beacon The beacon address to use logic contract resolution
     * @param owner The initial owner's address of the new contract
     *
     * @dev This MUST emit ProxyCreated with the address of the new proxy contract
     * @return The address of the newly created proxy contract
     */
    function createBeaconProxyWithOwner(address beacon, address owner) external returns (address);

    /**
     * @dev Creates a new identity with the address as the owner and registers it with a handle
     * @param beacon The beacon address to use logic contract resolution
     * @param owner The initial owner's address of the new contract
     * @param handle The handle the new identity proxy under which should be registered
     *
     * @dev This MUST emit ProxyCreated with the address of the new proxy contract
     * @dev This MUST revert if registration reverts
     * @dev This MUST emit a DSNPRegistryUpdate
     */
    function createAndRegisterBeaconProxy(
        address beacon,
        address owner,
        string calldata handle
    ) external;
}

Beacon Interface

A beacon contract follows the same interface as the OpenZeppelin 4 IBeacon. Updating the beacon logic address is left to the implementation of the beacon, but the OpenZeppelin 4 UpgradeableBeacon is suggested.

/**
 * @dev This is the interface that {BeaconProxy} expects of its beacon.
 */
interface IBeacon {
    /**
     * @dev Must return an address that can be used as a delegate call target.
     *      This follows the interface from OpenZeppelin 4.0.0 [IBeacon](https://github.com/OpenZeppelin/openzeppelin-contracts/blob/v4.0.0/contracts/proxy/beacon/IBeacon.sol)
     *
     * @return A contract address that implements the logic for the proxy
     */
    function implementation() external view returns (address);
}

Identity Registry

A registry allows for distinct user identifiers and human readable discovery of network members. The DSNP User Id Registry is a simple contract that allows switching handles, identity contracts, and chain migration while maintaining all graph connections, public and private.

Purpose

  1. Describes how the Identity Registry resolves a DSNP User Id to an identity contract address.
  2. Describes how the Identity Registry allows for handle resolution.
  3. Presents the interface for the Identity Registry.
  4. Describes rejected alternatives.

Assumptions

  • Ids will need to be moved from Betanet to Mainnet.
  • Handles are for display and discovery purposes only.

Discovery via DSNP Handles

  • Contract addresses or numerical ids are not easy to remember.
  • Most networks rely on text based handles for discovery of users on a network.
  • DSNP handles are an easy way to allow easy user lookup.

Handles

  • Handles are simple UTF-8 strings.
  • No limitations are placed on length or contents, although different clients may not have support for the full unicode set.
  • A user MAY NOT register multiple handles that point to the same DSNP User Id.
  • Handles must be unique.

Homograph Attack Mitigation

UTF-8 support for handles opens handle users up to homograph attacks, not to mention case-sensitivity issues. This issue is of ongoing discussion both for the DSNP as well as in for ICANN domain names and other projects working with internationalization support.

Because the DSNP User Id is stable, attacks would only be successful in cases where the DSNP User Id were unknown. Punycode is used by some software to prevent homographs by encoding all non-Latin characters into Latin characters. So Punycode does not properly present non-Latin characters which isn't reaching the level of internationalization support desired by the DSNP.

Current Mitigation Strategies

Clients resolving handles MUST implement a method to detect potential homographs and check both user settings and potentially check the registry for additional potential matching DSNP User Ids.

EIP 712 Methods, Replay Attacks and Nonces

The Registry supports EIP 712 methods to permit a second party to pay gas costs for address and handle changes. Once an EIP 712 transaction is made, anyone may replay that action without further authorization. This breaks our security guarantees when the registration owner has made an additional change to either address or handle. To mitigate this, the Registry contract MUST store a nonce for every registration. When it receives an EIP 712 transaction, it MUST check that the nonce parameter matches the stored nonce, and it MUST increment the stored nonce if the transaction succeeds.

If a handle is changed, the registry MUST preserve the stored nonce for the old handle.

DSNP User Ids

Ethereum contract addresses are currently 160 bit values which is much larger than needed for unique identification. Identification can be reduced to just 64 bit identifiers with the registry and enable contract changing.

Remember: Only DSNP User Ids are safe for long term data connections.

  • DSNP Identity contract addresses are not guaranteed long term for upgrade and migration reasons.
  • Handles can change.

Resolutions

Resolutions are possible between any of the three pieces of data: Handle, DSNP User Id, and Contract Address. While a utility method is provided for ease of moving from handle to the contract address, the other resolutions require using contract log events.

Current Handle -> Current DSNP User Id and Contract Address

IRegistry.resolveRegistration provides easy access to the current registration for the given handle.

Current Handle -> Nonce

The IRegistry.resolveHandleToNonce method is the only method for discovery of the next nonce to use.

Other Lookups & Historical Values

The DSNPRegistryUpdate event is provided to resolve DSNP User Ids, handles, and contract addresses. The DSNP User Id and contract address are indexed in the event so use a log search using the event and the search data as topics.

A search by contract address may produce more than one result meaning that the contract address is currently or previously attached to other DSNP User Ids. To test for the current value, the query would need to be run again with each of the resulting DSNP User Ids retrieving the most recent DSNPRegistryUpdate event.

A search by DSNP User Id will retrieve the history of all handles and contract addresses that have been connected to that DSNP User Id. The most recent event (the one with the highest block number), will give the current handle and contract address for the given DSNP User Id.

Handles may be reused if a DSNP User Id changes to a new handle. While time consuming, discovering previous owners of a given handle requires locally filtering all DSNPRegistryUpdate events for events with the given handle.

EIP 721

DSNP support for the NFT standard EIP 721 for handles is under consideration.

NFT Concerns

  • The EIP 721 standard has its own ownership and permission system that is too limited for use across the DSNP.
  • Supporting two ownership systems adds needless complexity.
  • Identity contract ownership would require additional complexity to receive and transfer 721 tokens.

Rejected Alternatives

  • Why not the Ethereum Name Service (ENS)?
    • Overly complex and expensive for the simple use case of handles
    • Support can be added later through custom ENS resolution
    • Didn't provide a numerical id for efficient graph storage
    • Still subject to homograph attacks

Contract Interface

/**
 * @dev DSNP Registry Interface
 * @dev Suggested data storage implementation:
 *   uint64 internal currentIdSequenceMarker = 0x1; // Must not start at 0
 *   mapping(string => [id, address]) internal handleToIdAndAddress;
 */
interface IRegistry {
    struct AddressChange {
        uint32 nonce;
        address addr;
        string handle;
    }

    struct HandleChange {
        uint32 nonce;
        string oldHandle;
        string newHandle;
    }

    /**
     * @dev Log when a resolution address is changed
     * @param id The DSNP User Id
     * @param addr The address the DSNP User Id is pointing at
     * @param handle The actual UTF-8 string used for the handle
     */
    event DSNPRegistryUpdate(uint64 indexed id, address indexed addr, string handle);

    /**
     * @dev Register a new DSNP User Id
     * @param addr Address for the new DSNP User Id to point at
     * @param handle The handle for discovery
     *
     * MUST reject if the handle is already in use
     * MUST emit DSNPRegistryUpdate
     * MUST check that addr implements IDelegation interface
     * @return id for new registration
     */
    function register(address addr, string calldata handle) external returns (uint64);

    /**
     * @dev Alter a DSNP User Id resolution address
     * @param newAddr Original or new address to resolve to
     * @param handle The handle to modify
     *
     * MUST be called by someone who is authorized on the contract
     *      via `IDelegation(oldAddr).isAuthorizedTo(oldAddr, Permission.OWNERSHIP_TRANSFER, block.number)`
     * MUST emit DSNPRegistryUpdate
     * MUST check that newAddr implements IDelegation interface
     */
    function changeAddress(address newAddr, string calldata handle) external;

    /**
     * @dev Alter a DSNP User Id resolution address by EIP-712 Signature
     * @param v EIP-155 calculated Signature v value
     * @param r ECDSA Signature r value
     * @param s ECDSA Signature s value
     * @param change Change data containing nonce, new address and handle
     *
     * MUST be signed by someone who is authorized on the contract
     *      via `IDelegation(oldAddr).isAuthorizedTo(ecrecovedAddr, Permission.OWNERSHIP_TRANSFER, block.number)`
     * MUST check that newAddr implements IDelegation interface
     * MUST emit DSNPRegistryUpdate
     */
    function changeAddressByEIP712Sig(uint8 v, bytes32 r, bytes32 s, AddressChange calldata change) external;

    /**
     * @dev Alter a DSNP User Id handle
     * @param oldHandle The previous handle for modification
     * @param newHandle The new handle to use for discovery
     *
     * MUST NOT allow a registration of a handle that is already in use
     * MUST be called by someone who is authorized on the contract
     *      via `IDelegation(oldHandle -> addr).isAuthorizedTo(ecrecovedAddr, Permission.OWNERSHIP_TRANSFER, block.number)`
     * MUST emit DSNPRegistryUpdate
     */
    function changeHandle(string calldata oldHandle, string calldata newHandle) external;

    /**
     * @dev Alter a DSNP User Id handle by EIP-712 Signature
     * @param v EIP-155 calculated Signature v value
     * @param r ECDSA Signature r value
     * @param s ECDSA Signature s value
     * @param change Change data containing nonce, old handle and new handle
     *
     * MUST NOT allow a registration of a handle that is already in use
     * MUST be signed by someone who is authorized on the contract
     *      via `IDelegation(handle -> addr).isAuthorizedTo(ecrecovedAddr, Permission.OWNERSHIP_TRANSFER, block.number)`
     * MUST emit DSNPRegistryUpdate
     */
    function changeHandleByEIP712Sig(uint8 v, bytes32 r, bytes32 s, HandleChange calldata change) external;

    /**
     * @dev Resolve a handle to a DSNP User Id and contract address
     * @param handle The handle to resolve
     *
     * Returns zeros if not found
     * @return A tuple of the DSNP User Id and the Address of the contract
     */
    function resolveRegistration(string calldata handle) view external returns (uint64, address);

    /**
     * @dev Resolve a handle to a EIP 712 nonce
     * @param handle The handle to resolve
     *
     * rejects if not found
     * @return expected nonce for next EIP 712 update
     */
    function resolveHandleToNonce(string calldata handle) view external returns (uint32);
}

Announcement Publishing

On Ethereum, all Announcements are published via Batch Publication Files. Publishing is accomplished via an Ethereum Log Event.

Ethereum Log Event

The event topic for DSNPBatchPublication follows the standard Solidity event name to hash standard.

0xe63a4904ccacc079f71e52aad2cf99c00a7d4963566562a94d7c07610f1df576 = keccak-256("DSNPBatchPublication(int16,bytes32,string)")

Log Event Data

FieldDescriptionTypeIndexed
announcementTypeThe single announcement type in the given fileint16YES
fileHashkeccak-256 hash of the batch filebytes32no
fileUrlURL to retrieve the referenced batch file via an approved schemastringno

Batch File Retrieval

  • Batch File URLs MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890.
  • Batch File URLs MUST use one of the supported URL Schemes.

Supported URL Schemes

SchemeDescriptionReferenceDSNP Version Added
HTTPSHypertext Transfer Protocol SecureRFC28181.0

Ordering

The DSNPBatchPublication Ethereum events are ordered by information provided in the transaction.

  1. DSNPBatchPublication Block number ascending
  2. DSNPBatchPublication Transaction index ascending
  3. DSNPBatchPublication Log index ascending
  4. DSNP Standard: Order Announcements in a Batch Publication File by row appearance order

Publisher Contract Requirements

Contracts that allow for generating DSNPBatchPublication are called publishers. A standard interface is available for use.

InterfaceRequired
IPublishRequired
ERC165Optional

IPublish Interface

interface IPublish {
    struct Publication {
        int16 announcementType;
        string fileUrl;
        bytes32 fileHash;
    }

    /**
     * @dev Log Event for each batch published
     * @param announcementType The type of Announcement in the batch file
     * @param fileHash The keccak hash of the batch file
     * @param fileUrl A url that resolves to the batch file
     */
    event DSNPBatchPublication(int16 indexed announcementType, bytes32 fileHash, string fileUrl);

    /**
     * @param publications Array of Batch struct to publish
     */
    function publish(Publication[] calldata publications) external;
}

Announcement Validation

DSNP Announcements on Ethereum are validated at read time.

  1. Read DSNPBatchPublication events.
  2. Fetch and validate the Batch Publication.
  3. Validate the chain of delegation from the Announcement's fromId to the sender of the transaction with the DSNPBatchPublication event.

Batch Publication Validation

  1. Hash the batch publication file using keccak-256.
  2. Retrieve the published hash for the given file from the DSNPBatchPublication Event.
  3. The file hash MUST match the retrieved hash from the DSNPBatchPublication event.

Announcement Validation

  1. Get the Ethereum address of the sender for the transaction from the Batch (aka the Batch Publisher).
  2. Find the Identity Contract for the Announcement's fromId.
  3. Test the Batch Publisher's Ethereum address against the Identity Contract via IDelegation.isAuthorizedTo with the permission ANNOUNCE and the block number from the DSNPBatchPublication event.

Announcement Duplicates

Duplicate Announcements MUST be rejected or ignored.

Activity Content Specification

Version 1.1.0

Content references shared via the DSNP consists of URLs pointing to documents containing Activity Streams JSON objects. For the purposes of the DSNP, restrictions are placed on the Activity Streams 2.0 specification.

JSON-LD and Activity Streams

All DSNP Activity Content is compatible with the Activity Streams 2.0 specification. While there are some DSNP extensions, they are guaranteed to use non-colliding terms. Therefore, we set the JSON-LD @context field to https://www.w3.org/ns/activitystreams according to Activity Streams 2.0 §2.1.

Core Activity Content Types

DSNP uses only the following content types at the root level.

NameDescriptionDSNP Announcements
Notestandard user contentBroadcast, Reply
Profileuser profile contentProfile

Associated Types

NameDescriptionSpecification
Locationadd a location to contentActivity Vocabulary
Tagadd a tag to contentActivity Vocabulary
Attachmentssupported attachment typesActivity Vocabulary
Hashcontent validation hashDSNP Extension

Supported URL Schema

URLs in DSNP-compatible Activity Content MUST to use one of the following URL schemes.

SchemeDescriptionReferenceDSNP Version Added
HTTPSHypertext Transfer Protocol SecureRFC28181.0
HTTPHypertext Transfer ProtocolRFC26161.0

Libraries

NameLanguage(s)
LibertyDSNP/activity-contentJavaScript/TypeScript
LibertyDSNP/activity-content-javaJava/Kotlin
LibertyDSNP/activity-content-swiftSwift

Releases

VersionDescriptionRelease DateChangelog
1.1.0DIP-1582022-05-05Changelog
1.0.0Initial Release2021-09-09Changelog

Non-Normative

Additional Fields

Implementers may choose to support more of the Activity Streams standard as long as it does not conflict with this specification, but as a warning, other implementations may not recognize those additions. Implementers who extend their support for Activity Streams objects beyond the subset defined here do so at their own risk.

Activity Stream Type: Note

Activity Vocabulary: Note

PropertyBase SpecRequiredDescriptionRestrictions
@contextActivity Streams 2.0YESJSON-LD @contextMUST be set to https://www.w3.org/ns/activitystreams
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Note
contentActivity Vocabulary 2.0YESText content of the note
mediaTypeActivity Vocabulary 2.0YESMIME type for the content fieldMUST be set to a supported MIME type
publishedActivity Vocabulary 2.0YESThe time of publishingMUST be ISO8601
nameActivity Vocabulary 2.0noThe display name for the note
attachmentActivity Vocabulary 2.0noArray of attached links or mediaMUST be one of the Supported Attachments
tagActivity Vocabulary 2.0noArray of tags/mentionsMUST follow Tag Type
locationActivity Vocabulary 2.0noFor locationMUST follow Location Type

Supported Content MIME Types

FormatMIME TypeSpecification(s)
Plaintext/plain

Examples

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "content": "Hello world!",
  "mediaType": "text/plain",
  "published": "1970-01-01T00:00:00+00:00"
}
{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "content": "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.",
  "mediaType": "text/plain",
  "summary": "Lorem ipsum dolor sit amet, consectetur adipiscing elit.",
  "attachment": [
    {
      "type": "Link",
      "href": "https://en.wikipedia.org/wiki/Citation_needed"
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Activity Stream Type: Profile

Profiles are used to provide additional user information display.

Activity Vocabulary: Profile

PropertyBase SpecRequiredDescriptionRestrictions
@contextActivity Streams 2.0YESJSON-LD @contextMUST be set to https://www.w3.org/ns/activitystreams
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Profile
nameActivity Vocabulary 2.0noThe display name for the profile
iconActivity Vocabulary 2.0noAn array of avatars of the profileMUST follow Image Link Type
summaryActivity Vocabulary 2.0noUsed as a plain text biography of the profile
publishedActivity Vocabulary 2.0noThe time of publishingMUST be ISO8601
locationActivity Vocabulary 2.0noFor locationMUST follow Location Type
tagActivity Vocabulary 2.0noFor tags or mentionsMUST follow Tag Type

Examples

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Profile",
  "name": "John Doe",
  "summary": "John Doe is actually a small kitten. See pfp.",
  "icon": [
    {
      "type": "Link",
      "href": "https://placekitten.com/256/256",
      "mediaType": "image/jpeg",
      "width": "256",
      "height": "256"
    },
    {
      "type": "Link",
      "href": "https://placekitten.com/64/64",
      "mediaType": "image/jpeg",
      "width": "64",
      "height": "64"
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Associated Type: Hash

NOT part of the Activity Streams 2.0 Vocabulary.

Activity objects linking to external content such as audio, image or video files must include a "hash" field for users to validate linked content. The value of this "hash" field must be an array of objects representing multiple hashes. AT LEAST ONE hash in the array MUST be one of the supported algorithms although others may also be used.

PropertyRequiredDescriptionRestrictions
algorithmYESThe algorithm of the given hash
valueYESHash value serialization

Supported Algorithms

AlgorithmDescriptionValue SerializationSpecification(s)
keccak256keccak-256 hashhexadecimalThe Keccak SHA-3 submission v3

Example

{

  "hash": [
    {
      "algorithm": "keccak256",
      "value": "0x1234567890ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF"
    },
    {
      "algorithm": "Ripemd256",
      "value": "0x96a9209006748794193d1811ef2dd5f447782b8b1635841165bc031bb3db64da"
    }
  ]
}

Associated Type: Location

Activity Vocabulary: location

PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Place
nameActivity Vocabulary 2.0YESThe display name for the location
accuracyActivity Vocabulary 2.0noThe accuracy of the coordinates as a percentage. (e.g. "94.0" means "94.0% accurate")
altitudeActivity Vocabulary 2.0noThe altitude of the location
latitudeActivity Vocabulary 2.0noThe latitude of the location
longitudeActivity Vocabulary 2.0noThe longitude of the location
radiusActivity Vocabulary 2.0noThe area around the given point that comprises the location
unitsActivity Vocabulary 2.0noThe units for radius and altitude (defaults to meters)MUST be one of: cm, feet, inches, km, m, miles

Warning

Location data may pose a privacy danger to users. Users should be warned before publishing location data.

Example

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "content": "I'm in NYC!",
  "mediaType": "text/plain",
  "location": {
    "type": "Place",
    "name": "New York City, NY",
    "latitude": "40.73",
    "longitude": "-73.93",
    "accuracy": "94.0"
  },
  "published": "1970-01-01T00:00:00+00:00"
}

Associated Type: Tag

Activity Vocabulary: tag

Hashtag

PropertyBase SpecRequiredDescriptionRestrictions
nameActivity Vocabulary 2.0YESThe text of the tag

Example

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "content": "I love the #dsnp spec!",
  "mediaType": "text/plain",
  "tag": [
    {
      "name": "#dsnp"
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Mention

PropertyBase SpecRequiredDescriptionRestrictions
nameActivity Vocabulary 2.0noThe text of the tag
typeActivity Vocabulary 2.0YESIdentifies the tag as type MentionMUST be Mention
idActivity Vocabulary 2.0YESLink to the user mentionedMUST be a DSNP User URI

Example

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "content": "@sally should check out the #dsnp spec!",
  "mediaType": "text/plain",
  "tag": [
    {
      "name": "#dsnp"
    },
    {
      "name": "@sally",
      "type": "Mention",
      "id": "dsnp://0x12345678"
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Associated Attachments

Audio

Activity Vocabulary: Audio

PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Audio
urlActivity Vocabulary 2.0YESAn array of links for given audio content in different formatsMUST be an Audio Link AND MUST have at least one supported audio MIME type
nameActivity Vocabulary 2.0noThe display name for the audio file
durationActivity Vocabulary 2.0noApproximate duration of the audio
PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Link
hrefActivity Vocabulary 2.0YESThe URL for the given audio contentMUST be a Supported URL Schema
mediaTypeActivity Vocabulary 2.0YESMIME type of href contentMUST follow
hashDSNP 1.0YESArray of hashes for linked content validationMUST include at least one supported hash

Supported Audio MIME Types

FormatMIME TypeSpecification(s)
MP3audio/mpegRFC3003
OGGaudio/oggRFC5334
WebMaudio/webmWebM standard

Example

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "attachments": [
    {
      "type": "Audio",
      "name": "The Scream",
      "url": [
        {
          "type": "Link",
          "href": "https://upload.wikimedia.org/wikipedia/commons/d/d9/Wilhelm_Scream.ogg",
          "mediaType": "audio/ogg",
          "hash": [
            {
              "algorithm": "keccak256",
              "value": "0x3b33df3d163e86514e9041ac97e3d920a75bbafa8d9c1489e631897874b762cc"
            }
          ]
        }
      ],
      "duration": "1S"
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Image

Activity Vocabulary: Image

PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Image
urlActivity Vocabulary 2.0YESAn array of links for given image content in different formatsMUST be an Image Link AND MUST have at least one supported image MIME type
nameActivity Vocabulary 2.0noThe display name or alt text for the image
PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Link
hrefActivity Vocabulary 2.0YESThe URL for the given imageMUST be a Supported URL Schema
mediaTypeActivity Vocabulary 2.0YESMIME type of href content
hashDSNP 1.0YESArray of hashes for linked content validationMUST include at least one supported hash
heightActivity Vocabulary 2.0noA hint as to the rendering height in device-independent pixels
widthActivity Vocabulary 2.0noA hint as to the rendering width in device-independent pixels

Supported Image MIME Types

FormatMIME TypeSpecification(s)
JPEGimage/jpegRFC2045
PNGimage/pngW3C PNG Standard
SVGimage/svg+xmlW3C SVG standard
WebPimage/webpWebP standard
GIFimage/gifRFC2045

Example

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "attachments": [
    {
      "type": "Image",
      "name": "One of the founders of DSNP",
      "url": [
        {
          "type": "Link",
          "href": "https://upload.wikimedia.org/wikipedia/commons/a/ae/Mccourt.jpg",
          "width": 350,
          "height": 228,
          "mediaType": "image/jpg",
          "hash": [
            {
              "algorithm": "keccak256",
              "value": "0x90b3b09658ec527d679c2de983b5720f6e12670724f7e227e5c360a3510b4cb5"
            }
          ]
        }
      ]
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Video

Activity Vocabulary: Video

PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Video
urlActivity Vocabulary 2.0YESAn array of links for given video content in different formatsMUST be a Video Link AND MUST have at least one supported video MIME type
nameActivity Vocabulary 2.0noThe display name for the video
durationActivity Vocabulary 2.0noApproximate duration of the video
PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Link
hrefActivity Vocabulary 2.0YESThe URL for the given contentMUST be a Supported URL Schema
mediaTypeActivity Vocabulary 2.0YESMIME type of href content
hashDSNP 1.0YESArray of hashes for linked content validationMUST include at least one supported hash
heightActivity Vocabulary 2.0noA hint as to the rendering height in device-independent pixels
widthActivity Vocabulary 2.0noA hint as to the rendering width in device-independent pixels

Supported Video MIME Types

FormatMIME TypeSpecification(s)
MPEGvideo/mpegRFC2045
OGGvideo/oggRFC5334
WebMvideo/webmWebM standard
H265video/H265RFC7798
MP4video/mp4RFC4337

Example

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "attachments": [
    {
      "type": "Video",
      "name": "One of the founders of DSNP",
      "duration": "PT10M32S",
      "url": [
        {
          "type": "Link",
          "href": "https://upload.wikimedia.org/wikipedia/commons/c/c0/Big_Buck_Bunny_4K.webm",
          "width": 4000,
          "height": 2250,
          "mediaType": "video/webm",
          "hash": [
            {
              "algorithm": "keccak256",
              "value": "0xf841950dfcedc968dbd63132da844b9f28faea3dbfd4cf326b3831b419a20e9a"
            }
          ]
        }
      ]
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Activity Vocabulary: Link

PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Link
hrefActivity Vocabulary 2.0YESThe URL for the given linkMUST be a Supported URL Schema
nameActivity Vocabulary 2.0noThe display name for the link

Examples

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "content": "Check out the DSNP Website!",
  "mediaType": "text/plain",
  "attachment": [
    {
      "type": "Link",
      "name": "DSNP Website",
      "href": "https://dsnp.org"
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Glossary

  • Activity Content - The linked content connected to an "Announcement"
  • Announcement - An item of data, typically an image or note, posted to the blockchain via a "Batch Publication"
  • Bad actor - A user intentionally using the application for bad-faith or illegal purposes
  • Batch Publication - A collection of Announcements
  • Bot - An automated account, sometimes malicious but often providing some service ("Hey Alexa, what's the weather?")
  • Consumer - Someone who reads content from social media
  • Contract address - The unique number associated with a smart contract posted on the blockchain
  • Hash Collision - The vanishingly unlikely possibility of a hash providing the same output for two different inputs
  • Hash function - A cryptographic function whose output is effectively unique for any given input without any information from the input being accessible in the output
  • Hash - A string of characters generated from a hash function, see "Hash function"
  • Identity Contract - The smart contract used to store a user's delegated public addresses and permissions, see "Contract address"
  • Publisher - The contract that posts new content from a publisher to the blockchain through batch publications
  • Store - A means of keeping messages or data for an extended period of time
  • User - A human using our application. See article for an exception

DRAFT: Archivists Overview

The job of an Archivist is to permanently store Batch content and DSNP Announcements in a format that is easily validated and retrieved.

The Archivist must be able to access chain data and all DSNP Content.

Validations that Archivists could perform

  • Signature validation - proof that author provided a real signature
  • Signature authentication - proof that the From address is the signer, or that signer is a valid delegate (proof of authorship)
  • There is retrievable content at the URL given in the DSNP message
  • The content hash is valid - that is, the URL serves the claimed data

All signatures for the announcement are included in the batch regardless of how the signature was requested (or not)

Archive Storage Format

Specify the off-chain Archivist storage format.

Assumptions

  • Chain messages are on Ethereum.
  • Message data is posted via Ethereum log events.
  • Signature algorithm is secp256k1. This allows the use ecreover to get public keys. A public key also need not be included in a log event for ease of validation.
  • Content hashes are created via the same keccak-256 hashing algorithm used by Ethereum.

Archive Entry

An archive entry is a combination of data in a DSNP message and from the block in which it is included. It is a key-value map consisting of the following fields:

field namedescriptiontype
dsnpTypeDSNP message typenumber/enum. see DSNP Message Types
dsnpDataDSNP message datasee fields in DSNP Messages
signatureslist of signatures for this messagearray of Signatures

dsnpType

  • number
  • Indicates what type of message this is, useful for indexers and filters.

dsnpData

  • varies
  • This can be encrypted where appropriate. The decrypted, fully deserialized version must be one of the types described in DSNP Messages.

signatures

  • array
  • all the signatures applied to this message at the time of archival.

Batch

A Batch is data that is referenced by a Batch Announcement. It consists of one or more ArchiveEntries.

fielddescriptiontype
archivesa set of ArchiveEntriesmap[ArchiveEntry]
batchIDkeccak-256 hash of content stored at URLbytes32
blockHeightthe block number this message was included innumber
fromAddresssocial identity of batch announcer,i.e. message senderbytes
logIndexthe index within the logs of this messagenumber
signatureannouncer's signatureSignature
transactionIndexthe index of the transaction this message is associated withnumber
urlthe location of this archivestring

archives

The set of ArchiveEntries is a key-value map, with the key being the archiveEntryID, which is a:

  • bytes32
  • The keccak-256 hash of all of the Archive Entry fields in a keccak-256 hash with the archiveEntryID field being blank.

batchID

  • bytes32
  • the keccak-256 hash of content stored at the URL referenced in this batch.

blockHeight

  • number
  • The block in which this DSNP Message is included.

fromAddress

the social identity of the batch announcer, i.e. the message sender.

logIndex

  • number
  • The log index in which this DSNP Message is included

signature

  • Signature (see below)
  • The signature of this batch announcer

transactionIndex

  • number
  • The transaction index in which this DSNP Message is included

url

  • string
  • The permanent URL address where this archive is stored.

Signature

A Signature consists of two fields:

  • signature - A secp256k1 signature
  • result - Optional, bytes. A result of an operation performed. For example, if a signing entity wished to prove that they had performed some sort of validation or analysis on the message, they would put the result of the analysis in this field. It could be a meaningful number or string, some sort of proof hash, etc.

Diagram

Archive Messages Diagram