DSNP Specification

Welcome to the Decentralized Social Networking Protocol (DSNP) specification. Here you can find the detailed specification documentation for DSNP, official DSNP system specifications, and associated specifications.

Goals & Purpose

Free communication among users on the Internet faces a variety of problems in the modern day. These challenges include censorship by state and corporate actors, the amplification of misinformation through viral content, and an ever-shrinking collection of near monopolies with absolute power over social interaction in the twenty-first century. Through DSNP, we hope to mitigate and ideally solve these challenges in the way social interaction operates online.

How to Read This Specification

There are three core specifications and one system specification currently. Each specification is divided into several modules. Each module describes a mostly self-contained aspect of the specification.

Core Specifications

NameVersionDescription
DSNP1.3.0The system-agnostic DSNP specification
Activity Content1.3.0A specification for DSNP-referenced content (subset of W3C Activity Streams)
Verifiable Credentials1.3.0Use of W3C Verifiable Credentials and DIDs with DSNP

System Specifications

Consensus SystemDSNP System Specification
FrequencyDSNP Over Frequency

Versioning

DSNP specification versions follow Semantic Versioning 2.0 for releases.

Contributions

Development occurs on GitHub. All interactions must follow the Code of Conduct and Contribution Guidelines.

Learning More

In addition to this document, more resources regarding the project can be found at DSNP.org, including the blog, forum, code repositories, and other supporting software libraries and examples.

DSNP Specification

Version 1.3.0

DSNP (Decentralized Social Networking Protocol) is a social networking protocol designed to run on a blockchain. It specifies a set of social primitives along with requirements for interoperability. Go to Systems for specifics on how DSNP is expressed on a specific blockchain.

Overview

What is a DSNP System?

A DSNP system is a state machine which generates an ongoing, publicly observable and verifiable stream of state change records in response to public input. This specification defines a set of operations (the DSNP Operations) that a compliant DSNP system MUST make available. For each DSNP Operation, the DSNP specification defines the data to be provided as input, the data that will be generated as output, and the state change records (the DSNP State Change Records) that will be observable as a result of the Operation, depending on the input and prior state of the system.

The nature of blockchain technology means that all DSNP Operations are potentially asynchronous; that is, DSNP State Change Records need not be created immediately upon invoking a DSNP Operation, and may appear in the stream of state changes at any subsequent time. A DSNP Operation can therefore only be confirmed to have completed successfully by observing the stream of newly-generated DSNP Records.

All DSNP Records MUST be publicly visible. On a blockchain, for example, all nodes must share a common set of Records, and a DSNP system specification MUST define how external systems can subscribe to or otherwise view all state changes. In practical terms, this means that all users of a given DSNP system exist conceptually within a single shared social graph, and a given user MUST have the ability at a protocol level to see and participate in all public discourse. Note, however, that this is not an application-level requirement; an application that interacts with DSNP might choose to filter what content is shown.

Finally, to ensure decentralization, a DSNP system MUST avoid having any single point of failure (many nodes) and MUST avoid having any single entity that can override its consensus mechanisms (many operators).

DSNP System Compliance

Compliant DSNP system specifications MUST document how each of the required DSNP Operations (for a given version of the DSNP specification) can be invoked, including, for example, what wire protocols should be used for discovery, authentication and operation execution, and how inputs and outputs will be serialized.

A compliant specification MUST specify a mapping from its system-specific state change data (for example, the events emitted by a blockchain) to the DSNP State Change Records that data represents.

Releases

DSNP system specifications MUST specify the version(s) of the DSNP specification that may be used.

VersionDescriptionRelease DateChangelog
1.3.0DIP-257, DIP-263, DIP-267, DIP-2732024-09-17Changelog
1.2.0DIP-210, DIP-216, DIP-220, DIP-225, DIP-226, DIP-227, DIP-2282023-04-11Changelog
1.1.0DIP-148, DIP-149, DIP-150, DIP-165, DIP-1802022-05-06Changelog
1.0.0Initial Release2021-09-09Changelog

Identity

This specification is intended to cover the concept of identity and how identity is represented within the protocol.

Identifiers

DSNP Identifiers form the basis for pseudo-anonymous streams of content. While some users may choose to link or expose their real-world identity, DSNP implementations MUST NOT require such data exposure for account creation. The social graph is formed using this Identifier so that a user’s connections maintain integrity regardless of changes in any user’s client choices or access changes.

Pseudo Anonymity

  • An Identifier MUST default to being disconnected from a real-world identity.
  • An Identifier MUST be unique to the implementation.

Ownership

A user’s ownership of their identity is expressed via ownership and control of their pseudo-anonymous Identifier(s). Control entails the power to invoke DSNP Operations including publishing Announcements that create, update and delete (Tombstone) content associated with the Identifier, delegating these powers to others, managing keys associated with the Identifier, and retiring the Identifier.

Key Management

An initial control key must be created in order to acquire an Identifier. Each distinct Identifier MUST have distinct control keys; that is, the same key MUST NOT be linked to multiple Identifiers. Optionally, an implementation MAY allow the user to add and remove additional keys. An implementation MUST NOT allow the user to remove the only or last remaining control key.

Retirement

A user may choose to retire their Identifier at any time. Once an Identifier is retired, an implementation MAY remove all state data associated with that Identifier, provided that an indication that the Identifier is retired remains, so it may not be reused in the future. This means that all data previously sent from the Identifier, the keys associated with the Identifier, any user data (see below) associated with the Identifier, and the delegations (see next section) associated with the Identifier may be removed.

After an Identifier is retired, any existing or future Announcements from the Identifier should be treated as if they have been tombstoned (for Announcement Types that support tombstoning). A retired Identifier MUST NOT be allowed to act as a principal for any additional DSNP Operations.

Delegation

  • A user MUST be able to delegate and revoke permissions to other parties to submit each type of announcement or replace each type of user data on their behalf.
  • A user MUST be able to revoke delegated permissions.
  • Announcements and user data updates from a delegate MUST be able to be verified as to which delegate was responsible for the action.
  • Delegation revocation MUST NOT be retroactive.

Identity-Linked State Data

DSNP identities can have linked User Data, which, like the identifier and delegations, forms part of the shared state of the consensus system. As consensus system state storage is generally much more expensive than external data storage, shared state storage is only required for data considered critical for maintaining user agency and discoverability with low latency and high availability.

User data is utilized for the following types of identity-linked data:

  • Public keys that can be used to encrypt data that can later be decrypted by the user or used to verify signatures on documents created by the user
  • Social Graphs, both public and private (encrypted)
  • Links to Profile documents

Non-normative

Retroactive Revocation of Delegation

There are times when one might desire retroactive revocation of delegation, if for example, a key were found to have been compromised at an earlier time. However, retroactive revocation is much more difficult from a caching and performance perspective. Instead reverting any undesirable Announcements via Tombstones allows a user to choose which specific events need reverting and then notify the network of that change.

To this end, any Batch Publications can always be validated from the perspective of the time they were published, and do not require re-validation at future read times. The validity of a Batch is thus immutable.

  1. Collect all the DSNP Ids used in a Batch.
  2. Validate that each DSNP Id had delegated to the publisher of the Batch at the time of publishing.
  3. Validate that any failures from step 2 were from DSNP Ids that revoked delegation within the acceptance window of prior blocks.
  4. Record the batch as valid or invalid.

Identifiers

The DSNP Identifiers form the basis for pseudo-anonymous streams of content. Graph connections are formed through the DSNP User Id.

DSNP User Id

  • 64-bit Unsigned Integer
  • MUST be serialized as decimal
  • MUST be unique per implementation

DSNP Content Hash

  • MUST be a multibase string using the base32 encoding
  • MUST represent a valid multihash encoding of the hashing algorithm output for the bytes of the content
  • MUST use a Supported Hashing Algorithm

Serialization Steps

  1. Apply the Supported Hashing Algorithm to create a digest of the content.
  2. Prepend the leading bytes from the table below indicating the hashing algorithm in the multicodec table and the length of the hash output.
  3. Serialize as a base32 multibase string.

Example

  1. Applying the BLAKE3 algorithm to the DSNP Whitepaper yields the following 32 bytes: 0x3a0393e3ee6c6fec1b13885763225fd0927884b2d431ed262899523ade281cb4.
  2. Prepending the multihash indicator (0x1e for blake3) and hash length (0x20 for 32 bytes) gives 0x1e203a0393e3ee6c6fec1b13885763225fd0927884b2d431ed262899523ade281cb4.
  3. Serializing as a base32 multibase string gives us the final DSNP Content Hash of bdyqdua4t4pxgy37mdmjyqv3dejp5betyqsznimpneyujsur23yubzna.

Supported Hashing Algorithms

AlgorithmMultihash NameLeading bytesReferenceDSNP Version Added
SHA-256sha2-2560x1220RFC 62341.2
BLAKE3blake30x1e20blake3.io1.3

DSNP Protocol Scheme

  • MUST always be the string dsnp://

DSNP User URI

The DSNP User URI consists of two parts: the scheme and the user id. It is used to identify a user via a URI.

Example

dsnp://1311768467294899700
partvalue
Schemedsnp://
User Id1311768467294899700

DSNP Content URI

The DSNP Content URI consists of three parts: the scheme, the user id, and the content hash. It is used to uniquely identify an Announcement from a given user with content.

Any Announcement Types with a fromId and contentHash have a DSNP Content URI. When encoding a DSNP Content URI, the contentHash field MUST be serialized exactly as it appears in the Announcement (that is, as a base32 multihash string).

Example

dsnp://78187493520/bdyqdua4t4pxgy37mdmjyqv3dejp5betyqsznimpneyujsur23yubzna
partvalue
Schemedsnp://
User Id78187493520
Content Hashbdyqdua4t4pxgy37mdmjyqv3dejp5betyqsznimpneyujsur23yubzna

Graph

This specification describes the social network graph and how it is represented in the protocol.

In this context a social graph means a graph that represents social relationships between entities.

Nodes

Users are represented in the DSNP graph by nodes uniquely identified by their DSNP User Ids. Relationships between users are the graph edges.

Edges

Graph edges that originate from a user are stored as DSNP User Data associated with a given user. DSNP user data structures capture three distinct sets of relationships, as described below.

Public Follows

Each public follow records a unidirectional relationship where one user (the controller of the graph) is interested in following the public speech of another user. When follows are public, they are visible to any other user or system, including the user being followed.

Private Follows

A private follow records an interest in following the public speech of another user, but the data is stored in encrypted form. The relationship is not visible to the followed user or any other user that does not have the decryption key.

Private Connections

Private connections represent one half of a bidirectional relationship between two DSNP Users. Private connections are stored on both sides of the bidirectional relationship between two DSNP Users. Each user can then update the status of the relationship without loss of privacy. Each DSNP user maintains and controls two sets of data:

  1. An encrypted data set that records which users are recognized as connections.
  2. A public data set that is used to signal connection status via Pseudonymous Relationship Identifiers.

Pseudonymous Relationship Identifiers

Private connections are complemented by publishing private connection Pseudonymous Relationship Identifiers (PRIds), which provide a means of making the connection relationship visible to the other user while maintaining privacy with respect to any third parties. Applications are encouraged to verify connection relationships by checking to see that they are reciprocated, with the appropriate identifier present in both users’ data.

For each pair of users Alice and Bob, and a specified PRId context, a pair of PRIds can be generated from Alice and Bob’s shared secret, with one representing a simplex channel from Alice to Bob, and the other representing the corresponding simplex channel from Bob to Alice.

PRIds have the following properties:

  • If Alice has generated a PRId from Alice to Bob, only Bob can generate the same PRId. This allows Bob to check if he is listed in Alice’s published connections.
  • Bob can prove to a third party that a given PRId in Alice’s published connections is his, without allowing the third party to discover Alice or Bob’s relationships with any other user.

A detailed discussion of the cryptography for PRIds can be found in the PRId object definition.

User Data

DSNP stores user-related data of well defined formats associated with a user’s DSNP Identifier. DSNP User Data is designed for data with the following characteristics:

  • The data is intended to be accessible across the network with the strongest possible durability guarantees
  • The data may change frequently.
  • The data does not need to be widely announced and instead can be read just-in-time for a particular user.

Data may be public or stored in encrypted form. The Avro schema specification is used to define the binary representation of relevant data types.

User Data Types

DSNP implementations MUST support the following User Data Types:

System NameVersionEncryption AlgorithmCompression CodecAvro Object Type
publicFollows1.2NONE
DEFLATEGraphEdge
privateFollows1.2curve25519xsalsa20poly1305
DEFLATEGraphEdge
privateConnections1.2curve25519xsalsa20poly1305
DEFLATEGraphEdge
privateConnectionPRIds1.2NONENONEPRId
keyAgreementPublicKeys1.3NONENONEPublicKey
assertionMethodPublicKeys1.3NONENONEPublicKey
profileResources1.3NONENONEProfileResource

Data for each data type is initially formatted as a stream of Avro objects that should conform to the schema specified. A DSNP system MAY limit the number of objects allowed for a given user data type; if so, this MUST be documented. Avro file- and block-level information (including in-stream schema) is omitted. The Avro stream is then compressed and/or encrypted as specified.

curve25519xsalsa20poly1305 (that is, X25519 key exchange, XSalsa20 encryption, and Poly1305 message authentication) is the default authenticated encryption algorithm used in the NaCl (“Salt”) library, and its successor libsodium. In the specification of cryptographic operations below, relevant methods from these libraries are noted. While these specific libraries are not required for DSNP compatibility, they are highly recommended.

Data Chunks

Because consensus systems often have specific limits to the amount of data that can be included in a given transaction, operations on user data deal with the data in discrete chunks. As implementation strategies may vary, implementations MUST define their own maximum chunk size in bytes to be used in the operations described below.

Entity Tags

To allow for application-layer caching and prevent race conditions between different applications attempting to update the same data, an implementation MUST provide a non-empty etag (entity tag) value with each chunk fetched, and update this for each chunk replaced.

An etag is any ASCII-encoded string with a minimum length of one byte and a maximum length of 255 bytes.

An implementation MUST fail the Replace User Data Operation if the etag values supplied do not match the current etag values for all chunks of the specified data type.

Replace User Data Operation

The Replace User Data Operation takes the following parameters:

  • A DSNP User Id
    • Implementations MUST ensure that the principal invoking this Operation is this user, or a transparent chain of delegation from the user to the principal exists.
  • The index of the keyAgreementPublicKeys key pair used to encrypt any private data in the operation. (If only unencrypted user data types are included, the key index may be omitted.)
  • A map containing the set of data types to update as the keys, and tuples consisting of (1) the schema version used to encode the data type, and (2) a list where each element includes a data chunk and its associated entity tag, as the values.

If the Operation is successful, any previous data associated with the user for each data type included in the input MUST be removed and replaced by the new data.

Data chunks should be generated for each included data type using the following sequence of operations:

  1. Serialize the data records, in Avro binary format, according to the versioned schema.
  2. Divide the data records into one or more chunks, with each chunk containing a maximum byte count specified by the implementation.
  • Note that the combined chunks constitute a different, and smaller, binary than a complete Avro file, as they exclude the header information, schema, and any block-level metadata. Each chunk should start and end with a complete Avro object; in other words, chunk boundaries MUST NOT occur mid-record. Applications SHOULD try to include as many records as possible within the byte length limit, but because compression and encryption may alter the byte size versus the raw Avro binary records, they may wish to use a heuristic approach that sometimes produces non-optimal chunks.
  1. For each chunk generated, the application should then:
    1. If the data type requires compression, apply the compression codec noted.
    2. If the data type requires encryption,
      1. Retrieve the user’s active keyAgreementPublicKey key, Upublic, and note its index. If no key exists, one should be created and published as User Data before invoking the Operation. By convention, the key with the highest index (the last object in the Avro stream) is the active key.
      2. Create a sealed box (a payload encrypted with a symmetric key derived from an ephemeral key pair, and accompanied by the ephemeral public key), as in the libsodium function crypto_box_seal, using Upublic.
      3. Include the previous etag value for the chunk. If the chunk is new, etag should be set to null. If any chunks are to be deleted, they should be included in the input identified with the existing etag and a null value for the data.

If the Operation is invoked successfully the implementation MUST synchronously return a new set of etag values for each data type replaced, corresponding to the updated state of the data (with new chunks added and deleted chunks removed). Applications should not interpret this response as an indication that the operation was completed and a state change record emitted, as this typically occurs asynchronously. However, this strategy allows applications to make the optimistic assumption that, in due course, the DSNP system will reflect the intended changes, without needing to wait for asynchronous confirmation.

If, on the other hand, an invocation of Replace User Data is rejected due to entity tag discrepancies, this indicates that the relevant data on the network has changed since the entity tags were acquired by the application, and the application should fetch the most current version with the Get User Data Operation, reapply any intended changes, and retry the operation.

The Replace User Data Operation MUST generate a User Data Replaced Record containing the DSNP User Id and the set of updated User Data Types (but not the data itself). If the implementation detects that no change has occurred, it SHOULD omit the relevant unchanged data types from the state change record.

Examples

The following section is non-normative. The JSON schema and encoding used is provided for illustration only and implementations are free to define their own encoding.

The following example illustrates the input to the Replace User Data Operation corresponding to the following scenario (utilizing social graph data types):

  • The user’s previously stored data consists of one chunk for public follows, two chunks for private connections, and one chunk for private connection declarations.
  • The user adds several follows to their public list, causing it to exceed the maximum capacity for a single chunk and require a new chunk.
  • The user adds a relationship to their private connections, but it still fits in two chunks. The private connection’s PRId is added to the private connection PRId list, but it remains a single chunk.
{
  "publicFollows": {
    "version": "1.2",
    "chunks": [
      {
        "etag": string                             // unchanged chunk
      },
      {
        "data": base64(compress(chunk2)),
        "etag": null,                              // new chunk
      }
    ]
  },
  "privateFollows": {
    "version": "1.2",
    "chunks": [
      {
        "etag": string                             // unchanged chunk
      },
      {
        "data": base64(encrypt(compress(chunk2))), // updated chunk
        "etag": string
      }
    ]
  },
  "privateConnectionPRIds": {
    "version": "1.2",
    "chunks": [
      {
        "data": base64(compress(chunk1)),
        "etag": string
      }
    ]
  }
}

Deletion of records may cause situations where the number of chunks decreases. To ensure that the deleted chunk was up to date, the deleted chunk should still be included in the array with the relevant entity tag value, using an explicit null value for the data field to indicate deletion, as in the following snippet:

  "publicFollows": {
    "version": "1.2",
    "chunks": [
      {
        "etag": string                     // unchanged chunk
      },
      {
        "data": null,                      // deleted chunk
        "etag": string,
      }
    ]
  }

Get User Data Operation

The Get User Data Operation takes the following parameters:

  • The DSNP User Id of the user who controls the data
    • Note: While writing user data is reserved for the user and any delegates, anyone on the network can read any user’s data (though it may be encrypted).
  • The User Data Types (by system name) that should be retrieved.

The operation returns a mapping of User Data Type to data chunks, with each data chunk annotated with an entity tag and (optionally) a key index. (Note that this is the same general structure as the input data for Replace User Data, for each requested data type. If no chunks for a requested data type exist, an implementation MAY omit that data type from the response.

To transform the data from the output to Avro binary records, a consumer should apply the following algorithm to each data type included:

  1. Determine the relevant encryption algorithm, compression codec, and object schema from the User Data Type and version noted.
  2. For each chunk,
    1. If encryption is indicated, decrypt the chunk data using the user’s secret key (identified using the key index) as in the libsodium function crypto_box_seal_open.
    2. If compression is required, uncompress the chunk data using the specified codec.
    3. Deserialize the uncompressed data to logical records according to the Avro object schema.
    4. Retain the chunk’s etag value if needed for any updates.

Examples

The following section is non-normative. The JSON schema and encoding used is provided for illustration only and implementations are free to define their own encoding.

The following example illustrates the output of a Get User Data Operation invocation requesting data for publicFollows, privateConnections, and privateConnectionPRIds:

{
  "publicFollows": {
    "version": "1.2",
    "chunks": [
      {
        "data": base64_string,
        "etag": string
      },
      {
        "data": base64_string,
        "etag": string
      }
    ]
  },
  "privateConnections": {
    "version": "1.2",
    "chunks": [
      {
        "data": base64_string,
        "etag": string,
        "keyId": integer
      },
      {
        "data": base64_string,
        "etag": string,
        "keyId": integer
      }
    ]
  },
  "privateConnectionPRIds": {
    "version": "1.2",
    "chunks": [
      {
        "data": base64_string,
        "etag": string
      }
    ]
  }
}

Graph Edge

Relationships between users are represented using the GraphEdge object.

Serialization

GraphEdge object serialization MUST conform to the following Avro schema:

{
    "namespace": "org.dsnp",
    "name": "GraphEdge",
    "type": "record",
    "doc": "A relationship to another DSNP user",
    "fields": [
        {
            "name": "userId",
            "type": "long",
            "doc": "The other user's DSNP User Id"
        },
        {
            "name": "since",
            "type": "long",
            "doc": "Timestamp in Unix epoch seconds when this relationship was originally established"
        }
    ]
}

Generation

userId

To allow for optimal compression, User Ids are stored using the long type in Avro schema, which is a 64-bit signed integer. Care should be taken to ensure that User Id values greater than or equal to 263, where used by an implementation, are converted correctly between signed and unsigned representations.

since

PRId

A Pseudonymous Relationship Identifier is represented by the PRId object type.

Serialization

PRId object serialization MUST conform to the following Avro schema:

{
    "namespace": "org.dsnp",
    "name": "PRId",
    "type": "fixed",
    "size": 8,
    "doc": "Pseudonymous Relationship Identifier"
}

Generation

PRIds are generated cryptographically to represent a relationship from one user to another within a specified context, in a privacy-preserving manner.

Contexts

The following context values are currently defined for PRIds. All other values are reserved for future use.

Context IdDescriptionContext string
0ConnectionPRIdCtx0

Algorithm

In the following section, the Alice to Bob identifier for context C is called PRIdA→B,C, and the corresponding Bob to Alice identifier is called PRIdB→A,C.

A PRId is derived from Alice and Bob’s keyAgreement key pairs, using a key exchange protocol as follows. To illustrate the cryptographic operations required, the relevant functions from libsodium are noted. Sodium is a stable, fast, free, and cross-platform cryptography library, and supports all encryption algorithms used in the DSNP specification out of the box.

Definitions:

  • IdA = DSNP User Id of A (little-endian)
  • IdB = DSNP User Id of B (little-endian)

Algorithm:

  1. Both Alice and Bob generate an asymmetric key pair for use with X25519 ECIES. Each uses the Replace User Data Operation to publish their generated public key in keyAgreementPublicKeys.
LibsodiumAlgorithm
crypto_box_keypair(
  &a_public,
  &a_secret);
crypto_box_keypair(
  &b_public, 
  &b_secret);
  
(Apublic, Asecret) ← KGF()
(Bpublic, Bsecret) ← KGF()
  1. When Alice wants to interact with Bob, she looks up Bob’s public key and performs an X25519 Elliptic-curve Diffie-Hellman key exchange operation using her secret key and Bob’s public key, generating a root shared secret.
LibsodiumAlgorithm
crypto_box_beforenm(
  &root_shared_secret,
  b_public,
  a_secret);
RootSharedSecretABECDH(Bpublic, Asecret)
  1. Alice derives a context-specific subkey CtxSharedSecretA→B from the shared secret RootSharedSecretAB as the master key, Bob’s DSNP User Id as the 64-bit key identifier, and the ASCII encoding of the PRId Context string ("PRIdCtx0" for connections).
LibsodiumAlgorithm
crypto_kdf_derive_from_key(
  ctx_shared_secret,
  32,
  b_user_id,
  "PRIdCtx0",
  root_shared_secret);
CtxSharedSecretA→B,C ←
  Blake2b256(
    key = RootSharedSecretAB,
    message = {},
    salt = IdB || {0},
    personal = "PRIdCtx0" || {0})
  1. Alice uses Bob’s DSNP User Id to form an 8-byte little-endian message. Alice encrypts this message using XSalsa20 with the PRId key CtxSharedSecretA→B,C and a nonce of her own User Id (little-endian) followed by 16 zero bytes.
LibsodiumAlgorithm
char nonce[24] = {0};
int i;
for (i = 0; i < 8; i++) {
  nonce[i] = (user_id_a >> (i*8))
    & 0xff;
}
crypto_secretbox_detached( &prid, &mac_unused, user_id_b, 8, nonce, ctx_shared_secret);
  • Alice’s act of publishing provides authentication, so the MAC is unused.
PRIdA→B,C ←
  XSalsa20(
    message = IdB,
    key = CtxSharedSecretA→B,C,
    nonce = Padded24BytesLE(IdA)
  )
  1. Alice adds the generated PRId to the relevant list of PRIds and publishes an updated copy via the Replace User Data Operation.

Similarly, Bob can calculate the same root shared secret RootSharedSecret using Alicepublic and Bobsecret and derive the same PRIdA→B,C in order to check if it is in Alice’s published PRIds. Bob can also derive the PRId subkey for Alice’s DSNP User Id and encrypt Alice’s User Id, using his own as the nonce, to generate the Bob-to-Alice PRId (PRIdB→A,C), and then publish it to his own list, if desired.

If Alice or Bob wants to prove to a third party that their PRIds are in each other’s PRId list, they can provide the third party with their own subkey CtxSharedSecretA→B,C or CtxSharedSecretB→A,C. The third party can repeat the encryption step using Alice and Bob’s User Ids, and check that the output is present in the published set of PRIds. The root shared secret RootSharedSecret (used as a master key in this algorithm) should not be divulged.

Test Vector

For the following inputs:

InputValue
Asecret0xc9432ed5c0c5c24e8a4ff190619893918b4d1265a67d123895023fa7324b43e0
Apublic0x0fea2cafabdc83752be36fa5349640da2c828add0a290df13cd2d8173eb2496f
Bsecret0xdc106e1371293ee9536956e1253f43f8941d4a5c4e40f15968d24b75512b6920
Bpublic0xd0d4eb21db1df63369c147e63b2573816dd4b3fe513e95bf87f7ed1835407e62
IdA42
IdB478
ContextPRIdCtx0

An implementation of the PRId generation algorithm should produce the following outputs:

OutputValue
PRIdA→B0xace4d2995b1a829c
CtxSharedSecretA→B,C0x37cb1a870f0c1dce06f5116faf145ac2cf7a2f7d30136be4eea70c324932e6d2
PRIdB→A0x1a53b02a26503600
CtxSharedSecretB→A,C0x32c45c49fcfe12f9db60e74fa66416c5a05832c298814d82032a6783a4b1fca0

Public Key

Represents an encoding of a public key, one half of a cryptographic key pair.

Serialization

PublicKey object serialization MUST conform to the following Avro schema:

{
    "namespace": "org.dsnp",
    "name": "PublicKey",
    "type": "record",
    "fields": [
        {
            "name": "publicKey",
            "type": "bytes",
            "doc": "Multicodec public key"
        }
    ]
}

Generation

publicKey

  • MUST be a public key of an allowed key type for the associated User Data type, encoded in multicodec format

The byte encoding consists of a multicodec key identifier (as a varint) followed by the public key’s binary data in the codec’s described format.

Allowed Key Types

User Data TypeAllowed Algorithms (multicodec)Purpose
keyAgreementPublicKeysx25519-pubA Curve25519 public key that can be used in key exchange protocols to generate a shared secret
assertionMethodPublicKeysed25519-pubA public key for the EdDSA signature scheme using SHA-512 and Curve25519 that can be used to verify cryptographic signatures

ProfileResource

A profile-linked resource is represented by the ProfileResource object type.

Serialization

ProfileResource object serialization MUST conform to the following Avro schema:

{
    "namespace": "org.dsnp",
    "name": "ProfileResource",
    "type": "record",
    "doc": "Profile-linked resource",
    "fields": [
        {
            "name": "type",
            "type": "int",
            "doc": "Type of resource"
        },
        {
            "name": "contentAddress",
            "type": "string",
            "doc": "Content address for the resource"
        }
    ]
}

Generation

type

  • MUST be a supported DSNP profile-linked resource type.

Supported Profile-Linked Resource Types

ValueDescriptionSpecificationContent TypeMaximum File SizeContent Address TypeDSNP Version Added
1Activity Content Profile with DSNP extensionsDSNP Profileapplication/json256 KbSupported IPFS CID11.3

All other resource type values are reserved for future expansion.

1Supported IPFS CIDs must be CID version 1, using either sha2-256 or blake3 hashes with the raw codec and the base32 serialization.

contentAddress

  • MUST be a valid content address for the specified type (see table above)
  • MUST contain sufficient information for an application to perform content integrity validation, for example by comparing the contentAddress field to the address derived by recalculating the content address from the bytes of the retrieved resource

Batch Publications

A Batch Publication is an Apache Parquet file with a collection of Announcements.

Implementation Requirements

Discoverable

Implementations MUST have publicly discoverable Batch Publications.

Validity

Implementations MUST be able to validate Parquet file contents. Validity MUST be immutable.

Historical

Implementations MUST retain proof of existence of a Batch Publication.

Transparent Chain of Delegation

All Announcements in a Batch file MUST be able to be proven to be from or have a chain of delegation to the publisher of the Batch.

File Requirements

Batch files are stored and transferred in Apache Parquet format.

  • Batch files MUST match the spec for a single Announcement Type.
  • Batch files MUST have Bloom filters set in accordance with the Announcement Type Spec.
  • Batch files MUST have NO MORE THAN 128*1024 rows.

Bloom Filter

Calculation for filter bits is different and is nearly a factor of 10 lower than for a standard Bloom filter: 128*1024 rows with a 0.001 false-positive rate results in around 29,000 bits for a Split Block Bloom filter.

Bloom filters are ONLY added to some fields. See also Announcement Types.

Columns with Bloom Filters

ColumnPrimitive TypeLogical TypeConverted Type (deprecated)
contentHashBYTE_ARRAYSTRINGUTF8
emojiBYTE_ARRAYSTRINGUTF8
fromIdINT64INT(64, false)UINT_64
inReplyToBYTE_ARRAYSTRINGUTF8
targetContentHashBYTE_ARRAYSTRINGUTF8

Non-Normative

Design Requirements

Batch files need to be quickly and easily searchable. Minimal storage size and fast, simple querying are preferred over guarantees of no false positives or advanced data manipulation and column relationships. The files are parseable by client applications, web views, or browsers running pure JavaScript without needing to convert the format.

Applications need to know if a given Batch file has any information of interest without downloading the file first.

Why Parquet?

  1. Parquet is a column-oriented format. Since DSNP Batch Message data will have a very small column-to-row ratio compared to a typical web application database, it makes sense to prefer a column-oriented format.
  2. Parquet format has been field tested under extreme network conditions. It has broad support in cloud storage solutions, with libraries in multiple languages.
  3. Bloom filters are already supported in the Parquet specification, which allows for fast and accurate searching (with caveats for proper configuration).
  4. Amazon S3 support: We anticipate that some Batch Announcers (and possibly Archivists) will store Batch files on Amazon S3. Amazon Athena also supports storage in Parquet, and its API supports SQL-like queries.
  5. Parquet also allows references to the same column across files, which could enable multi-file querying in the future.
  6. Parquet supports compression formats such as Brötli, which is already a browser standard and offers a demonstrated improvement in compression speed and file size over older formats.
  7. Parquet files can be transferred directly to clients, which can parse the files in the app or browser. No conversion to a serialization format is necessary. This eliminates an entire class of bugs and makes both fetching and querying faster.
  8. Parquet uses schemas, which additionally reduce file size.

Rejected Alternatives

  1. Cassandra, RocksDB, CouchDB, MongoDB, and HBASE were rejected since DSNP data needs neither a database for storage nor the overhead of one. Each of these was designed for use cases ranging from somewhat different to drastically different from the DSNP network.
  2. JSON, BSON, and SQLite, while sometimes used for storage, are intended for serialization. They are schemaless, which results in redundant information and therefore a larger size than formats with schemas. They also don’t support Bloom filters; thus, indexing would be required, or new batches would need to be downloaded entirely. The exception is SQLite, which does support more advanced queries. However, SQLite was designed for in-memory storage.

Batch Validity and Order

Batch validity is immutable and is usually based (in part) on the validation of the delegation of authors listed inside the batch to the publisher. Due to the nature of distributed systems, it is possible that a race condition occurs such that a user’s delegation revocation presents before a Batch that contains a message from that user via the revoked delegate. While those individual messages should be considered invalid, a window of time for historical testing is suggested before considering the entire batch invalid. This is analogous to the idea of a confirmation time, but only applies to the past rather than the future.

Attribute Sets

Attribute sets enable structured data to be associated with DSNP users (self or others), DSNP content, or any other content (original or not) that has a public URL. Attribute sets have a cryptographically authenticated creator, and a subject (the entity being described), which may be the same. Attribute set data may appear in several modes: it can be published as a DSNP Announcement, attached to a user’s Profile document, or attached to social media content.

Data Model

The conceptual model for attribute sets includes three types of data:

  1. A schema that encodes rules for validating attribute set data. This MUST be in the form of a DSNP Verifiable Credential Schema (a Verifiable Credential that contains a JSON Schema document).
  2. The attribute set data itself. This MUST be serialized as a DSNP Verifiable Credential.
  3. A reference to the attribute set data, which may take one of several different forms depending on the desired usage pattern.

Attribute Set Type

An Attribute Set Type is an identifier that is used to group attribute sets that share the same data structure and semantic meaning. Attribute Set Types have a well known canonical name and (in most cases) a well defined schema, expressed using a Verifiable Credential Schema document.

Versioning

DSNP supports (but does not mandate) Attribute Set Types that can be described in different Verifiable Credential Schema documents over time in order to preserve backward and forward compatibility. This allows the exact details of the schema to evolve over time without compromising the discovery and verifiability of previously shared credentials.

This behavior is achieved by allowing a schema issuer to retain a common identifier across multiple versions of a schema, provided that the proof signatures are all issued from the same DSNP user (the schema author). This allows applications to perform reliable lookups against an Attribute Set Type, regardless of the specific version that may be used by an individual credential document. This is facilitated by the following naming scheme.

Canonical naming

Attribute Set Type canonical names are constructed as follows:

  • MUST be in the format attributeSetTypeNamespace + “$” + attributeSetTypeName, where attributeSetTypeNamespace MUST be either a multihash content hash (encoded as a multibase string), the DSNP DID of the schema author (beginning with “did:dsnp:”), or the empty string (for schemaless attribute set types).
  • attributeSetTypeName MUST match a declared type value in the Verifiable Credential document.
  • If attributeSetTypeNamespace is a DID, it MUST match the issuer of the Verifiable Credential Schema document referenced from the credential document, and the schema document MUST include a proof that can be verified using the issuer’s public key.
  • If attributeSetTypeNamespace is empty, the credential document MUST NOT reference a schema.
  • If attributeSetTypeNamespace is a multibase string, it MUST match the multihash content hash of the schema file referenced from the credential document.

Examples:

  • Schemaless: $IsHuman
  • Unsigned schema: bciqoatksnf4szlitvzfpsdgzegmkmz74i4mpsmb76q7pro42c6fvpzq$BSC
  • Signed schema: did:dsnp:1234567890$TimeZone

Usage Patterns

Attribute Set Announcements

The Announcement model allows attribute sets to be consumed as events providing context or metadata to the social network. Attribute Set Announcements allow DSNP Users to associate data or make assertions about their own account, other DSNP Users, content on DSNP, or even content external to DSNP. Credential documents are anchored to the announcement by the url and hash fields.

Applications consuming these announcements can index and use the associated attribute sets to inform their user experience. For example, a fact-checking organization can publish Attribute Set Announcements to flag content it deems to be misinformation, and interested applications that trust the organization’s determinations can provide warning labels on social media posts. Similarly, an organization might attach metadata in the form of attribute sets to denote DSNP User Ids that are operated by government actors.

Attribute Set Announcements cannot be updated once published, but can be tombstoned. Applications MUST treat tombstoned Attribute Set Announcements as nonexistent.

Announcement Types

Attribute Set announcements are expressed using three announcement types, depending on the type of subject being described.

Attestation Attachments

Attribute sets associated with and controlled by a DSNP User can be referenced as Attestation Attachments to a user’s Profile document, or to an Activity Content Note object that is referenced from a Broadcast or Reply announcement.

Profile-linked attestations are necessary in cases where an attestation is required in order for a consumer to verify chains of trust designated by schema controllers, such as an accreditation that gives the organization represented by the DSNP User Id the authority to issue credentials to others.

Applications enabling users to update their Profile document should take care to preserve any Profile-linked attestations, even if the meaning of these is opaque to the updating application.

Attestation attachments link to credentials that identify a DSNP User as their subject. That is, the credentialSubject.id field should match the user’s DSNP DID.

On-Demand Presentation

Holders of Verifiable Credentials may wish to present these only on demand rather than have a public record of the credential. Several emerging specifications are developing for this usage pattern and are outside the scope of this specification. Developers are encouraged to consider the Verifiable Presentation Request specification.

Public Key References

Both Verifiable Credential Schema documents and Verifiable Credential documents may include proof sections.

DSNP Users may control one or more key pairs for use in signing these documents and make these discoverable using the assertionMethod User Data type. A key announced in this fashion can be referenced within the proof using a DSNP DID with a key identifier, as in did:dsnp:123456#key1.

A verifier MUST ensure that the DSNP User Id referenced this way (that is, the substring of the DID before the first # character) is the same as the issuer field.

Following the principle of least privilege, the key pair used to issue credentials SHOULD be different from any control keys used to authenticate transactions.

Trust and Verification

Trust in an attribute set may be assigned based on a combination of its Attribute Set Type and issuer. It is left to each application that acts as a DSNP consumer to determine which attribute sets it will trust.

Trust MUST be accompanied by verification of the documents linked to an attribute set reference.

Summary of Verification Responsibilities

When verifying a credential document, a consumer MUST:

  • Retrieve the Verifiable Credential document from the announcement or attachment link’s url field (or, for on-demand credentials, receive the credential document by some other mechanism).
  • Calculate and verify that the content hash of the credential document matches the value specified in the referring item (an Attribute Set Announcement’s contentHash field, for example).
  • Verify that the credential document is well formed (it should comply with the generic JSON schema for verifiable credential documents, and include a valid combination of fields).
  • Verify that the credential’s expiration date (the expirationDate or validUntil field), if present, has not passed.
  • If the subject of the credential document is not a URL beginning with “dsnp://”, verify the subjectContentHash by retrieving the subject URL and applying the indicated hash function. URLs beginning with “dsnp://” do not need a hash check as they already include a self-reflective content hash. They should, however, be checked for existence.
  • Retrieve the schema document from the URL specified in the credentialSchema.id field.
  • For schema of type JsonSchemaCredential, validate the schema document against the generic JSON schema for Verifiable Credential Schema, and validate the credential document against the JSON schema found in the Verifiable Credential Schema document’s credentialSubject.jsonSchema property.
  • For schema of type JsonSchema, validate that the schema document is a well formed JSON schema, then validate the credential document against the schema document.
  • Construct the attributeSetType for the credential document by following the rules for schemaless, unsigned, or signed schemas, and ensure it matches the declared type in the announcement or link object. Note that this MAY require verification of the issuer proof of the Verifiable Credential Schema, and potentially a chain of trust (see below).
  • Verify the issuer proof on the credential document, if present.

Verifiers should take care that transient errors (for example, a URL being unreachable due to temporary network issues) do not lead to long-term caching of false negatives.

Trust chains

Applications are free to make their own trust decisions, and display or incorporate Verifiable Credentials based on their issuer. However, in many real world scenarios, the originator of a particular type of credential may authorize agents to issue the credential, based on any number of approval or certification processes external to DSNP. When utilizing Verifiable Credential Schema documents, the originator of a DSNP Attribute Set Type can encode its requirements for trusted agents within a DSNP extension field.

To do so, DSNP uses an optional, protocol-specific trust key within the dsnp key under the credentialSubject section of the schema credential. The trust key can contain one or both subkeys oneOf or allOf, which in turn contain a list of Attribute Set Types, or further embedded oneOf or allOf groupings. These indicate to the consumer that the author of the credential schema requires that a credential of the defined type be trusted only if its issuer can be shown to possess Verifiable Credentials of the indicated type.

In the case of oneOf, the verifier should check that the issuer is the subject of a Verifiable Credential conforming to at least one of the given Attribute Set Types. In the case of allOf, the verifier should check that the issuer is the subject of Verifiable Credentials for every given Attribute Set Type.

For example, the Worldwide Whale Foundation, identified by DSNP User Id 123456, may authorize its agents, including Acme Ocean Certification Lab, to issue credentials of type did:dsnp:123456$CertifiedWhaleBiologist to individuals that meet the certification standard that the Foundation defines. The Worldwide Whale Foundation defines a second attribute set type, did:dsnp:123456$AuthorizedWhaleBiologistCertifier by publishing a Verifiable Schema Credential for that type, and then issues a Verifiable Credential of that type (an accreditation) to Acme Ocean Certification Lab, which is then published online. Worldwide Whale Foundation then includes the did:dsnp:123456$AuthorizedWhaleBiologistCertifier Attribute Set Type in the credentialSubject.dsnp.trust.allOf array of the CertifiedWhaleBiologist Verifiable Credential Schema (in this example, using oneOf would be equivalent), generates and attaches the signature proof, and publishes the schema document. When Acme Ocean Certification Lab wants to issue did:dsnp:123456$CertifiedWhaleBiologist credentials to users, it should include the URL, attribute set type, and content hash of its accreditation credential in the issuer.authority array of each credential issued. A consumer of the credential can retrieve and verify that Acme Ocean Certification Lab is an accredited issuer.

Displaying credentials

The DSNP schema credential document MAY contain an additional display key within the dsnp key under credentialSubject. This allows the authors of attribute set types to recommend how a credential should be displayed within a user interface.

To allow for localizable text, a map of content language codes (following BCP-47, so using the same form as the HTTP Content-Language header) to display text can be used under the subkey label.

Example:

...
"credentialSubject": {
  "type": "JsonSchema",
  "jsonSchema": {
    ...
  },
  "dsnp": {
    "display": {
      "label": {
        "en-US": "Whale Biologist",
        "es-ES": "Biólogo de Ballenas"
      }
    }
  }
}

DSNP applications should treat the display section as a recommendation but not a mandate, and are free to indicate the presence or absence of a Verifiable Credential in other forms.

Announcements Overview

Announcements are content or references to content that communicate new user activity to the rest of the network. Announcements are associated with an Identifier that can be validated as the creator of the Announcement. Depending on the implementation, Announcements may be published directly to the network, included in Batch Publication Files, or some combination of those two.

Announcement Types

Each Announcement has an enumerated type for use when separating out a stream of Announcements.

ValueNameDescriptionDSNP Content URITombstone Allowed
0Tombstonean invalidation of previously announced contentnono
1Graph Changeasocial graph changesnono
2Broadcasta public postYESYES
3Replya public response to a BroadcastYESYES
4Reactiona public visual reply to a Broadcastnono
5Profileca profileYESno
6Updatean update to contentYESno
7Public Keyba public key for secure communicationnono
8User Attribute Setan attribute set for a DSNP UserYESYES
9DSNP Content Attribute Setan attribute set for a DSNP content itemYESYES
10External Content Attribute Setan attribute set for a non-DSNP content itemYESYES

a Since DSNP version 1.2, social graph changes use User Data operations as described in the Graph section.

b Since DSNP version 1.3, public keys use User Data operations.

c Since DSNP version 1.3, profile changes use User Data operations.

Announcement Validation

There is no guarantee that, at time of creation, a given Announcement will be from the fromId claimed in the Announcement. The reader MUST perform a validation of the Announcement at read time to ensure authenticity. Implementations MUST provide a way to validate that the identifier associated with a given Announcement is authentic.

External Content URLs and Hashes

Where Announcements refer to external documents (such as Activity Content documents), these are referenced by both a URL and a DSNP Content Hash. The content hash MUST be generated by applying a Supported Hashing Algorithm to the full, unaltered contents of the document.

The URL associated with a content hash should be construed as a hint to initially locate a document matching the content hash, but is in no way meant to be the only way to locate the indicated document. Over time, a URL may cease to reference the specified document, or might have its contents altered; therefore, the content hash should be considered the authoritative value and the URL only one of many possible ways of locating a document. For example, services may cache documents or retrieve them from a content-addressed file system by applying the content hash (or a value derived from the content hash, such as a CID). When readers retrieve content referenced in an Announcement, they can validate the authenticity of the content, regardless of where it is hosted, by regenerating the hash output and comparing it with the content hash recorded in the Announcement.

Duplicate Handling

Due to the nature of asynchronous communication, duplicate Announcements may occur. In the case of duplicates, the first Announcement should be considered the ONLY valid Announcement. Additional duplicate Announcements MUST be rejected or ignored.

Ordering Announcements

  1. Order Batch Publications by implementation order.
  2. Order Announcements in a Batch Publication File by row appearance order.

Reverting an Announcement

Announcements may not be deleted, but may be marked invalid by using a Tombstone Announcement, or updated by using an Update Announcement. For example, if a user creates a Reaction Announcement, they may remove that reaction by creating a Tombstone Announcement.

Non-Normative

Duplicate Announcements

Due to the distributed nature of DSNP, duplicate Announcements are possible from time to time. These should be discarded and ignored.

Replay Attacks

Implementations typically restrict replay attacks by testing that the chain transaction sender is authorized (often via delegation) to publish an Announcement.

Announcement Ordering and Activity Content Published Timestamp

Activity Content has a published field that contains a user-generated timestamp. User-generated timestamps cannot be validated, but may be used to indicate ordering other than the network order for Announcements (which are not time dependent.)

Announcement Reference Ordering

Some Announcements contain references to other Announcements via the inReplyTo field. Due to the distributed nature of DSNP, the canonical order can have an Announcement that refers to another Announcement appearing later in the network order. For display purposes, these messages should be considered to have occurred after the reference.

DSNP v1.0 Announcement Signatures

In DSNP v1.0, Announcements had individual signatures, producing Batch Publications that were generic and disconnected from the user. Announcements could be submitted to the chain via anyone—not just delegates or users.

In DSNP v1.1, Announcement signatures were removed in favor of the implementation being responsible for the connection between the on-chain signature and the user. Implementations require that the transaction that produces a Batch be performed by the user or delegate directly. This creates batches that are delegate specific, but allows for faster testing of the validity of individual Announcements in a Batch.

For more information see DIP-145.

Tombstone Announcement

A Tombstone Announcement is a way to note that a previously announced content is invalid and the related Announcement should be considered reverted. It is NOT possible to revert a tombstone.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (0)enumdecimalINT32no
fromIdId of the user creating the Announcement and the Tombstoned Announcement64-bit unsigned integerdecimalUINT_64YES
targetAnnouncementTypetarget tombstoned Announcement typeenumdecimalINT32no
targetContentHashtarget contentHash of the original Announcement to tombstoneUTF-8base32 multibaseUTF8YES

Field Requirements

announcementType

  • MUST be fixed to 0

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation

targetAnnouncementType

  • MUST be the Announcement Type of the target Announcement
  • MUST ONLY be a Tombstone allowed Announcement Type

Tombstone Allowed Announcement Types

targetContentHash

  • MUST be the contentHash of a previous Announcement of an Allowed Announcement Type with the same fromId as the Tombstone Announcement

Broadcast Announcement

A Broadcast Announcement is a way to send a public message to everyone.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (2)enumdecimalINT32no
contentHashDSNP Content Hash of contentUTF-8base32 multibaseUTF8YES
fromIdid of the user creating the Announcement64-bit unsigned integerdecimalUINT_64YES
urlcontent URLUTF-8UTF-8UTF8no

Field Requirements

announcementType

  • MUST be fixed to 2

contentHash

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation

url

  • MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
  • Resource MUST be one of the supported Activity Content Types
  • MUST use one of the supported URL Schemes

Supported URL Schemes

SchemeDescriptionReferenceDSNP Version Added
HTTPSHypertext Transfer Protocol SecureRFC28181.0

Reply Announcement

A Reply Announcement is the same as a Broadcast Announcement, but includes an inReplyTo field for noting it as a reply to a given DSNP Content URI.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (3)enumdecimalINT32no
contentHashDSNP Content Hash of contentUTF-8base32 multibaseUTF8YES
fromIdid of the user creating the Announcement64-bit unsigned integerdecimalUINT_64YES
inReplyToTarget DSNP Content URIUTF-8UTF-8UTF8YES
urlcontent URLUTF-8UTF-8UTF8no

Field Requirements

announcementType

  • MUST be fixed to 3

contentHash

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation

inReplyTo

url

  • MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
  • Resource MUST be one of the supported Activity Content Types
  • MUST use one of the supported URL Schemes

Supported URL Schemes

SchemeDescriptionReferenceDSNP Version Added
HTTPSHypertext Transfer Protocol SecureRFC28181.0

Reaction Announcement

A Reaction Announcement is for publishing emoji reactions to anything with a DSNP Content URI.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (4)enumdecimalINT32no
emojithe encoded reactionUTF-8UTF-8UTF8YES
applyhow to apply the reaction8-bit unsigned integerdecimalUINT_8no
fromIdid of the user creating the relationship64-bit unsigned integerdecimalUINT_64YES
inReplyToTarget DSNP Content URIUTF-8UTF-8UTF8YES

Field Requirements

announcementType

  • MUST be fixed to 4

emoji

  • Emoji fields must not be empty
  • Emoji fields must consist only of Unicode points from U+2000 to U+2BFF, from U+E000 to U+FFFF, or from U+1F000 to U+10FFFF

Examples

All of the following should be considered valid emojis:

"😀", "🤌🏼", "👩🏻‍🎤", "🧑🏿‍🏫", "🏳️‍🌈", "🏳️‍⚧️", "⚛︎", "🃑", "♻︎"

None of the following should be considered valid:

"F", ":custom-emoji:", "<custom-emoji>", "ᚱ", "ᘐ", "״"

apply

  • MUST be an UINT_8
  • Indicates whether the emoji should be applied and if so, at what “strength”.

Potential uses:

  • a single reaction
  • ratings
  • a range of responses, e.g. “strongly disagree” –> “strongly agree” = 1 –> 5 stars.
  • recommendation engines

Apply Enums

ValueNameDescription
0retractRemove the referenced emoji
napplyApply the referenced emoji N times

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation

inReplyTo

Non-Normative

Likes

Generic “likes” should default to the "❤️" or Unicode U+FE0F as the emoji in the reaction.

Update Announcement

An Update Announcement is a way to note intent to update previously announced content. If the original Broadcast/Reply is Tombstoned, subsequent Updates should be ignored.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (6)enumdecimalINT32no
fromIdid of the user creating the announcement64 bit unsigned integerdecimalUINT_64YES
contentHashDSNP Content Hash of contentUTF-8base32 multibaseUTF8YES
urlupdated content URLUTF-8UTF-8UTF8no
targetAnnouncementTypetarget updated Announcement typeenumdecimalINT32no
targetContentHashtarget contentHash of the original Announcement to updateUTF-8base32 multibaseUTF8YES

Field Requirements

announcementType

  • MUST be fixed to 6

fromId

contentHash

url

  • MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
  • Resource MUST be one of the supported Activity Content Types
  • MUST use one of the supported URL Schemes

targetAnnouncementType

  • MUST be the Announcement Type of the target Announcement
  • MUST ONLY be an Update allowed Announcement Type

Update Allowed Announcement Types

ValueName
2Broadcast
3Reply

targetContentHash

  • MUST be the contentHash of a previous Announcement of an Allowed Announcement Type with the same fromId as the Update Announcement

User Attribute Set Announcement

A User Attribute Set Announcement is a way to create an authenticated (and, optionally, attested) attribute set for a DSNP User.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (8)enumdecimalINT32no
fromIdId of the user creating the announcement64 bit unsigned integerdecimalUINT_64YES
subjectDSNP User Id of the attribute set subject64 bit unsigned integerdecimalUINT_64YES
urlURL for the Verifiable Credential documentUTF-8UTF-8UTF8no
contentHashDSNP Content Hash of contentUTF-8base32 multibaseUTF8YES
attributeSetTypeCanonical name of Attribute Set TypeUTF-8UTF-8UTF8YES
issuerURI of issuerUTF-8UTF-8UTF8YES

Field Requirements

announcementType

  • MUST be fixed to 8

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the announcement, either directly or via a transparent chain of delegation

subject

url

  • MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
  • Resource MUST be a Verifiable Credential Document
  • MUST use one of the supported URL Schemes

Supported URL Schemes

SchemeDescriptionReferenceDSNP Version Added
HTTPSHypertext Transfer Protocol SecureRFC28181.3

contentHash

attributeSetType

issuer

  • MUST be a valid URI
  • MUST match the issuer value from the credential document retrieved from url

DSNP Content Attribute Set Announcement

A DSNP Content Attribute Set Announcement is a way to create an authenticated (and, optionally, attested) attribute set for a DSNP content item.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (9)enumdecimalINT32no
fromIdId of the user creating the announcement64 bit unsigned integerdecimalUINT_64YES
subjectDSNP Content URI of the attribute set subjectDSNP Content URIUTF-8UTF8YES
urlURL for the Verifiable Credential documentUTF-8UTF-8UTF8no
contentHashDSNP Content Hash of contentUTF-8base32 multibaseUTF8YES
attributeSetTypeCanonical name of Attribute Set TypeUTF-8UTF-8UTF8YES
issuerURI of issuerUTF-8UTF-8UTF8YES

Field Requirements

announcementType

  • MUST be fixed to 9

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the announcement, either directly or via a transparent chain of delegation

subject

url

  • MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
  • Resource MUST be a Verifiable Credential Document
  • MUST use one of the supported URL Schemes

Supported URL Schemes

SchemeDescriptionReferenceDSNP Version Added
HTTPSHypertext Transfer Protocol SecureRFC28181.3

contentHash

attributeSetType

issuer

  • MUST be a valid URI
  • MUST match the issuer value from the credential document retrieved from url

External Content Attribute Set Announcement

An External Content Attribute Set Announcement is a way to create an authenticated (and, optionally, attested) attribute set for content external to the DSNP system.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (10)enumdecimalINT32no
fromIdId of the user creating the announcement64 bit unsigned integerdecimalUINT_64YES
subjectURL of the external contentURLUTF-8UTF8no
subjectContentHashDSNP Content Hash of content at subjectUTF-8base32 multibaseUTF8YES
urlURL for the Verifiable Credential documentUTF-8UTF-8UTF8YES
contentHashDSNP Content Hash of content at urlUTF-8base32 multibaseUTF8YES
attributeSetTypeCanonical name of Attribute Set TypeUTF-8UTF-8UTF8YES
issuerURI of issuerUTF-8UTF-8UTF8YES

Field Requirements

announcementType

  • MUST be fixed to 10

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the announcement, either directly or via a transparent chain of delegation

subject

Optional. If present,

  • MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
  • MUST use one of the supported URL Schemes

subjectContentHash

url

  • MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
  • Resource MUST be a Verifiable Credential Document
  • MUST use one of the supported URL Schemes

contentHash

attributeSetType

issuer

  • MUST be a valid URI
  • MUST match the issuer value from the credential document retrieved from url

Supported URL Schemes

SchemeDescriptionReferenceDSNP Version Added
HTTPSHypertext Transfer Protocol SecureRFC28181.3

Migrated Announcements

Graph Change Announcement

*Since DSNP version 1.2, social graph changes use [User Data](../UserData.md) operations as described in the [Graph](../Graph.md) section.*

A Graph Change Announcement is for publishing relationship state changes for a user.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (1)enumdecimalINT32no
changeTypeType of relationship changeenumdecimalINT32no
fromIdId of the user creating the relationship64-bit unsigned integerdecimalUINT_64YES
objectIdId of the target of the relationship64-bit unsigned integerdecimalUINT_64YES

Field Requirements

announcementType

  • MUST be fixed to 1

changeType

  • MUST be one of the Change Type Enum

Change Type Enum

Different change types have different meanings.

ValueNameDescription
0UnfollowRemove a Follow relationship
1FollowCreate a Follow relationship

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation

objectId

Non-Normative

Graph Retrieval, Ordering and Reading

Each Graph Change event represents a state transition for the graph. The state of the graph at any time is given by taking the state of the graph at a previous time and applying all Graph Change events not previously applied in the order specified by Announcement Ordering.

Once those Graph Change events are retrieved, they can be ordered to reflect the current graph state (i.e. Charlie has followed Bob, then he unfollowed him, and then followed him again. The graph state reflects that Charlie is following Bob.)

To retrieve the graph, do the following:

  1. Retrieve the events with announcementType matching the enum for Graph Change.
  2. Filter the events to a particular DSNP User Id to retrieve information about the respective graph.
  3. Order the retrieved data by Announcement Ordering.

Public Key Announcement

*Since DSNP version 1.3, public keys use [User Data](../UserData.md) operations.*

A Public Key Announcement is a way to note a new cryptographic key that can be used in DSNP to secure and verify the authenticity of communications.

The most recently published key (if one exists) for a given key type should be treated as the active key of that key type.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (7)enumdecimalINT32no
fromIdid of the user creating the Announcement64 bit unsigned integerdecimalUINT_64YES
keyTypeKey Type EnumenumdecimalINT32YES
keyIduser-assigned identifier64 bit unsigned integerdecimalUINT_64no
publicKeypublic key in multikey formatvariable length byte arrayUTF-8BYTE_ARRAYno

Field Requirements

announcementType

  • MUST be fixed to 7

fromId

keyType

  • MUST be an allowed Key Type value

Allowed Key Types

ValueNameAllowed Algorithms (multicodec)Purpose
1keyAgreementx25519-pubA Curve25519 public key that can be used in key exchange protocols to generate a shared secret
2assertionMethoded25519-pubA public key that can be used for verifying digital signature proofs of verifiable credentials

keyId

  • A user-assigned 64-bit identifier for the key.

The user may assign a new keyId each time they announce a new key of a given keyType. A keyId value is useful when invoking certain DSNP Operations in order to indicate which key was used to encrypt data. It may also provide a hint to the user if they ever need to regenerate their private key (for example, many key derivation functions enable the use of a subkey identifier to deterministically create a subkey from a root key).

publicKey

  • MUST be a public key of an allowed algorithm for keyType, encoded in multikey format

The multikey encoding of public keys is described in the draft did:key Method specification. The byte encoding consists of a multicodec key identifier (as a varint) followed by the public key’s binary data in the codec’s described format.

If serializing the multicodec value as a string, base58btc encoding is recommended. For example, the string z6LStiZsmxiK4odS4Sb6JmdRFuJ6e1SYP157gtiCyJKfrYha decodes as a Base58 string using the x25519-pub multicodec value with a 32-byte raw key of 0xfd3384e132ad02a56c78f45547ee40038dc79002b90d29ed90e08eee762ae715.

Profile Announcement

A Profile Announcement is a constrained version of a Broadcast Announcement. The reference content MUST be of profile type.

Fields

FieldDescriptionData TypeSerializationParquet TypeBloom Filter
announcementTypeAnnouncement Type Enum (5)enumdecimalINT32no
contentHashmultihash-encoded hash of content stored at URLvariable length byte arrayhexadecimalBYTE_ARRAYYES
fromIdid of the user creating the Announcement64-bit unsigned integerdecimalUINT_64YES
urlprofile content URLUTF-8UTF-8UTF8no

Field Requirements

announcementType

  • MUST be fixed to 5

contentHash

fromId

  • MUST be a DSNP User Id
  • MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation

url

  • MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
  • Resource MUST be a valid Profile Activity Content Type
  • MUST use one of the supported URL Schemes

Supported URL Schemes

SchemeDescriptionReferenceDSNP Version Added
HTTPSHypertext Transfer Protocol SecureRFC28181.0

Non-Normative

Most Recent Profile

When displaying a DSNP user’s profile, the most recent profile should be considered the complete and correct version. Previous Profile Announcements from the same fromId may be disregarded.

Operations

DSNP implementations perform well-defined DSNP Operations and generate DSNP State Change Records.

Control Keys and Proofs

Each invocation of a DSNP Operation MUST have verifiable approval of the acting principal(s) via a Control Key Ownership Proof. The precise data type and representation of both the Control Key and the Control Key Ownership Proof MUST be defined by each DSNP implementation. For example, an implementation might use the public key of an asymmetric key pair as a control key, and provide a proof for each operation by producing a cryptographic signature of the user’s DSNP Identifier and some nonce value.

Where operations are listed as using control keys or ownership proofs as input parameters, this indicates that these keys or proofs should be provided in addition to those needed for invocation authentication.

Transaction Identifiers

Each invocation of a DSNP Operation should be associated with a Transaction Identifier. Transaction Identifiers are used to associate Operation invocations with asynchronously emitted State Change Records. It MUST be possible to associate a DSNP State Change Record with a Transaction Identifier from a particular DSNP Operation invocation. Transaction Identifiers MUST be unique within an implementation. Transaction Identifiers MUST be serializable as a string.

Failure Handling

Compliant implementations may respond to error conditions either synchronously, as a response to the invocation request, or asynchronously, by emitting a Failure Record.

List of Operations

OperationOptional?Principal(s)InputsState Change Record or Output
Create IdentifiernoNoneControl Key, Control Key Ownership ProofIdentifier Creation Record
Retire IdentifiernoUserNoneIdentifier Retirement Record
Define DelegationnoUser AND DelegateUser’s Identifier, Delegate’s Identifier, Set of Allowed Announcement Types, Set of Allowed User Data TypesDelegation Definition Record
Revoke DelegationnoUser OR DelegateUser’s Identifier, Delegate’s IdentifierDelegation Revocation Record
Add Control KeyYESUserKey, Key Ownership ProofControl Key Addition Record
Remove Control KeyYESUserKeyControl Key Removal Record
Publish Announcementno*User OR DelegateAnnouncementAnnouncement Published Record
Publish Batchno*User OR DelegateAnnouncement Type, Batch Publication URL, Batch Publication Content HashBatch Published Record
Get User DatanoAnyUser’s Identifier, Set of Requested User Data TypesMap of User Data Types to Data Chunks with optional key identifiers of encryption keys for each chunk
Replace User DatanoUser OR DelegateUser’s Identifier, Key Identifier, Map of User Data Types to Data ChunksUser Data Replaced Record

* For each Announcement Type, an implementation may support one or both of these operations. Implementations MUST document which of the operations is available for each Announcement Type.

State Change Records

State Change Records constitute the observable output of a DSNP system. Implementations MUST specify how applications can translate implementation-specific output into State Change Records.

Each Record is associated with a Transaction Identifier, which allows Operations to be asynchronously associated with their results.

Records consists of one or more fields.

Record Type Field Field Data Type
Identifier Creation Record Control Key Implementation dependent
Control Key Ownership Proof Implementation dependent
Identifier Retirement Record User's Identifier DSNP User Id
Delegation Definition Record User Id DSNP User Id
Delegate Id DSNP User Id
Allowed Announcement Types List of enum values
Allowed User Data Types List of enum values
Delegation Revocation Record User Id DSNP User Id
Delegate Id DSNP User Id
Control Key Addition Record User Id DSNP User Id
Key Implementation dependent
Key Ownership Proof Implementation dependent
Control Key Removal Record User Id DSNP User Id
Key Implementation dependent
Announcement Published Record Announcement One of the types described in Announcements
Batch Published Record From Id DSNP User Id
Announcement Type Enum value
URL String
Content Hash DSNP Content Hash
Failure Record Message String
User Data Replaced Record User Id DSNP User Id
User Data Types Set of User Data Types

Serializations

Serialization is how the value should be stringified for signing and for transfer between systems. Most serializations use outside standards, but some requiring additional clarifications are provided here.

decimal

Used to represent integers. Strings are used to avoid issues with different implementations of numbers.

  • MUST use 0-9 representation
  • MUST NOT have spaces or separators
  • MUST be a string
InvalidWhyValid
"0x123"Must be decimal"291"
291Must be a string"291"
"291n"BigInt(291) serialization appends an n"291"

base32 multibase

Used to represent bytes. A base32 multibase string is self-identifying and always begins with the b character. The Multibase Table describes this encoding as “RFC4648 case-insensitive - no padding”.

  • MUST use RFC4648 §6 alphabet abcdefghijklmnopqrstuvwxyz234567
  • MUST be lowercase
  • MUST be prefixed with b
  • MUST NOT have spaces or separators
  • MUST NOT end with or contain padding characters (=)
InvalidWhyValid
BDYQDUA4TMust user lowercasebdyqdua4t
dyqdua4tMissing b prefixbdyqdua4t
b3og3k0sjWrong alphabet (base32hex was used)bdyqdua4t
bdyqdua4t=Must not have padding charactersbdyqdua4t

DSNP Systems

DSNP Over Frequency

Language and Framework

Frequency is a Polkadot Parachain written in Rust, using the Substrate framework.

DSNP Over Frequency Schemas

Official schemas may be found in GitHub.

NameSchema NameSchema Id MainnetSchema Id Testnet (Paseo)
Tombstonedsnp.tombstone1616
Broadcastdsnp.broadcast1717
Replydsnp.reply1818
Reactiondsnp.reaction44
Updatedsnp.update1919
Key Agreement Public Keysdsnp.public-key-key-agreement77
Public Followsdsnp.public-follows88
Private Followsdsnp.private-follows99
Private Connectionsdsnp.private-connections1010
Assertion Method Public Keysdsnp.public-key-assertion-method1414
Profile Resourcesdsnp.profile-resources1515
User Attribute Setdsnp.user-attribute-set2020
DSNP Content Attribute Setdsnp.dsnp-content-attribute-set1212
External Content Attribute Setdsnp.ext-content-attribute-set1313
Last Update DateFrequency ReleaseDSNP Version
2024-09-201.13.0+1.3.0

Frequency Identity

Identity

  • Name: Message Source Account or MSA
  • Docs: MSA Pallet
  • Representation: The following data that constitute a Message Source Account are stored in the MSA pallet:
    • The DSNP User Id associated with this MSA
    • Delegation relationships to Providers
    • Schema permissions granted to Providers
    • MSA user state (Profile, Graph, etc…)
    • Control keys

DSNP User Identifier

  • Name: Message Source Account Identifier, or MSA Id.
  • Data Type: uint64
  • Docs: MessageSourceId
  • Mapping: The MSA Id is able to be used directly as the DSNP User Id
  • Description: At least one public key MUST be associated with an MSA Id for it to be considered active.

Control Keys

  • Name: Referred to as: public_key, provider_key, or delegator_key
  • Data Type: AccountId, Schnorrkel/Ristretto X25519 (“sr25519”) derived cryptographic public key
  • Docs: AccountId
  • Description: See Cryptography on Polkadot and Polkadot Protocol Specification. A public key CANNOT be associated with more than one MSA at a time.

Delegation

  • Name: delegation
  • Docs: Delegation
  • Representation: The following data storage relates necessary information for retrieving and validating delegations:
    • Provider registry
    • Delegations
    • Schema permissions granted to Providers

User

  • Name: Delegator
  • Representation: MSA Id

Delegate

  • Name: Provider
  • Representation: MSA Id
  • Description: A Provider MUST already have an MSA Id (via msa::create()) and be approved as a Provider (via msa::propose_to_be_provider()).

Announcement Publishing

On Frequency, Announcements are mapped to Schemas which in turn publish Frequency Messages. Frequency Messages are either individual Announcements from a particular user, or a Batch Publication with a multitude of possible users.

EnumAnnouncementTypeDeployed Schema IdFrequency Model TypeFrequency Payload Location
0TombstoneBatchedMainnet: 16
Testnet (Paseo): 16
Parquet
IPFS
2BroadcastBatchedMainnet: 17
Testnet (Paseo): 17
Parquet
IPFS
3ReplyBatchedMainnet: 18
Testnet (Paseo): 18
Parquet
IPFS
4ReactionBatchedMainnet: 4
Testnet (Paseo): 4
Parquet
IPFS
6UpdateBatchedMainnet: 19
Testnet (Paseo): 19
Parquet
IPFS
8User Attribute SetBatchedMainnet: 20
Testnet (Paseo): 20
Parquet
IPFS
9DSNP Content Attribute SetBatchedMainnet: 12
Testnet (Paseo): 12
Parquet
IPFS
10External Content Attribute SetBatchedMainnet: 13
Testnet (Paseo): 13
Parquet
IPFS

Source code for each schema is located in the LibertyDSNP/schemas repository.

Batch Publications

Frequency uses DSNP Batch Publications for some types of Announcements. Parquet files are stored on IPFS, but are discovered through Frequency Messages.

DSNP Batch Publications MUST be validated upon fetching to ensure data and permission integrity.

Announcement Validation

DSNP Announcements are validated differently depending on the type of Announcement. Non-batched Announcements are on chain, are validated at write time, and do not need to be re-validated at read time. Batched Announcements are off chain and MUST be validated at read time (See: Validation).

Ordering Announcements

Frequency Messages are well ordered:

  1. Frequency: Block number ascending
  2. Frequency: Block index ascending (unique)
  3. DSNP Standard: Order Announcements in a Batch Publication File by row appearance order.

Human Order

Due to the asynchronous nature of networks and batching, it is possible that the canonical ordering of Announcements is wrong from a human viewpoint. With dependent Announcements, where one Announcement refers to another Announcement, the order may be inferred differently than the canonical ordering. It is left to user interfaces to handle these situations.

Retrieval

Frequency nodes provide an RPC interface messages.getBySchemaId() with paginated responses that differ based on the Schema.

Frequency nodes can provide a websocket interface that will emit an event for each block that has one or more messages of a given schema in that block. The messages::MessagesStored event can be used to know when to call the RPC interface to retrieve the messages.

See the Frequency Documentation for more details on Message retrieval.

Batch Publication Validation

The Frequency Message for a Batch Publication has several important fields for validation:

FieldDescription
provider_msa_idMSA Id of the provider sending the message
cidThe Content IDentifier v1 for IPFS content
payload_lengthExpected length of the content from IPFS
block_numberBlock number that the message was recorded on the chain.

File Validation

  1. Retrieve the file from the IPFS network using the cid.
  2. Verify the file hash by comparing it to the hash included in the cid. (Required for non-trusted IPFS nodes.)
  3. Verify that the byte length of the retrieved file matches the payload_length.

Publication Announcements Validation

  1. Collect the unique set of fromId values.
  2. Use the Custom RPC msa.checkDelegations() with the fromId values as the delegator_msa_ids and the provider_msa_id at the block_number.
  3. The fromId values that msa.checkDelegations() verifies as having a delegation at block_number are valid Announcements.
  4. Set the schema_id parameter to the Schema Id used in the Frequency Message

Announcement Duplicates

Duplicate Announcements MUST be rejected or ignored.

Operations

Method of execution

DSNP Operations are executed on Frequency via on-chain transactions, also known as extrinsic calls. An extrinsic is a type of function defined in a Substrate Pallet. A Pallet is a Substrate runtime module and also a Cargo crate.

MSA Pallet

Responsible for DSNP Identity Operations and Delegation management.

Schema Pallet

Responsible for managing the data structures for DSNP Announcements and User Data.

Messages Pallet

Responsible for most DSNP Announcement Operations (see Publishing).

Stateful Storage Pallet

Responsible for DSNP User Data (see User Data) and select Announcement Operations (see Publishing).

Principals

Every Frequency transaction for DSNP is accompanied by an Schnorrkel/Ristretto X25519 (“sr25519”) derived cryptographic signature and associated public key. When the transaction occurs, the signature is validated and the MSA Id is retrieved.

DSNP TermFrequency Term
UserUser / Delegator
DelegateProvider

Failure Handling

Frequency has a variety of errors that fall into these classes:

  • Timeout: A transaction with an unknown status after a set time.
  • Node Rejection: Public nodes may reject transactions from unknown parties.
  • Node Validation Error: A node will reject malformed or invalid transactions when possible.
  • Execution Error: A transaction that was included in a block but failed upon execution.

Transactions are not automatically resubmitted. Check with the Frequency Documentation for more information regarding errors.

List of Operations

Write operations are via Transactions (also called Extrinsics): pallet::extrinsic()

OperationPrincipal(s)Pallet::ExtrinsicState Change Record
Create IdentifierNone
msa::create(),
msa::create_sponsored_account_with_delegation()Identifier Creation Record
Retire IdentifierUser
msa::retire_msa()Identifier Retirement Record
Define DelegationUser AND Delegate
msa::grant_delegation(),
msa::create_sponsored_account_with_delegation()Delegation Definition Record
Revoke DelegationUser
msa::revoke_delegation_by_delegator()Delegation Revocation Record
Revoke DelegationDelegate
msa::revoke_delegation_by_provider()Delegation Revocation Record
Add Control KeyUser
msa::add_public_key_to_msa()Control Key Addition Record
Remove Control KeyUser
msa::delete_msa_public_key()Control Key Removal Record
Publish AnnouncementUser OR Delegate
messages::add_onchain_message()Announcement Published Record
Publish BatchUser OR Delegate
messages::add_ipfs_message()Batch Published Record
Get User DataAnySee User Data: Read Operation Mapping-
Replace User DataUser OR DelegateSee User Data: Write Operation MappingUser Data Replaced Record

State Change Records

State Change Records constitute the observable output of a DSNP system. Frequency uses a combination of on chain data to store and Events to notify for Records.

Frequency Transaction Identifier

Frequency uses the hash of the Operation transaction to then request a node report the transaction status and any Events related to that hash. (Client libraries usually have this built in.)

Frequency Records

Frequency produces three types of data that map or point to DSNP Records.

Events

Events are generated on each block and may contain a pointer to the Record data instead of the entire Record.

Events are referenced by pallet::EventName.

Messages

Frequency Messages store Announcements or Batch Publication Records. It uses the Messages Pallet.

Messages are retrieved via state queries (pallet.stateQuery) or RPC calls (pallet.rpcCall()).

State

Frequency State stores data associated with an Identity. It can also be used to look up the state of prior Records related to an Identity.

State data is retrieved via state queries (pallet.stateQuery) or RPC calls (pallet.rpcCall()).

Record Mappings

Record TypeRecord Pointer/Location
Identifier Creation RecordEvent:
msa::MsaCreated
State:
msa.publicKeyToMsaId
Identifier Retirement RecordEvent:
msa::MsaRetired
Delegation Definition RecordEvent:
msa::DelegationGranted,
msa::DelegationUpdated
RPC:
msa.checkDelegations(),
msa.getGrantedSchemasByMsaId()
Delegation Revocation RecordEvent:
msa::DelegationRevoked
RPC:
msa.checkDelegations()
Control Key Addition RecordEvent:
msa::PublicKeyAdded
State:
msa.publicKeyToMsaId
Control Key Removal RecordEvent:
msa::PublicKeyDeleted
State:
msa.publicKeyToMsaId
Announcement Published RecordEvent:
messages::MessagesStored
RPC:
messages.getBySchemaId()
Batch Published RecordEvent:
messages::MessagesStored
RPC:
messages.getBySchemaId()
Failure RecordSee section on Failure Handling
User Data Replaced RecordEvent:
stateful_storage::ItemizedPageUpdated,
stateful_storage::ItemizedPageDeleted,
stateful_storage::PaginatedPageUpdated,
stateful_storage::PaginatedPageDeleted
RPC:
statefulStorage.getPaginatedStorage(),
statefulStorage.getItemizedStorage()
See Also: Read User Data

Frequency User Data

On Frequency, User Data is mapped to Schemas which use Stateful Storage for storage and retrieval of the data.

User Data Sets

User Data SetDeployed Schema IdsFrequency Model TypeFrequency Payload LocationSettings
Public Key (Key Agreement)Mainnet: 7
Testnet (Paseo): 7
AvroBinary
ItemizedAppend Only, Signature Required
Public FollowsMainnet: 8
Testnet (Paseo): 8
AvroBinary
PaginatedNone
Private FollowsMainnet: 9
Testnet (Paseo): 9
AvroBinary
PaginatedNone
Private ConnectionsMainnet: 10
Testnet (Paseo): 10
AvroBinary
PaginatedNone
Public Key (Assertion Method)Mainnet: 14
Testnet (Paseo): 14
AvroBinary
ItemizedSignature Required
Profile ResourcesMainnet: 15
Testnet (Paseo): 15
AvroBinary
ItemizedNone

Private Connection Pseudonymous Relationship Identifiers (PRIds) are stored alongside Private Connections in the same Stateful Storage page.

For historical reasons, bytes for the Public Key (Key Agreement) schema are serialized to the corresponding Avro bytes field without the leading multicodec identifier. Readers should prepend or assume leading 0xec 0x01 bytes (indicating the x25519-pub multicodec). Writers should omit these leading bytes when inserting new items.

Source code for each schema is located in the LibertyDSNP/schemas repository.

Read Operation Mapping

Stateful data is retrieved via state queries (pallet.stateQuery) or RPC calls (pallet.rpcCall()).

Model TypeDSNP PropertiesQuery/Call
ItemizedEntity Tag
stateful_storage.getItemizedStorage()
PaginatedEntity Tag, Chunked
stateful_storage.getPaginatedStorage()

Write Operation Mapping

Write operations are via Transactions (also called Extrinsics): pallet::extrinsic()

Model TypeOperationPallet::Extrinsic
ItemizedInsert, Update, Delete
stateful_storage::apply_item_actions()
stateful_storage::apply_item_actions_with_signature()
PaginatedInsert, Update
stateful_storage::upsert_page()
stateful_storage::upsert_page_with_signature()
PaginatedDelete
stateful_storage::delete_page()
stateful_storage::delete_page_with_signature()

Entity Tags

Frequency requires the hash of current state for any Stateful Storage change.

Activity Content Specification

Version 1.3.0

Content references shared via the DSNP consist of URLs pointing to documents containing Activity Streams JSON objects. For the purposes of the DSNP, restrictions are placed on the Activity Streams 2.0 specification.

JSON-LD and Activity Streams

All DSNP Activity Content is compatible with the Activity Streams 2.0 specification. While there are some DSNP extensions, they are guaranteed to use non-colliding terms. Therefore, the JSON-LD @context field is set to https://www.w3.org/ns/activitystreams according to Activity Streams 2.0 §2.1.

Core Activity Content Types

DSNP uses only the following content types at the root level:

NameDescriptionDSNP Announcements
Notestandard user contentBroadcast, Reply, Update
Profileuser profile contentProfile

Associated Types

NameDescriptionSpecification
Locationadd a location to contentActivity Vocabulary
Tagadd a tag to contentActivity Vocabulary
Attachmentssupported attachment typesActivity Vocabulary
Hashcontent validation hashDSNP Extension

Supported URL Schema

URLs in DSNP-compatible Activity Content MUST use one of the following URL schemes:

SchemeDescriptionReferenceDSNP Version Added
HTTPSHypertext Transfer Protocol SecureRFC28181.0
HTTPHypertext Transfer ProtocolRFC26161.0

Libraries

Releases

VersionDescriptionRelease DateChangelog
1.3.0DIP-2572024-09-17Changelog
1.2.0DIP-2102023-04-11Changelog
1.1.0DIP-1582022-05-05Changelog
1.0.0Initial Release2021-09-09Changelog

Non-Normative

Additional Fields

Implementers may choose to support more of the Activity Streams standard as long as it does not conflict with this specification, but should note that other implementations may not recognize those additions. Implementers who extend their support for Activity Streams objects beyond the subset defined here do so at their own risk.

Activity Stream Type: Note

Activity Vocabulary: Note

PropertyBase SpecRequiredDescriptionRestrictions
@contextActivity Streams 2.0YESJSON-LD @contextMUST be set to https://www.w3.org/ns/activitystreams
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Note
contentActivity Vocabulary 2.0YESText content of the note
mediaTypeActivity Vocabulary 2.0YESMIME type for the content fieldMUST be set to a supported MIME type
publishedActivity Vocabulary 2.0YESThe time of publishingMUST be ISO8601
nameActivity Vocabulary 2.0noThe display name for the note
attachmentActivity Vocabulary 2.0noArray of attached links or mediaMUST be one of the Supported Attachments
tagActivity Vocabulary 2.0noArray of tags/mentionsMUST follow Tag Type
locationActivity Vocabulary 2.0noFor locationMUST follow Location Type

Supported Content MIME Types

FormatMIME TypeSpecification(s)
Plaintext/plain

Examples

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "content": "Hello world!",
  "mediaType": "text/plain",
  "published": "1970-01-01T00:00:00+00:00"
}
{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "content": "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.",
  "mediaType": "text/plain",
  "attachment": [
    {
      "type": "Link",
      "href": "https://en.wikipedia.org/wiki/Citation_needed"
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Activity Stream Type: Profile

Profiles are used to provide additional user information display.

Activity Vocabulary: Profile

PropertyBase SpecRequiredDescriptionRestrictions
@contextActivity Streams 2.0YESJSON-LD @contextMUST be set to https://www.w3.org/ns/activitystreams
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Profile
nameActivity Vocabulary 2.0noThe display name for the profile
iconActivity Vocabulary 2.0noAn array of avatars of the profileMUST follow Image Link Type
summaryActivity Vocabulary 2.0noUsed as a plain text biography of the profile
publishedActivity Vocabulary 2.0noThe time of publishingMUST be ISO8601
locationActivity Vocabulary 2.0noFor locationMUST follow Location Type
tagActivity Vocabulary 2.0noFor tags or mentionsMUST follow Tag Type

Examples

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Profile",
  "name": "John Doe",
  "summary": "John Doe is actually a small kitten. See pfp.",
  "icon": [
    {
      "type": "Link",
      "href": "https://upload.wikimedia.org/wikipedia/commons/thumb/1/1d/1-month-old_kittens_32.jpg/256px-1-month-old_kittens_32.jpg",
      "mediaType": "image/jpeg",
      "width": "256",
      "height": "171",
      "hash": [
        "bdyqphnphmjdoumkxqbsuspribxvlsx2hx6525u3fh2dkr5bxnqritzi"
      ]
    },
    {
      "type": "Link",
      "href": "https://upload.wikimedia.org/wikipedia/commons/thumb/1/1d/1-month-old_kittens_32.jpg/64px-1-month-old_kittens_32.jpg",
      "mediaType": "image/jpeg",
      "width": "64",
      "height": "43",
      "hash": [
        "bdyqktr7p5hc27bx4ernmngs6tj7uyukfb4atrtq44mdmx4yntuh2s5y"
      ]
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Associated Type: Hash

NOT part of the Activity Streams 2.0 Vocabulary.

Activity objects linking to external content such as audio, image or video files must include a "hash" field for users to validate linked content. The value of this "hash" field must be an array of strings. Each item in the array MUST be a valid DSNP Content Hash for the content associated with the hash.

Example

This example gives SHA-256 and BLAKE3 hashes for the PDF version of the DSNP whitepaper.

{
  "hash": [
    "bciqdnu347gcfmxzbkhgoubiobphm6readngitfywktdtbdocgogop2q",
    "bdyqhwoxp2mc6oyaqpqyd2fvaxralslk32ggazv6nxpp342iec6652tq"
  ]
}

Associated Type: Location

Activity Vocabulary: location

PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Place
nameActivity Vocabulary 2.0YESThe display name for the location
accuracyActivity Vocabulary 2.0noThe accuracy of the coordinates as a percentage. (e.g. “94.0” means “94.0% accurate”)
altitudeActivity Vocabulary 2.0noThe altitude of the location
latitudeActivity Vocabulary 2.0noThe latitude of the location
longitudeActivity Vocabulary 2.0noThe longitude of the location
radiusActivity Vocabulary 2.0noThe area around the given point that comprises the location
unitsActivity Vocabulary 2.0noThe units for radius and altitude (defaults to meters)MUST be one of these: cm, feet, inches, km, m, miles

Warning

Location data may pose a privacy danger to users. Users should be warned before publishing location data.

Example

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "content": "I'm in NYC!",
  "mediaType": "text/plain",
  "location": {
    "type": "Place",
    "name": "New York City, NY",
    "latitude": "40.73",
    "longitude": "-73.93",
    "accuracy": "94.0"
  },
  "published": "1970-01-01T00:00:00+00:00"
}

Associated Type: Tag

Activity Vocabulary: tag

Hashtag

PropertyBase SpecRequiredDescriptionRestrictions
nameActivity Vocabulary 2.0YESThe text of the tag

Example

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "content": "I love the #dsnp spec!",
  "mediaType": "text/plain",
  "tag": [
    {
      "name": "#dsnp"
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Mention

PropertyBase SpecRequiredDescriptionRestrictions
nameActivity Vocabulary 2.0noThe text of the tag
typeActivity Vocabulary 2.0YESIdentifies the tag as type MentionMUST be Mention
idActivity Vocabulary 2.0YESLink to the user mentionedMUST be a DSNP User URI

Example

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "content": "@sally should check out the #dsnp spec!",
  "mediaType": "text/plain",
  "tag": [
    {
      "name": "#dsnp"
    },
    {
      "name": "@sally",
      "type": "Mention",
      "id": "dsnp://12345678"
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Associated Attachments

Audio

Activity Vocabulary: Audio

PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Audio
urlActivity Vocabulary 2.0YESAn array of links for given audio content in different formatsMUST be an Audio Link AND MUST have at least one supported audio MIME type
nameActivity Vocabulary 2.0noThe display name for the audio file
durationActivity Vocabulary 2.0noApproximate duration of the audio
PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Link
hrefActivity Vocabulary 2.0YESThe URL for the given audio contentMUST be a Supported URL Schema
mediaTypeActivity Vocabulary 2.0YESMIME type of href contentMUST follow
hashDSNP 1.0YESArray of hashes for linked content validationMUST include at least one supported hash

Supported Audio MIME Types

FormatMIME TypeSpecification(s)
MP3audio/mpegRFC3003
OGGaudio/oggRFC5334
WebMaudio/webmWebM standard

Example

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "attachments": [
    {
      "type": "Audio",
      "name": "The Scream",
      "url": [
        {
          "type": "Link",
          "href": "https://upload.wikimedia.org/wikipedia/commons/d/d9/Wilhelm_Scream.ogg",
          "mediaType": "audio/ogg",
          "hash": [
            "bdyqbcji3okmzxobvaqgduz5prixmumyndzopyufultmslndi4pdebii"
          ]
        }
      ],
      "duration": "1S"
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Image

Activity Vocabulary: Image

PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Image
urlActivity Vocabulary 2.0YESAn array of links for given image content in different formatsMUST be an Image Link AND MUST have at least one supported image MIME type
nameActivity Vocabulary 2.0noThe display name or alt text for the image
PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Link
hrefActivity Vocabulary 2.0YESThe URL for the given imageMUST be a Supported URL Schema
mediaTypeActivity Vocabulary 2.0YESMIME type of href content
hashDSNP 1.0YESArray of hashes for linked content validationMUST include at least one supported hash
heightActivity Vocabulary 2.0noA hint as to the rendering height in device-independent pixels
widthActivity Vocabulary 2.0noA hint as to the rendering width in device-independent pixels

Supported Image MIME Types

FormatMIME TypeSpecification(s)
JPEGimage/jpegRFC2045
PNGimage/pngW3C PNG Standard
SVGimage/svg+xmlW3C SVG standard
WebPimage/webpWebP standard
GIFimage/gifRFC2045

Example

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "attachments": [
    {
      "type": "Image",
      "name": "One of the founders of DSNP",
      "url": [
        {
          "type": "Link",
          "href": "https://upload.wikimedia.org/wikipedia/commons/a/ae/Mccourt.jpg",
          "width": 350,
          "height": 228,
          "mediaType": "image/jpg",
          "hash": [
            "bciqjiqcidmzuqpvrl5cocu3l4z2uhj22xqruht3d5kx7qijvfbnjlda",
            "bdyqbjzt5drgji5w7xhsddsynusgx2vdmakcsrr4sfin5fyfkwlpup6q"
          ]
        }
      ]
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Video

Activity Vocabulary: Video

PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Video
urlActivity Vocabulary 2.0YESAn array of links for given video content in different formatsMUST be a Video Link AND MUST have at least one supported video MIME type
nameActivity Vocabulary 2.0noThe display name for the video
durationActivity Vocabulary 2.0noApproximate duration of the video
PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Link
hrefActivity Vocabulary 2.0YESThe URL for the given contentMUST be a Supported URL Schema
mediaTypeActivity Vocabulary 2.0YESMIME type of href content
hashDSNP 1.0YESArray of hashes for linked content validationMUST include at least one supported hash
heightActivity Vocabulary 2.0noA hint as to the rendering height in device-independent pixels
widthActivity Vocabulary 2.0noA hint as to the rendering width in device-independent pixels

Supported Video MIME Types

FormatMIME TypeSpecification(s)
MPEGvideo/mpegRFC2045
OGGvideo/oggRFC5334
WebMvideo/webmWebM standard
H265video/H265RFC7798
MP4video/mp4RFC4337

Example

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "attachments": [
    {
      "type": "Video",
      "name": "One of the founders of DSNP",
      "duration": "PT10M32S",
      "url": [
        {
          "type": "Link",
          "href": "https://upload.wikimedia.org/wikipedia/commons/c/c0/Big_Buck_Bunny_4K.webm",
          "width": 4000,
          "height": 2250,
          "mediaType": "video/webm",
          "hash": [
            "bdyqed7dnok3batd7tr64trqmovfxam5tqsgxkiv2op5765pq43swtui"
          ]
        }
      ]
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Activity Vocabulary: Link

PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Link
hrefActivity Vocabulary 2.0YESThe URL for the given linkMUST be a Supported URL Schema
nameActivity Vocabulary 2.0noThe display name for the link

Examples

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "content": "Check out the DSNP Website!",
  "mediaType": "text/plain",
  "attachment": [
    {
      "type": "Link",
      "name": "DSNP Website",
      "href": "https://dsnp.org"
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Attestations

Attestation attachments are DSNP extensions to the Activity Content model that allow users to attach Verifiable Credentials corresponding to an Attribute Set Type with their DSNP profile or a social media post.

The Verifiable Credential found at the indicated URL must include the user’s DSNP User URI in its credentialSubject.id field.

PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Attestation
urlActivity Vocabulary 2.0YESAn array of links for the given credentialMUST be a Verifiable Credential Link
nameActivity Vocabulary 2.0noThe display name for the attestation

Attestation attachments must contain a link to a Verifiable Credential document. The link must contain a relationship identifier in the form of a DSNP Attribute Set Type (the rel field) to allow applications to determine the type of credential expected.

PropertyBase SpecRequiredDescriptionRestrictions
typeActivity Vocabulary 2.0YESIdentifies the type of the objectMUST be set to Link
relActivity Vocabulary 2.0YESThe attestation’s attribute set typeMUST be a DSNP Attribute Set Type corresponding to the referenced credential document
hrefActivity Vocabulary 2.0YESThe URL for the associated Verifiable CredentialMUST be a Supported URL Schema
mediaTypeActivity Vocabulary 2.0YESMIME type of href contentMUST be set to
application/vc
hashDSNP extensionYESArray of hashes for linked content validationMUST include at least one supported hash

Example

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Profile",
  "content": "I am a doctor. You're doing this all wrong.",
  "mediaType": "text/plain",
  "attachments": [
    {
      "type": "Attestation",
      "name": "My Degree",
      "url": [
        {
          "type": "Link",
          "rel": "did:dsnp:123456$AcmeMedicalAssociationPhysician",
          "href": "https://acmemedicalassn.org/credentials/degree-123.json",
          "mediaType": "application/vc",
          "hash": [
		    "bciqdnu347gcfmxzbkhgoubiobphm6readngitfywktdtbdocgogop2q"
          ]
        }
      ]
    }
  ],
  "published": "1970-01-01T00:00:00+00:00"
}

Verifiable Credentials Specification

Version 1.3.0

Attribute Sets and Attestation attachments shared via DSNP reference data documents containing Verifiable Credentials and related objects. For DSNP purposes, certain restrictions and extensions are applied to the base World Wide Web Consortium (W3C) specification documents noted below. When there are DSNP extensions, they are guaranteed to use non-colliding terms.

It is anticipated that these W3C specifications will evolve, and applications interacting with Verifiable Credentials, Schemas, and DIDs via DSNP are encouraged to be as flexible as possible in their syntax support as new versions and supporting software libraries become available. Therefore, compatibility with the following versions and specifications should be construed as the minimal level of support required to ensure interoperability between DSNP applications.

Compatibility

Current usage with DSNP relies on the following specifications:

SpecificationVersion/StatusRelevant JSON-LD @context Values
Verifiable Credentials Data Model1.1 (W3C Recommendation)
2.0 (W3C Candidate Recommendation Draft 15 September 2024)
https://www.w3.org/2018/credentials/v1
https://www.w3.org/ns/credentials/v2
Verifiable Credential Data Integrity1.0 (W3C Candidate Recommendation Draft 16 September 2024)https://w3id.org/security/data-integrity/v2
https://w3id.org/security/multikey/v1
Verifiable Credentials JSON Schema(W3C Candidate Recommendation Draft 12 September 2024)
Decentralized Identifiers (DIDs)1.0 (W3C Recommendation 19 July 2022)https://www.w3.org/ns/did/v1

Cryptography

The Data Integrity specification provides a generic format for expressing cryptographic proofs, where the detailed representation of each data item is defined in individual cryptosuites. DSNP compliant applications MUST support the following cryptosuites, which correspond to the allowed algorithms for the DSNP User Data item assertionMethodPublicKeys:

SpecificationVersion/StatusMultikey codec
Data Integrity EdDSA Cryptosuites1.0 (W3C Candidate Recommendation Draft 16 September 2024)ed25519-pub

DSNP Usage Details

DSNP incorporates the following W3C JSON-LD types. See the individual pages for details of restrictions to and extensions on each type.

NameSince DSNP Version
Verifiable Credential1.3
Verifiable Credential Schema1.3
DID1.3

Libraries

NameLanguage(s)
LibertyDSNP/dsnp-verifiable-credentialsJavaScript/TypeScript
LibertyDSNP/dsnp-did-resolverJavaScript/TypeScript

Releases

VersionDescriptionRelease DateChangelog
1.3.0DIP-2572024-09-17Changelog

Non-Normative

Additional Fields

DSNP application developers may choose to support more of the relevant JSON-LD vocabularies from the specifications above as long as doing so does not conflict with this specification, but should note that other conforming applications may not recognize those additions. Implementers who extend their support for Verifiable Credentials objects beyond the subset defined here do so at their own risk.

Verifiable Presentations

Verifiable Presentations combine one or more Verifiable Credentials in a single document. These might be existing Verifiable Credentials, whether or not previously published, or new Verifiable Credentials derived from non-public source credentials using methods like zero-knowledge proofs. At this time, Verifiable Presentations are not directly tied to DSNP content or announcements. However, application and service providers may be interested in implementing functionality using Verifiable Presentations with DSNP Verifiable Credentials. Applications are encouraged to follow the verification guidelines for DSNP Verifiable Credentials even when encountering those credentials as part of a Verifiable Presentation.

Verifiable Credential

Credentials should conform to the Verifiable Credentials Data Model 1.1 or Verifiable Credentials Data Model 2.0, expressed as JSON-LD. The fields noted below consist only of those with requirements or semantics that differ from the underlying specification.

JSON-LD Contexts

The following JSON-LD context values are valid for use with DSNP:

Object@context
Verifiable Credentialhttps://www.w3.org/2018/credentials/v1
https://www.w3.org/ns/credentials/v2

Verifiable Credential Document

A DSNP Verifiable Credential JSON document specializes the following fields:

PropertyRequiredJSON TypeDescriptionRestrictions
@contextYESArray of stringsJSON-LD @contextMUST include "https://www.w3.org/2018/credentials/v1" or "https://www.w3.org/ns/credentials/v2"
typeYESArray of stringsType of credentialMUST contain "VerifiableCredential" per specification; if a credentialSchema is present, MUST also contain a string matching the referenced JSON Schema’s title.
issuerYESString or ObjectDID or object identifying credential issuerSee Issuer
credentialSubjectYESObjectObject identifying subject and claimSee Credential Subject
credentialSchemanoObjectReference to schema for this credentialSee Credential Schema
proofnoObjectCryptographic proof of authorship by issuerSee Proof

Issuer

The issuer field is required, even for credentials that do not include a proof. Unsigned documents should still use the document creator’s DID.

The issuer field may be a string that is the issuer’s DID, or an object with the issuer’s DID as the id property.

In the object form, the issuer may optionally include references to its own credentials, which may be used by a verifier processing the DSNP trust extension.

PropertyRequiredJSON TypeDescriptionRestrictions
idYESStringDID of issuerMust be a DSNP DID
authoritynoArrayList of relevant credentials with issuer as subjectSee Authority

Authority

Objects in the issuer.authority array MUST have the following properties:

PropertyRequiredJSON TypeDescriptionRestrictions
idYESStringURL of Verifiable CredentialMUST be a DSNP Verifiable Credential
relYESStringThe linked credential’s attribute set typeMUST be a DSNP Attribute Set Type corresponding to the referenced credential document
digestMultibaseYESArrayArray of hashes for linked content validationMUST include at least one supported hash

Examples:

"issuer": "did:dsnp:86420"
"issuer": {
  "id": "did:dsnp:86420",
  "authority": [
    {
      "id": "https://mydsnpcreds.net/86420-fp",
      "rel": "did:dsnp:123456$FairTradeProducer",
      "digestMultibase": [
        "bciqdnu347gcfmxzbkhgoubiobphm6readngitfywktdtbdocgogop2q"
      ]
    },
    {
      "id": "https://mydsnpcreds.net/86420-ap",
      "rel": "did:dsnp:123456$AppleProducer",
      "digestMultibase": [
        "bdyqhwoxp2mc6oyaqpqyd2fvaxralslk32ggazv6nxpp342iec6652tq"
      ]
    }
  ]
}

Credential Subject

PropertyRequiredJSON TypeDescriptionRestrictions
idYESStringSubject of the claim dataIf describing a DSNP User, MUST be a DSNP User URI; if describing DSNP content, MUST be a DSNP Content URI

The remainder of the contents of the credentialSubject value MUST conform to the JSON schema found via the credentialSchema object.

Credential Schema

PropertyRequiredJSON TypeDescriptionRestrictions
idYESStringURL of schema documentIf type is JsonSchemaCredential, MUST reference a DSNP Verifiable Credential Schema

Proof

PropertyRequiredJSON TypeDescriptionRestrictions
typeYESStringProof typeMUST be DataIntegrityProof
verificationMethodYESStringURI of public keyMUST reference a public key within a DSNP DID document
cryptosuiteYESStringCryptographic algorithm identifierMUST be eddsa-rdfc-2022
proofPurposeYESStringContext for assessing proofMUST be assertionMethod
proofValueYESStringDigital signature proofMUST be a multibase-encoded signature using base58btc encoding

Example

{
  "@context": [
    "https://www.w3.org/ns/credentials/v2"
  ],
  "type": [
    "VehicleOwner",
    "VerifiableCredential"
  ],
  "issuer": "did:dsnp:654321",
  "issuanceDate": "2024-02-12T03:09:40.497Z",
  "credentialSchema": {
    "type": "JsonSchemaCredential",
    "id": "https://dsnp.org/schema/examples/vehicle_owner.json"
  },
  "credentialSubject": {
    "id": "dsnp://999999",
    "make": "DeLorean",
    "model": "DMC-12",
    "year": 1981
  },
  "proof": {
    "type": "DataIntegrityProof",
    "created": "2024-02-12T03:09:44Z",
    "verificationMethod": "did:dsnp:654321#z6Mkumvf8FpJybzi9byLX7qAhTPuKpqH7d5rWyqcrKJ9Mies",
    "cryptosuite": "eddsa-rdfc-2022",
    "proofPurpose": "assertionMethod",
    "proofValue": "z2YLydotgaGsbRGRxPzmoscd7dH5CgGHydXLKXJXefcT2SJGExtxmkJxGfUGoe81Vm62JGEYrwcS6ht1ixEvuZF9c"
  }
}

Verifiable Credential Schema

The schema for a DSNP Verifiable Credential MAY be defined using the format described in the Verifiable Credentials JSON Schema W3C Candidate Recommendation Draft. This format provides a credential wrapper around a JSON schema document.

Empty schemas (Verifiable Credential Schema Documents with "jsonSchema": {}) are allowed; however, schemaless credentials may be preferred in this situation. Empty schemas are useful in situations where no attribute data fields are relevant but the schema author wishes to assert authorship.

Verifiable Credential Schema Document

A Verifiable Credential Schema document is a JSON-LD document that is itself a Verifiable Credential with a claim that includes the schema and (optionally) the DSNP extension described below. A schema credential specializes the meaning of the following fields:

PropertyRequiredJSON TypeDescriptionRestrictions
typeYESArray of stringsType of credentialMUST contain the strings "VerifiableCredential" and "JsonSchemaCredential"
credentialSubjectYESObjectObject containing JSON schema and DSNP extensionsSee Credential Subject
credentialSchemaYESObjectMetaschema defining JSON Schema typesSee Credential Schema

Credential Schema

The required credentialSchema object MUST follow the specification and contain:

{
  "id": "https://www.w3.org/2022/credentials/v2/json-schema-credential-schema.json",
  "type": "JsonSchema",
  "digestSRI": "sha384-S57yQDg1MTzF56Oi9DbSQ14u7jBy0RDdx0YbeV7shwhCS88G8SCXeFq82PafhCrW"
}

Credential Subject

A DSNP Verifiable Credential Schema document’s credentialSubject object uses the following keys:

PropertyRequiredJSON TypeDescriptionRestrictions
typeYESStringType of subject matterMUST be JsonSchema
jsonSchemaYESObjectEmbedded JSON Schema objectSee JSON Schema
dsnpnoObjectDSNP extension objectSee DSNP Extensions

JSON Schema

The JSON Schema object is formed as follows:

PropertyRequiredJSON TypeDescriptionRestrictions
$schemaYESStringJSON Schema version identifierMUST be a valid DSNP JSON Schema version
titleYESStringTitle of the schemaMUST match a string within the referencing credential’s type array

The remainder of the schema object should be interpreted as per the relevant JSON schema specification.

Valid DSNP JSON Schema Versions

DSNP applications MUST support the following JSON Schema versions:

$schema valueSpecification
https://json-schema.org/draft/2020-12/schemaJSON Schema 2020-12 Update

Other JSON Schema versions MAY be supported, but creators should be aware that not all DSNP applications will be able to correctly validate credentials against schema definitions that are not universally supported.

DSNP Extensions

An optional dsnp object within the credentialSubject provides additional semantics for DSNP applications and users interacting with the schema.

Display Extension

The optional display key within the dsnp extension object MUST contain a label key that has an object value where the object is a map of one or more language tags to human-readable string values. Language tags should follow BCP-47/RFC-5646 (as used in the HTTP Content-Language header). A content language key of "*" indicates a wildcard or default value, as in HTTP.

Example:

"display": {
  "label": {
    "en-US": "Verified User",
    "de-DE": "Verifizierter Benutzer"
  }
}

A DSNP application MAY use the indicated values as a hint when displaying information about the subject of a Verifiable Credential utilizing this schema.

Trust Extension

The optional trust key within the dsnp extension object enables the author of a schema to describe the types of Verifiable Credentials that an issuer of credentials with this schema must have to be considered an accredited source and therefore trusted by an application. These accreditations MUST be included by the issuer using the authority key under issuer in each relevant Verifiable Credential document they create.

The trust object contains one or both of the oneOf or allOf keys, each containing an array of strings. Each string MUST be a valid DSNP Attribute Set Type.

See Attribute Sets for a detailed discussion of Attribute Set Types and their derivation.

A trust object containing both oneOf and allOf sections should be construed as requiring any of the oneOf constraints as well as all of the allOf constraints.

Example:

"trust": {
  "oneOf": [
    "did:dsnp:123456$AppleProducer",
    "did:dsnp:123456$OrangeProducer"
  ],
  "allOf": [
    "did:dsnp:999999$FairTradeProducer"
  ]
}

The above example might be translated as “A credential conforming to this schema should only be trusted if its issuer is accredited as a FairTradeProducer and is also either an AppleProducer or an OrangeProducer.”

Example

{
  "@context": [
    "https://www.w3.org/ns/credentials/v2"
  ],
  "id": "https://dsnp.org/schema/examples/vehicle_owner.json",
  "type": [
    "VerifiableCredential",
    "JsonSchemaCredential"
  ],
  "issuer": "did:dsnp:123456",
  "issuanceDate": "2024-02-12T03:09:40.497Z",
  "expirationDate": "2099-01-01T00:00:00.000Z",
  "credentialSchema": {
    "id": "https://www.w3.org/2022/credentials/v2/json-schema-credential-schema.json",
    "type": "JsonSchema",
    "digestSRI": "sha384-S57yQDg1MTzF56Oi9DbSQ14u7jBy0RDdx0YbeV7shwhCS88G8SCXeFq82PafhCrW"
  },
  "credentialSubject": {
    "type": "JsonSchema",
    "jsonSchema": {
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "title": "VehicleOwner",
      "type": "object",
      "properties": {
        "credentialSubject": {
          "type": "object",
          "properties": {
            "make": {
              "type": "string"
            },
            "model": {
              "type": "string"
            },
            "year": {
              "type": "number"
            }
          },
          "required": [
            "make",
            "model",
            "year"
          ]
        }
      }
    },
    "dsnp": {
      "display": {
        "label": {
          "en-US": "Vehicle Owner"
        }
      },
      "trust": {
        "oneOf": [
          "did:dsnp:123456$AuthorizedCarDealership",
          "did:dsnp:123456$OfficialTaxOffice"
        ]
      }
    }
  },
  "proof": {
    "type": "DataIntegrityProof",
    "created": "2024-02-12T03:09:44Z",
    "verificationMethod": "did:dsnp:123456#z6MkhFqFV1UD6GYbQ1V4HSF3pnGprovQceXbgwbLKrhxnbny",
    "cryptosuite": "eddsa-rdfc-2022",
    "proofPurpose": "assertionMethod",
    "proofValue": "z3ENMBo7UyKvZkJSBMqvGFDB1uvGChP1QTEgiCsNzt23ciq4RTffk2xxz2noXxxHL6wPmN1Bp2fSvTuMnvHQFP9tp"
  }
}

DIDs

DSNP Users are referenced in Verifiable Credentials documents via DIDs compliant with the Decentralized Identifiers v1.0 specification.

Applications that make use of Verifiable Credentials issued by DSNP Users MUST be able to resolve DSNP User DIDs to DID documents in order to verify that signatures were created with keys controlled by that user. A DSNP DID document is effectively a document aggregating DSNP User Data. Compliant DSNP systems are encouraged to provide their own DID resolver libraries.

DID Method Syntax

A DSNP DID uses dsnp (lowercase) as the method name and the DSNP User Id as the method-specific identifier, as the following example illustrates:

did:dsnp:123456

This DID identifies the DSNP User with User Id 123456. In this format, the DSNP User Id is serialized in decimal form with no additional punctuation.

References to identifiers within a DID document are formed by appending URL fragments to a DID. For example, a Verifiable Credential might reference the public key to be used to verify a document’s signature as did:dsnp:123456#key1, assuming the document included a verificationMethod with "id": "key1".

DID Document

A DSNP DID document is a JSON-LD document representing key material associated with a DSNP User.

PropertyRequiredJSON TypeDescriptionRestrictions
@contextYESArray of stringsJSON-LD @contextMUST include "https://www.w3.org/ns/did/v1"
idYESStringThe DID described by this documentMUST be of the form did:dsnp:{userId}
verificationMethodnoArray of Verification Method objectsSet of public keys that may be referenced from assertionMethod, authentication, and keyAgreement arrays
assertionMethodnoArraySet of public keys used to generate digital signaturesMUST include or reference all relevant keys present in DSNP User Data assertionMethodPublicKeys
authenticationnoArraySet of public keys used as DSNP control keysMAY include or reference any keys used as control keys
keyAgreementnoArraySet of public keys used to generate shared secretsMUST include or reference all relevant keys present in DSNP User Data keyAgreementPublicKeys

Additional properties defined in the DID specification MAY be present.

Public Key Representation

As per the DID specification, each element of the assertionMethod and keyAgreement arrays may be a string reference to the object with a matching id field in the verificationMethod array, or the Verification Method object itself. Each Verification Method describes a public key. These keys MUST be taken from the DSNP User Data associated with the user referenced by the DID.

DSNP DID resolvers MUST serialize keys in the Multikey format, defined in Verifiable Credential Data Integrity 1.0.

PropertyRequiredJSON TypeDescriptionRestrictions
@contextYESStringJSON-LD @contextMUST be https://w3id.org/security/multikey/v1
idYESStringThe full URI of this keyMUST be of the form did:dsnp:{userId}#{identifier}; MUST be unique within the document
typeYESStringThe type of this verification methodMUST be Multikey
controllerYESStringThe controller of this keyMUST be the DID of the enclosing document
publicKeyMultibaseYESStringThe public keyMUST be a valid multicodec-prefixed public key in base58btc encoding

Example

{
  "@context": [
    "https://www.w3.org/ns/did/v1"
  ],
  "id": "did:dsnp:645313",
  "authentication": [
    {
      "@context": "https://w3id.org/security/multikey/v1",
      "id": "did:dsnp:645313#z6QP1gZa1xAGCtsPzZSc5mdTDtrGsWUyf12TmU6pSu15SXUr",
      "type": "Multikey",
      "controller": "did:dsnp:645313",
      "publicKeyMultibase": "z6QP1gZa1xAGCtsPzZSc5mdTDtrGsWUyf12TmU6pSu15SXUr"
    }
  ],
  "assertionMethod": [],
  "keyAgreement": [
    {
      "@context": "https://w3id.org/security/multikey/v1",
      "id": "did:dsnp:645313#z6LSoYFtPRBEizFQ2zYXEXBPP96t9gNFWjZVJTfXWqzMhw9e",
      "type": "Multikey",
      "controller": "did:dsnp:645313",
      "publicKeyMultibase": "z6LSoYFtPRBEizFQ2zYXEXBPP96t9gNFWjZVJTfXWqzMhw9e"
    }
  ]
}

Glossary

  • Activity Content - User-generated social networking content in a JSON format defined by the Activity Streams 2.0 specification with DSNP extensions. Some DSNP Announcements contain the URL and hash for an Activity Content document.
  • Announcement - Content, or references to content, that communicate user activity to the network.
  • Application - A computer program that helps a User interact with a DSNP system
  • Batch Publication - A collection of Announcements bundled together into a single file using the Parquet format
  • Consumer - An application or user who reads content from a DSNP system
  • DSNP System - A consensus system (and its surrounding services) that enables DSNP operations and generates DSNP state change records
  • Hash - A string of bytes generated from a hash function, a cryptographic function whose output is effectively unique for any given input without any information from the input being accessible in the output
  • User - A human using a DSNP application. See article for an exception