DSNP Specification
Welcome to the Decentralized Social Networking Protocol (DSNP) specification. Here you can find the detailed specification documentation for DSNP, official DSNP system specifications, and associated specifications.
Goals & Purpose
Free communication among users on the Internet faces a variety of problems in the modern day. These challenges include censorship by state and corporate actors, the amplification of misinformation through viral content, and an ever-shrinking collection of near monopolies with absolute power over social interaction in the twenty-first century. Through DSNP, we hope to mitigate and ideally solve these challenges in the way social interaction operates online.
How to Read This Specification
There are three core specifications and one system specification currently. Each specification is divided into several modules. Each module describes a mostly self-contained aspect of the specification.
Core Specifications
Name | Version | Description |
---|---|---|
DSNP | 1.3.0 | The system-agnostic DSNP specification |
Activity Content | 1.3.0 | A specification for DSNP-referenced content (subset of W3C Activity Streams) |
Verifiable Credentials | 1.3.0 | Use of W3C Verifiable Credentials and DIDs with DSNP |
System Specifications
Consensus System | DSNP System Specification |
---|---|
Frequency | DSNP Over Frequency |
Versioning
DSNP specification versions follow Semantic Versioning 2.0 for releases.
Contributions
Development occurs on GitHub. All interactions must follow the Code of Conduct and Contribution Guidelines.
Learning More
In addition to this document, more resources regarding the project can be found at DSNP.org, including the blog, forum, code repositories, and other supporting software libraries and examples.
DSNP Specification
Version 1.3.0
DSNP (Decentralized Social Networking Protocol) is a social networking protocol designed to run on a blockchain. It specifies a set of social primitives along with requirements for interoperability. Go to Systems for specifics on how DSNP is expressed on a specific blockchain.
Overview
What is a DSNP System?
A DSNP system is a state machine which generates an ongoing, publicly observable and verifiable stream of state change records in response to public input. This specification defines a set of operations (the DSNP Operations) that a compliant DSNP system MUST make available. For each DSNP Operation, the DSNP specification defines the data to be provided as input, the data that will be generated as output, and the state change records (the DSNP State Change Records) that will be observable as a result of the Operation, depending on the input and prior state of the system.
The nature of blockchain technology means that all DSNP Operations are potentially asynchronous; that is, DSNP State Change Records need not be created immediately upon invoking a DSNP Operation, and may appear in the stream of state changes at any subsequent time. A DSNP Operation can therefore only be confirmed to have completed successfully by observing the stream of newly-generated DSNP Records.
All DSNP Records MUST be publicly visible. On a blockchain, for example, all nodes must share a common set of Records, and a DSNP system specification MUST define how external systems can subscribe to or otherwise view all state changes. In practical terms, this means that all users of a given DSNP system exist conceptually within a single shared social graph, and a given user MUST have the ability at a protocol level to see and participate in all public discourse. Note, however, that this is not an application-level requirement; an application that interacts with DSNP might choose to filter what content is shown.
Finally, to ensure decentralization, a DSNP system MUST avoid having any single point of failure (many nodes) and MUST avoid having any single entity that can override its consensus mechanisms (many operators).
DSNP System Compliance
Compliant DSNP system specifications MUST document how each of the required DSNP Operations (for a given version of the DSNP specification) can be invoked, including, for example, what wire protocols should be used for discovery, authentication and operation execution, and how inputs and outputs will be serialized.
A compliant specification MUST specify a mapping from its system-specific state change data (for example, the events emitted by a blockchain) to the DSNP State Change Records that data represents.
Releases
DSNP system specifications MUST specify the version(s) of the DSNP specification that may be used.
Version | Description | Release Date | Changelog |
---|---|---|---|
1.3.0 | DIP-257, DIP-263, DIP-267, DIP-273 | 2024-09-17 | Changelog |
1.2.0 | DIP-210, DIP-216, DIP-220, DIP-225, DIP-226, DIP-227, DIP-228 | 2023-04-11 | Changelog |
1.1.0 | DIP-148, DIP-149, DIP-150, DIP-165, DIP-180 | 2022-05-06 | Changelog |
1.0.0 | Initial Release | 2021-09-09 | Changelog |
Identity
This specification is intended to cover the concept of identity and how identity is represented within the protocol.
Identifiers
DSNP Identifiers form the basis for pseudo-anonymous streams of content. While some users may choose to link or expose their real-world identity, DSNP implementations MUST NOT require such data exposure for account creation. The social graph is formed using this Identifier so that a user’s connections maintain integrity regardless of changes in any user’s client choices or access changes.
Pseudo Anonymity
- An Identifier MUST default to being disconnected from a real-world identity.
- An Identifier MUST be unique to the implementation.
Ownership
A user’s ownership of their identity is expressed via ownership and control of their pseudo-anonymous Identifier(s). Control entails the power to invoke DSNP Operations including publishing Announcements that create, update and delete (Tombstone) content associated with the Identifier, delegating these powers to others, managing keys associated with the Identifier, and retiring the Identifier.
Key Management
An initial control key must be created in order to acquire an Identifier. Each distinct Identifier MUST have distinct control keys; that is, the same key MUST NOT be linked to multiple Identifiers. Optionally, an implementation MAY allow the user to add and remove additional keys. An implementation MUST NOT allow the user to remove the only or last remaining control key.
Retirement
A user may choose to retire their Identifier at any time. Once an Identifier is retired, an implementation MAY remove all state data associated with that Identifier, provided that an indication that the Identifier is retired remains, so it may not be reused in the future. This means that all data previously sent from the Identifier, the keys associated with the Identifier, any user data (see below) associated with the Identifier, and the delegations (see next section) associated with the Identifier may be removed.
After an Identifier is retired, any existing or future Announcements from the Identifier should be treated as if they have been tombstoned (for Announcement Types that support tombstoning). A retired Identifier MUST NOT be allowed to act as a principal for any additional DSNP Operations.
Delegation
- A user MUST be able to delegate and revoke permissions to other parties to submit each type of announcement or replace each type of user data on their behalf.
- A user MUST be able to revoke delegated permissions.
- Announcements and user data updates from a delegate MUST be able to be verified as to which delegate was responsible for the action.
- Delegation revocation MUST NOT be retroactive.
Identity-Linked State Data
DSNP identities can have linked User Data, which, like the identifier and delegations, forms part of the shared state of the consensus system. As consensus system state storage is generally much more expensive than external data storage, shared state storage is only required for data considered critical for maintaining user agency and discoverability with low latency and high availability.
User data is utilized for the following types of identity-linked data:
- Public keys that can be used to encrypt data that can later be decrypted by the user or used to verify signatures on documents created by the user
- Social Graphs, both public and private (encrypted)
- Links to Profile documents
Related Operations
- Create Identifier
- Retire Identifier
- Define Delegation
- Revoke Delegation
- Add Control Key
- Remove Control Key
- Replace User Data
- Get User Data
Non-normative
Retroactive Revocation of Delegation
There are times when one might desire retroactive revocation of delegation, if for example, a key were found to have been compromised at an earlier time. However, retroactive revocation is much more difficult from a caching and performance perspective. Instead reverting any undesirable Announcements via Tombstones allows a user to choose which specific events need reverting and then notify the network of that change.
To this end, any Batch Publications can always be validated from the perspective of the time they were published, and do not require re-validation at future read times. The validity of a Batch is thus immutable.
- Collect all the DSNP Ids used in a Batch.
- Validate that each DSNP Id had delegated to the publisher of the Batch at the time of publishing.
- Validate that any failures from step 2 were from DSNP Ids that revoked delegation within the acceptance window of prior blocks.
- Record the batch as valid or invalid.
Identifiers
The DSNP Identifiers form the basis for pseudo-anonymous streams of content. Graph connections are formed through the DSNP User Id.
DSNP User Id
- 64-bit Unsigned Integer
- MUST be serialized as decimal
- MUST be unique per implementation
DSNP Content Hash
- MUST be a multibase string using the
base32
encoding - MUST represent a valid multihash encoding of the hashing algorithm output for the bytes of the content
- MUST use a Supported Hashing Algorithm
Serialization Steps
- Apply the Supported Hashing Algorithm to create a digest of the content.
- Prepend the leading bytes from the table below indicating the hashing algorithm in the multicodec table and the length of the hash output.
- Serialize as a base32 multibase string.
Example
- Applying the BLAKE3 algorithm to the DSNP Whitepaper yields the following 32 bytes:
0x3a0393e3ee6c6fec1b13885763225fd0927884b2d431ed262899523ade281cb4
. - Prepending the multihash indicator (
0x1e
forblake3
) and hash length (0x20
for 32 bytes) gives0x1e203a0393e3ee6c6fec1b13885763225fd0927884b2d431ed262899523ade281cb4
. - Serializing as a base32 multibase string gives us the final DSNP Content Hash of
bdyqdua4t4pxgy37mdmjyqv3dejp5betyqsznimpneyujsur23yubzna
.
Supported Hashing Algorithms
Algorithm | Multihash Name | Leading bytes | Reference | DSNP Version Added |
---|---|---|---|---|
SHA-256 | sha2-256 | 0x1220 | RFC 6234 | 1.2 |
BLAKE3 | blake3 | 0x1e20 | blake3.io | 1.3 |
DSNP Protocol Scheme
- MUST always be the string
dsnp://
DSNP User URI
The DSNP User URI consists of two parts: the scheme and the user id. It is used to identify a user via a URI.
Example
dsnp://1311768467294899700
part | value |
---|---|
Scheme | dsnp:// |
User Id | 1311768467294899700 |
DSNP Content URI
The DSNP Content URI consists of three parts: the scheme, the user id, and the content hash. It is used to uniquely identify an Announcement from a given user with content.
Any Announcement Types with a fromId
and contentHash
have a DSNP Content URI.
When encoding a DSNP Content URI, the contentHash
field MUST be serialized exactly as it appears in the Announcement (that is, as a base32 multihash string).
Example
dsnp://78187493520/bdyqdua4t4pxgy37mdmjyqv3dejp5betyqsznimpneyujsur23yubzna
part | value |
---|---|
Scheme | dsnp:// |
User Id | 78187493520 |
Content Hash | bdyqdua4t4pxgy37mdmjyqv3dejp5betyqsznimpneyujsur23yubzna |
Graph
This specification describes the social network graph and how it is represented in the protocol.
In this context a social graph means a graph that represents social relationships between entities.
Nodes
Users are represented in the DSNP graph by nodes uniquely identified by their DSNP User Ids. Relationships between users are the graph edges.
Edges
Graph edges that originate from a user are stored as DSNP User Data associated with a given user. DSNP user data structures capture three distinct sets of relationships, as described below.
Public Follows
Each public follow records a unidirectional relationship where one user (the controller of the graph) is interested in following the public speech of another user. When follows are public, they are visible to any other user or system, including the user being followed.
Private Follows
A private follow records an interest in following the public speech of another user, but the data is stored in encrypted form. The relationship is not visible to the followed user or any other user that does not have the decryption key.
Private Connections
Private connections represent one half of a bidirectional relationship between two DSNP Users. Private connections are stored on both sides of the bidirectional relationship between two DSNP Users. Each user can then update the status of the relationship without loss of privacy. Each DSNP user maintains and controls two sets of data:
- An encrypted data set that records which users are recognized as connections.
- A public data set that is used to signal connection status via Pseudonymous Relationship Identifiers.
Pseudonymous Relationship Identifiers
Private connections are complemented by publishing private connection Pseudonymous Relationship Identifiers (PRIds), which provide a means of making the connection relationship visible to the other user while maintaining privacy with respect to any third parties. Applications are encouraged to verify connection relationships by checking to see that they are reciprocated, with the appropriate identifier present in both users’ data.
For each pair of users Alice and Bob, and a specified PRId context, a pair of PRIds can be generated from Alice and Bob’s shared secret, with one representing a simplex channel from Alice to Bob, and the other representing the corresponding simplex channel from Bob to Alice.
PRIds have the following properties:
- If Alice has generated a PRId from Alice to Bob, only Bob can generate the same PRId. This allows Bob to check if he is listed in Alice’s published connections.
- Bob can prove to a third party that a given PRId in Alice’s published connections is his, without allowing the third party to discover Alice or Bob’s relationships with any other user.
A detailed discussion of the cryptography for PRIds can be found in the PRId object definition.
User Data
DSNP stores user-related data of well defined formats associated with a user’s DSNP Identifier. DSNP User Data is designed for data with the following characteristics:
- The data is intended to be accessible across the network with the strongest possible durability guarantees
- The data may change frequently.
- The data does not need to be widely announced and instead can be read just-in-time for a particular user.
Data may be public or stored in encrypted form. The Avro schema specification is used to define the binary representation of relevant data types.
User Data Types
DSNP implementations MUST support the following User Data Types:
System Name | Version | Encryption Algorithm | Compression Codec | Avro Object Type |
---|---|---|---|---|
publicFollows | 1.2 | NONE | ||
DEFLATE | GraphEdge | |||
privateFollows | 1.2 | curve25519xsalsa20poly1305 | ||
DEFLATE | GraphEdge | |||
privateConnections | 1.2 | curve25519xsalsa20poly1305 | ||
DEFLATE | GraphEdge | |||
privateConnectionPRIds | 1.2 | NONE | NONE | PRId |
keyAgreementPublicKeys | 1.3 | NONE | NONE | PublicKey |
assertionMethodPublicKeys | 1.3 | NONE | NONE | PublicKey |
profileResources | 1.3 | NONE | NONE | ProfileResource |
Data for each data type is initially formatted as a stream of Avro objects that should conform to the schema specified. A DSNP system MAY limit the number of objects allowed for a given user data type; if so, this MUST be documented. Avro file- and block-level information (including in-stream schema) is omitted. The Avro stream is then compressed and/or encrypted as specified.
curve25519xsalsa20poly1305
(that is, X25519 key exchange, XSalsa20 encryption, and Poly1305 message authentication) is the default authenticated encryption algorithm used in the NaCl (“Salt”) library, and its successor libsodium.
In the specification of cryptographic operations below, relevant methods from these libraries are noted. While these specific libraries are not required for DSNP compatibility, they are highly recommended.
Data Chunks
Because consensus systems often have specific limits to the amount of data that can be included in a given transaction, operations on user data deal with the data in discrete chunks. As implementation strategies may vary, implementations MUST define their own maximum chunk size in bytes to be used in the operations described below.
Entity Tags
To allow for application-layer caching and prevent race conditions between different applications attempting to update the same data, an implementation MUST provide a non-empty etag
(entity tag) value with each chunk fetched, and update this for each chunk replaced.
An etag
is any ASCII-encoded string with a minimum length of one byte and a maximum length of 255 bytes.
An implementation MUST fail the Replace User Data Operation if the etag
values supplied do not match the current etag
values for all chunks of the specified data type.
Replace User Data Operation
The Replace User Data Operation takes the following parameters:
- A DSNP User Id
- Implementations MUST ensure that the principal invoking this Operation is this user, or a transparent chain of delegation from the user to the principal exists.
- The index of the
keyAgreementPublicKeys
key pair used to encrypt any private data in the operation. (If only unencrypted user data types are included, the key index may be omitted.) - A map containing the set of data types to update as the keys, and tuples consisting of (1) the schema version used to encode the data type, and (2) a list where each element includes a data chunk and its associated entity tag, as the values.
If the Operation is successful, any previous data associated with the user for each data type included in the input MUST be removed and replaced by the new data.
Data chunks should be generated for each included data type using the following sequence of operations:
- Serialize the data records, in Avro binary format, according to the versioned schema.
- Divide the data records into one or more chunks, with each chunk containing a maximum byte count specified by the implementation.
- Note that the combined chunks constitute a different, and smaller, binary than a complete Avro file, as they exclude the header information, schema, and any block-level metadata. Each chunk should start and end with a complete Avro object; in other words, chunk boundaries MUST NOT occur mid-record. Applications SHOULD try to include as many records as possible within the byte length limit, but because compression and encryption may alter the byte size versus the raw Avro binary records, they may wish to use a heuristic approach that sometimes produces non-optimal chunks.
- For each chunk generated, the application should then:
- If the data type requires compression, apply the compression codec noted.
- If the data type requires encryption,
- Retrieve the user’s active
keyAgreementPublicKey
key, Upublic, and note its index. If no key exists, one should be created and published as User Data before invoking the Operation. By convention, the key with the highest index (the last object in the Avro stream) is the active key. - Create a sealed box (a payload encrypted with a symmetric key derived from an ephemeral key pair, and accompanied by the ephemeral public key), as in the libsodium function
crypto_box_seal
, using Upublic. - Include the previous
etag
value for the chunk. If the chunk is new,etag
should be set tonull
. If any chunks are to be deleted, they should be included in the input identified with the existingetag
and anull
value for the data.
- Retrieve the user’s active
If the Operation is invoked successfully the implementation MUST synchronously return a new set of etag
values for each data type replaced, corresponding to the updated state of the data (with new chunks added and deleted chunks removed).
Applications should not interpret this response as an indication that the operation was completed and a state change record emitted, as this typically occurs asynchronously.
However, this strategy allows applications to make the optimistic assumption that, in due course, the DSNP system will reflect the intended changes, without needing to wait for asynchronous confirmation.
If, on the other hand, an invocation of Replace User Data is rejected due to entity tag discrepancies, this indicates that the relevant data on the network has changed since the entity tags were acquired by the application, and the application should fetch the most current version with the Get User Data Operation, reapply any intended changes, and retry the operation.
The Replace User Data Operation MUST generate a User Data Replaced Record containing the DSNP User Id and the set of updated User Data Types (but not the data itself). If the implementation detects that no change has occurred, it SHOULD omit the relevant unchanged data types from the state change record.
Examples
The following section is non-normative. The JSON schema and encoding used is provided for illustration only and implementations are free to define their own encoding.
The following example illustrates the input to the Replace User Data Operation corresponding to the following scenario (utilizing social graph data types):
- The user’s previously stored data consists of one chunk for public follows, two chunks for private connections, and one chunk for private connection declarations.
- The user adds several follows to their public list, causing it to exceed the maximum capacity for a single chunk and require a new chunk.
- The user adds a relationship to their private connections, but it still fits in two chunks. The private connection’s PRId is added to the private connection PRId list, but it remains a single chunk.
{
"publicFollows": {
"version": "1.2",
"chunks": [
{
"etag": string // unchanged chunk
},
{
"data": base64(compress(chunk2)),
"etag": null, // new chunk
}
]
},
"privateFollows": {
"version": "1.2",
"chunks": [
{
"etag": string // unchanged chunk
},
{
"data": base64(encrypt(compress(chunk2))), // updated chunk
"etag": string
}
]
},
"privateConnectionPRIds": {
"version": "1.2",
"chunks": [
{
"data": base64(compress(chunk1)),
"etag": string
}
]
}
}
Deletion of records may cause situations where the number of chunks decreases.
To ensure that the deleted chunk was up to date, the deleted chunk should still be included in the array with the relevant entity tag value, using an explicit null
value for the data field to indicate deletion, as in the following snippet:
"publicFollows": {
"version": "1.2",
"chunks": [
{
"etag": string // unchanged chunk
},
{
"data": null, // deleted chunk
"etag": string,
}
]
}
Get User Data Operation
The Get User Data Operation takes the following parameters:
- The DSNP User Id of the user who controls the data
- Note: While writing user data is reserved for the user and any delegates, anyone on the network can read any user’s data (though it may be encrypted).
- The User Data Types (by system name) that should be retrieved.
The operation returns a mapping of User Data Type to data chunks, with each data chunk annotated with an entity tag and (optionally) a key index. (Note that this is the same general structure as the input data for Replace User Data, for each requested data type. If no chunks for a requested data type exist, an implementation MAY omit that data type from the response.
To transform the data from the output to Avro binary records, a consumer should apply the following algorithm to each data type included:
- Determine the relevant encryption algorithm, compression codec, and object schema from the User Data Type and version noted.
- For each chunk,
- If encryption is indicated, decrypt the chunk data using the user’s secret key (identified using the key index) as in the libsodium function
crypto_box_seal_open
. - If compression is required, uncompress the chunk data using the specified codec.
- Deserialize the uncompressed data to logical records according to the Avro object schema.
- Retain the chunk’s
etag
value if needed for any updates.
- If encryption is indicated, decrypt the chunk data using the user’s secret key (identified using the key index) as in the libsodium function
Examples
The following section is non-normative. The JSON schema and encoding used is provided for illustration only and implementations are free to define their own encoding.
The following example illustrates the output of a Get User Data Operation invocation requesting data for publicFollows
, privateConnections
, and privateConnectionPRIds
:
{
"publicFollows": {
"version": "1.2",
"chunks": [
{
"data": base64_string,
"etag": string
},
{
"data": base64_string,
"etag": string
}
]
},
"privateConnections": {
"version": "1.2",
"chunks": [
{
"data": base64_string,
"etag": string,
"keyId": integer
},
{
"data": base64_string,
"etag": string,
"keyId": integer
}
]
},
"privateConnectionPRIds": {
"version": "1.2",
"chunks": [
{
"data": base64_string,
"etag": string
}
]
}
}
Graph Edge
Relationships between users are represented using the GraphEdge object.
Serialization
GraphEdge object serialization MUST conform to the following Avro schema:
{
"namespace": "org.dsnp",
"name": "GraphEdge",
"type": "record",
"doc": "A relationship to another DSNP user",
"fields": [
{
"name": "userId",
"type": "long",
"doc": "The other user's DSNP User Id"
},
{
"name": "since",
"type": "long",
"doc": "Timestamp in Unix epoch seconds when this relationship was originally established"
}
]
}
Generation
userId
- MUST be a DSNP User Id
To allow for optimal compression, User Ids are stored using the long type in Avro schema, which is a 64-bit signed integer. Care should be taken to ensure that User Id values greater than or equal to 263, where used by an implementation, are converted correctly between signed and unsigned representations.
since
- MUST be a value in seconds since the Unix epoch
PRId
A Pseudonymous Relationship Identifier is represented by the PRId object type.
Serialization
PRId object serialization MUST conform to the following Avro schema:
{
"namespace": "org.dsnp",
"name": "PRId",
"type": "fixed",
"size": 8,
"doc": "Pseudonymous Relationship Identifier"
}
Generation
PRIds are generated cryptographically to represent a relationship from one user to another within a specified context, in a privacy-preserving manner.
Contexts
The following context values are currently defined for PRIds. All other values are reserved for future use.
Context Id | Description | Context string |
---|---|---|
0 | Connection | PRIdCtx0 |
Algorithm
In the following section, the Alice to Bob identifier for context C is called PRIdA→B,C, and the corresponding Bob to Alice identifier is called PRIdB→A,C.
A PRId is derived from Alice and Bob’s keyAgreement
key pairs, using a key exchange protocol as follows. To illustrate the cryptographic operations required, the relevant functions from libsodium are noted. Sodium is a stable, fast, free, and cross-platform cryptography library, and supports all encryption algorithms used in the DSNP specification out of the box.
Definitions:
IdA = DSNP User Id of A (little-endian)
IdB = DSNP User Id of B (little-endian)
Algorithm:
- Both Alice and Bob generate an asymmetric key pair for use with X25519 ECIES.
Each uses the Replace User Data Operation to publish their generated public key in
keyAgreementPublicKeys
.
Libsodium | Algorithm |
---|---|
crypto_box_keypair( &a_public, &a_secret); crypto_box_keypair( &b_public, &b_secret); |
(Apublic, Asecret) ← KGF() (Bpublic, Bsecret) ← KGF() |
- When Alice wants to interact with Bob, she looks up Bob’s public key and performs an X25519 Elliptic-curve Diffie-Hellman key exchange operation using her secret key and Bob’s public key, generating a root shared secret.
Libsodium | Algorithm |
---|---|
crypto_box_beforenm( &root_shared_secret, b_public, a_secret); |
RootSharedSecretAB ← ECDH(Bpublic, Asecret) |
- Alice derives a context-specific subkey
CtxSharedSecretA→B
from the shared secretRootSharedSecretAB
as the master key, Bob’s DSNP User Id as the 64-bit key identifier, and the ASCII encoding of the PRId Context string ("PRIdCtx0"
for connections).
Libsodium | Algorithm |
---|---|
crypto_kdf_derive_from_key( ctx_shared_secret, 32, b_user_id, "PRIdCtx0", root_shared_secret); |
CtxSharedSecretA→B,C ← Blake2b256( key = RootSharedSecretAB, message = {}, salt = IdB || {0}, personal = "PRIdCtx0" || {0}) |
- Alice uses Bob’s DSNP User Id to form an 8-byte little-endian message.
Alice encrypts this message using XSalsa20 with the PRId key
CtxSharedSecretA→B,C
and a nonce of her own User Id (little-endian) followed by 16 zero bytes.
Libsodium | Algorithm |
---|---|
char nonce[24] = {0}; int i; for (i = 0; i < 8; i++) { nonce[i] = (user_id_a >> (i*8)) & 0xff; }
|
PRIdA→B,C ← XSalsa20( message = IdB, key = CtxSharedSecretA→B,C, nonce = Padded24BytesLE(IdA) ) |
- Alice adds the generated PRId to the relevant list of PRIds and publishes an updated copy via the Replace User Data Operation.
Similarly, Bob can calculate the same root shared secret RootSharedSecret
using Alicepublic
and Bobsecret
and derive the same PRIdA→B,C
in order to check if it is in Alice’s published PRIds.
Bob can also derive the PRId subkey for Alice’s DSNP User Id and encrypt Alice’s User Id, using his own as the nonce, to generate the Bob-to-Alice PRId (PRIdB→A,C
), and then publish it to his own list, if desired.
If Alice or Bob wants to prove to a third party that their PRIds are in each other’s PRId list, they can provide the third party with their own subkey CtxSharedSecretA→B,C
or CtxSharedSecretB→A,C
.
The third party can repeat the encryption step using Alice and Bob’s User Ids, and check that the output is present in the published set of PRIds. The root shared secret RootSharedSecret
(used as a master key in this algorithm) should not be divulged.
Test Vector
For the following inputs:
Input | Value |
---|---|
Asecret | 0xc9432ed5c0c5c24e8a4ff190619893918b4d1265a67d123895023fa7324b43e0 |
Apublic | 0x0fea2cafabdc83752be36fa5349640da2c828add0a290df13cd2d8173eb2496f |
Bsecret | 0xdc106e1371293ee9536956e1253f43f8941d4a5c4e40f15968d24b75512b6920 |
Bpublic | 0xd0d4eb21db1df63369c147e63b2573816dd4b3fe513e95bf87f7ed1835407e62 |
IdA | 42 |
IdB | 478 |
Context | PRIdCtx0 |
An implementation of the PRId generation algorithm should produce the following outputs:
Output | Value |
---|---|
PRIdA→B | 0xace4d2995b1a829c |
CtxSharedSecretA→B,C | 0x37cb1a870f0c1dce06f5116faf145ac2cf7a2f7d30136be4eea70c324932e6d2 |
PRIdB→A | 0x1a53b02a26503600 |
CtxSharedSecretB→A,C | 0x32c45c49fcfe12f9db60e74fa66416c5a05832c298814d82032a6783a4b1fca0 |
Public Key
Represents an encoding of a public key, one half of a cryptographic key pair.
Serialization
PublicKey object serialization MUST conform to the following Avro schema:
{
"namespace": "org.dsnp",
"name": "PublicKey",
"type": "record",
"fields": [
{
"name": "publicKey",
"type": "bytes",
"doc": "Multicodec public key"
}
]
}
Generation
publicKey
- MUST be a public key of an allowed key type for the associated User Data type, encoded in
multicodec
format
The byte encoding consists of a multicodec key identifier (as a varint) followed by the public key’s binary data in the codec’s described format.
Allowed Key Types
User Data Type | Allowed Algorithms (multicodec) | Purpose |
---|---|---|
keyAgreementPublicKeys | x25519-pub | A Curve25519 public key that can be used in key exchange protocols to generate a shared secret |
assertionMethodPublicKeys | ed25519-pub | A public key for the EdDSA signature scheme using SHA-512 and Curve25519 that can be used to verify cryptographic signatures |
ProfileResource
A profile-linked resource is represented by the ProfileResource object type.
Serialization
ProfileResource object serialization MUST conform to the following Avro schema:
{
"namespace": "org.dsnp",
"name": "ProfileResource",
"type": "record",
"doc": "Profile-linked resource",
"fields": [
{
"name": "type",
"type": "int",
"doc": "Type of resource"
},
{
"name": "contentAddress",
"type": "string",
"doc": "Content address for the resource"
}
]
}
Generation
type
- MUST be a supported DSNP profile-linked resource type.
Supported Profile-Linked Resource Types
Value | Description | Specification | Content Type | Maximum File Size | Content Address Type | DSNP Version Added |
---|---|---|---|---|---|---|
1 | Activity Content Profile with DSNP extensions | DSNP Profile | application/json | 256 Kb | Supported IPFS CID1 | 1.3 |
All other resource type values are reserved for future expansion.
1Supported IPFS CIDs must be CID version 1, using either sha2-256
or blake3
hashes with the raw
codec and the base32
serialization.
contentAddress
- MUST be a valid content address for the specified
type
(see table above) - MUST contain sufficient information for an application to perform content integrity validation, for example by comparing the
contentAddress
field to the address derived by recalculating the content address from the bytes of the retrieved resource
Batch Publications
A Batch Publication is an Apache Parquet file with a collection of Announcements.
Implementation Requirements
Discoverable
Implementations MUST have publicly discoverable Batch Publications.
Validity
Implementations MUST be able to validate Parquet file contents. Validity MUST be immutable.
Historical
Implementations MUST retain proof of existence of a Batch Publication.
Transparent Chain of Delegation
All Announcements in a Batch file MUST be able to be proven to be from or have a chain of delegation to the publisher of the Batch.
File Requirements
Batch files are stored and transferred in Apache Parquet format.
- Batch files MUST match the spec for a single Announcement Type.
- Batch files MUST have Bloom filters set in accordance with the Announcement Type Spec.
- Batch files MUST have NO MORE THAN 128*1024 rows.
Bloom Filter
- A Bloom filter MUST be a Split Block Bloom filter.
- The false-positive rate MUST be 0.001.
Calculation for filter bits is different and is nearly a factor of 10 lower than for a standard Bloom filter: 128*1024 rows with a 0.001 false-positive rate results in around 29,000 bits for a Split Block Bloom filter.
Bloom filters are ONLY added to some fields. See also Announcement Types.
Columns with Bloom Filters
Column | Primitive Type | Logical Type | Converted Type (deprecated) |
---|---|---|---|
contentHash | BYTE_ARRAY | STRING | UTF8 |
emoji | BYTE_ARRAY | STRING | UTF8 |
fromId | INT64 | INT(64, false) | UINT_64 |
inReplyTo | BYTE_ARRAY | STRING | UTF8 |
targetContentHash | BYTE_ARRAY | STRING | UTF8 |
Non-Normative
Design Requirements
Batch files need to be quickly and easily searchable. Minimal storage size and fast, simple querying are preferred over guarantees of no false positives or advanced data manipulation and column relationships. The files are parseable by client applications, web views, or browsers running pure JavaScript without needing to convert the format.
Applications need to know if a given Batch file has any information of interest without downloading the file first.
Why Parquet?
- Parquet is a column-oriented format. Since DSNP Batch Message data will have a very small column-to-row ratio compared to a typical web application database, it makes sense to prefer a column-oriented format.
- Parquet format has been field tested under extreme network conditions. It has broad support in cloud storage solutions, with libraries in multiple languages.
- Bloom filters are already supported in the Parquet specification, which allows for fast and accurate searching (with caveats for proper configuration).
- Amazon S3 support: We anticipate that some Batch Announcers (and possibly Archivists) will store Batch files on Amazon S3. Amazon Athena also supports storage in Parquet, and its API supports SQL-like queries.
- Parquet also allows references to the same column across files, which could enable multi-file querying in the future.
- Parquet supports compression formats such as Brötli, which is already a browser standard and offers a demonstrated improvement in compression speed and file size over older formats.
- Parquet files can be transferred directly to clients, which can parse the files in the app or browser. No conversion to a serialization format is necessary. This eliminates an entire class of bugs and makes both fetching and querying faster.
- Parquet uses schemas, which additionally reduce file size.
Rejected Alternatives
- Cassandra, RocksDB, CouchDB, MongoDB, and HBASE were rejected since DSNP data needs neither a database for storage nor the overhead of one. Each of these was designed for use cases ranging from somewhat different to drastically different from the DSNP network.
- JSON, BSON, and SQLite, while sometimes used for storage, are intended for serialization. They are schemaless, which results in redundant information and therefore a larger size than formats with schemas. They also don’t support Bloom filters; thus, indexing would be required, or new batches would need to be downloaded entirely. The exception is SQLite, which does support more advanced queries. However, SQLite was designed for in-memory storage.
Batch Validity and Order
Batch validity is immutable and is usually based (in part) on the validation of the delegation of authors listed inside the batch to the publisher. Due to the nature of distributed systems, it is possible that a race condition occurs such that a user’s delegation revocation presents before a Batch that contains a message from that user via the revoked delegate. While those individual messages should be considered invalid, a window of time for historical testing is suggested before considering the entire batch invalid. This is analogous to the idea of a confirmation time, but only applies to the past rather than the future.
Attribute Sets
Attribute sets enable structured data to be associated with DSNP users (self or others), DSNP content, or any other content (original or not) that has a public URL. Attribute sets have a cryptographically authenticated creator, and a subject (the entity being described), which may be the same. Attribute set data may appear in several modes: it can be published as a DSNP Announcement, attached to a user’s Profile document, or attached to social media content.
Data Model
The conceptual model for attribute sets includes three types of data:
- A schema that encodes rules for validating attribute set data. This MUST be in the form of a DSNP Verifiable Credential Schema (a Verifiable Credential that contains a JSON Schema document).
- The attribute set data itself. This MUST be serialized as a DSNP Verifiable Credential.
- A reference to the attribute set data, which may take one of several different forms depending on the desired usage pattern.
Attribute Set Type
An Attribute Set Type is an identifier that is used to group attribute sets that share the same data structure and semantic meaning. Attribute Set Types have a well known canonical name and (in most cases) a well defined schema, expressed using a Verifiable Credential Schema document.
Versioning
DSNP supports (but does not mandate) Attribute Set Types that can be described in different Verifiable Credential Schema documents over time in order to preserve backward and forward compatibility. This allows the exact details of the schema to evolve over time without compromising the discovery and verifiability of previously shared credentials.
This behavior is achieved by allowing a schema issuer to retain a common identifier across multiple versions of a schema, provided that the proof signatures are all issued from the same DSNP user (the schema author). This allows applications to perform reliable lookups against an Attribute Set Type, regardless of the specific version that may be used by an individual credential document. This is facilitated by the following naming scheme.
Canonical naming
Attribute Set Type canonical names are constructed as follows:
- MUST be in the format attributeSetTypeNamespace + “
$
” + attributeSetTypeName, where attributeSetTypeNamespace MUST be either a multihash content hash (encoded as a multibase string), the DSNP DID of the schema author (beginning with “did:dsnp:
”), or the empty string (for schemaless attribute set types). - attributeSetTypeName MUST match a declared type value in the Verifiable Credential document.
- If attributeSetTypeNamespace is a DID, it MUST match the issuer of the Verifiable Credential Schema document referenced from the credential document, and the schema document MUST include a proof that can be verified using the issuer’s public key.
- If attributeSetTypeNamespace is empty, the credential document MUST NOT reference a schema.
- If attributeSetTypeNamespace is a multibase string, it MUST match the multihash content hash of the schema file referenced from the credential document.
Examples:
- Schemaless:
$IsHuman
- Unsigned schema:
bciqoatksnf4szlitvzfpsdgzegmkmz74i4mpsmb76q7pro42c6fvpzq$BSC
- Signed schema:
did:dsnp:1234567890$TimeZone
Usage Patterns
Attribute Set Announcements
The Announcement model allows attribute sets to be consumed as events providing context or metadata to the social network.
Attribute Set Announcements allow DSNP Users to associate data or make assertions about their own account, other DSNP Users, content on DSNP, or even content external to DSNP.
Credential documents are anchored to the announcement by the url
and hash
fields.
Applications consuming these announcements can index and use the associated attribute sets to inform their user experience. For example, a fact-checking organization can publish Attribute Set Announcements to flag content it deems to be misinformation, and interested applications that trust the organization’s determinations can provide warning labels on social media posts. Similarly, an organization might attach metadata in the form of attribute sets to denote DSNP User Ids that are operated by government actors.
Attribute Set Announcements cannot be updated once published, but can be tombstoned. Applications MUST treat tombstoned Attribute Set Announcements as nonexistent.
Announcement Types
Attribute Set announcements are expressed using three announcement types, depending on the type of subject being described.
Announcement Type | Id | Subject identifier |
---|---|---|
User Attribute Set | 8 | DSNP DID |
DSNP Content Attribute Set | 9 | DSNP Content URI |
External Content Attribute Set | 10 | URL and hash |
Attestation Attachments
Attribute sets associated with and controlled by a DSNP User can be referenced as Attestation Attachments to a user’s Profile document, or to an Activity Content Note object that is referenced from a Broadcast or Reply announcement.
Profile-linked attestations are necessary in cases where an attestation is required in order for a consumer to verify chains of trust designated by schema controllers, such as an accreditation that gives the organization represented by the DSNP User Id the authority to issue credentials to others.
Applications enabling users to update their Profile document should take care to preserve any Profile-linked attestations, even if the meaning of these is opaque to the updating application.
Attestation attachments link to credentials that identify a DSNP User as their subject.
That is, the credentialSubject.id
field should match the user’s DSNP DID.
On-Demand Presentation
Holders of Verifiable Credentials may wish to present these only on demand rather than have a public record of the credential. Several emerging specifications are developing for this usage pattern and are outside the scope of this specification. Developers are encouraged to consider the Verifiable Presentation Request specification.
Public Key References
Both Verifiable Credential Schema documents and Verifiable Credential documents may include proof sections.
DSNP Users may control one or more key pairs for use in signing these documents and make these discoverable using the assertionMethod
User Data type.
A key announced in this fashion can be referenced within the proof using a DSNP DID with a key identifier, as in did:dsnp:123456#key1
.
A verifier MUST ensure that the DSNP User Id referenced this way (that is, the substring of the DID before the first #
character) is the same as the issuer field.
Following the principle of least privilege, the key pair used to issue credentials SHOULD be different from any control keys used to authenticate transactions.
Trust and Verification
Trust in an attribute set may be assigned based on a combination of its Attribute Set Type and issuer. It is left to each application that acts as a DSNP consumer to determine which attribute sets it will trust.
Trust MUST be accompanied by verification of the documents linked to an attribute set reference.
Summary of Verification Responsibilities
When verifying a credential document, a consumer MUST:
- Retrieve the Verifiable Credential document from the announcement or attachment link’s
url
field (or, for on-demand credentials, receive the credential document by some other mechanism). - Calculate and verify that the content hash of the credential document matches the value specified in the referring item (an Attribute Set Announcement’s
contentHash
field, for example). - Verify that the credential document is well formed (it should comply with the generic JSON schema for verifiable credential documents, and include a valid combination of fields).
- Verify that the credential’s expiration date (the
expirationDate
orvalidUntil
field), if present, has not passed. - If the subject of the credential document is not a URL beginning with “
dsnp://
”, verify thesubjectContentHash
by retrieving the subject URL and applying the indicated hash function. URLs beginning with “dsnp://
” do not need a hash check as they already include a self-reflective content hash. They should, however, be checked for existence. - Retrieve the schema document from the URL specified in the
credentialSchema.id
field. - For schema of type
JsonSchemaCredential
, validate the schema document against the generic JSON schema for Verifiable Credential Schema, and validate the credential document against the JSON schema found in the Verifiable Credential Schema document’scredentialSubject.jsonSchema
property. - For schema of type
JsonSchema
, validate that the schema document is a well formed JSON schema, then validate the credential document against the schema document. - Construct the
attributeSetType
for the credential document by following the rules for schemaless, unsigned, or signed schemas, and ensure it matches the declared type in the announcement or link object. Note that this MAY require verification of the issuer proof of the Verifiable Credential Schema, and potentially a chain of trust (see below). - Verify the issuer proof on the credential document, if present.
Verifiers should take care that transient errors (for example, a URL being unreachable due to temporary network issues) do not lead to long-term caching of false negatives.
Trust chains
Applications are free to make their own trust decisions, and display or incorporate Verifiable Credentials based on their issuer
.
However, in many real world scenarios, the originator of a particular type of credential may authorize agents to issue the credential, based on any number of approval or certification processes external to DSNP.
When utilizing Verifiable Credential Schema documents, the originator of a DSNP Attribute Set Type can encode its requirements for trusted agents within a DSNP extension field.
To do so, DSNP uses an optional, protocol-specific trust
key within the dsnp
key under the credentialSubject
section of the schema credential.
The trust
key can contain one or both subkeys oneOf
or allOf
, which in turn contain a list of Attribute Set Types, or further embedded oneOf
or allOf
groupings.
These indicate to the consumer that the author of the credential schema requires that a credential of the defined type be trusted only if its issuer can be shown to possess Verifiable Credentials of the indicated type.
In the case of oneOf
, the verifier should check that the issuer is the subject of a Verifiable Credential conforming to at least one of the given Attribute Set Types.
In the case of allOf
, the verifier should check that the issuer is the subject of Verifiable Credentials for every given Attribute Set Type.
For example, the Worldwide Whale Foundation, identified by DSNP User Id 123456, may authorize its agents, including Acme Ocean Certification Lab, to issue credentials of type did:dsnp:123456$CertifiedWhaleBiologist
to individuals that meet the certification standard that the Foundation defines.
The Worldwide Whale Foundation defines a second attribute set type, did:dsnp:123456$AuthorizedWhaleBiologistCertifier
by publishing a Verifiable Schema Credential for that type, and then issues a Verifiable Credential of that type (an accreditation) to Acme Ocean Certification Lab, which is then published online.
Worldwide Whale Foundation then includes the did:dsnp:123456$AuthorizedWhaleBiologistCertifier
Attribute Set Type in the credentialSubject.dsnp.trust.allOf
array of the CertifiedWhaleBiologist
Verifiable Credential Schema (in this example, using oneOf
would be equivalent), generates and attaches the signature proof, and publishes the schema document.
When Acme Ocean Certification Lab wants to issue did:dsnp:123456$CertifiedWhaleBiologist
credentials to users, it should include the URL, attribute set type, and content hash of its accreditation credential in the issuer.authority
array of each credential issued.
A consumer of the credential can retrieve and verify that Acme Ocean Certification Lab is an accredited issuer.
Displaying credentials
The DSNP schema credential document MAY contain an additional display
key within the dsnp
key under credentialSubject
.
This allows the authors of attribute set types to recommend how a credential should be displayed within a user interface.
To allow for localizable text, a map of content language codes (following BCP-47
, so using the same form as the HTTP Content-Language
header) to display text can be used under the subkey label
.
Example:
...
"credentialSubject": {
"type": "JsonSchema",
"jsonSchema": {
...
},
"dsnp": {
"display": {
"label": {
"en-US": "Whale Biologist",
"es-ES": "Biólogo de Ballenas"
}
}
}
}
DSNP applications should treat the display
section as a recommendation but not a mandate, and are free to indicate the presence or absence of a Verifiable Credential in other forms.
Announcements Overview
Announcements are content or references to content that communicate new user activity to the rest of the network. Announcements are associated with an Identifier that can be validated as the creator of the Announcement. Depending on the implementation, Announcements may be published directly to the network, included in Batch Publication Files, or some combination of those two.
Announcement Types
Each Announcement has an enumerated type for use when separating out a stream of Announcements.
Value | Name | Description | DSNP Content URI | Tombstone Allowed |
---|---|---|---|---|
0 | Tombstone | an invalidation of previously announced content | no | no |
1 | ||||
2 | Broadcast | a public post | YES | YES |
3 | Reply | a public response to a Broadcast | YES | YES |
4 | Reaction | a public visual reply to a Broadcast | no | no |
5 | ||||
6 | Update | an update to content | YES | no |
7 | ||||
8 | User Attribute Set | an attribute set for a DSNP User | YES | YES |
9 | DSNP Content Attribute Set | an attribute set for a DSNP content item | YES | YES |
10 | External Content Attribute Set | an attribute set for a non-DSNP content item | YES | YES |
a Since DSNP version 1.2, social graph changes use User Data operations as described in the Graph section.
b Since DSNP version 1.3, public keys use User Data operations.
c Since DSNP version 1.3, profile changes use User Data operations.
Announcement Validation
There is no guarantee that, at time of creation, a given Announcement will be from the fromId
claimed in the Announcement.
The reader MUST perform a validation of the Announcement at read time to ensure authenticity.
Implementations MUST provide a way to validate that the identifier associated with a given Announcement is authentic.
External Content URLs and Hashes
Where Announcements refer to external documents (such as Activity Content documents), these are referenced by both a URL and a DSNP Content Hash. The content hash MUST be generated by applying a Supported Hashing Algorithm to the full, unaltered contents of the document.
The URL associated with a content hash should be construed as a hint to initially locate a document matching the content hash, but is in no way meant to be the only way to locate the indicated document. Over time, a URL may cease to reference the specified document, or might have its contents altered; therefore, the content hash should be considered the authoritative value and the URL only one of many possible ways of locating a document. For example, services may cache documents or retrieve them from a content-addressed file system by applying the content hash (or a value derived from the content hash, such as a CID). When readers retrieve content referenced in an Announcement, they can validate the authenticity of the content, regardless of where it is hosted, by regenerating the hash output and comparing it with the content hash recorded in the Announcement.
Duplicate Handling
Due to the nature of asynchronous communication, duplicate Announcements may occur. In the case of duplicates, the first Announcement should be considered the ONLY valid Announcement. Additional duplicate Announcements MUST be rejected or ignored.
Ordering Announcements
- Order Batch Publications by implementation order.
- Order Announcements in a Batch Publication File by row appearance order.
Reverting an Announcement
Announcements may not be deleted, but may be marked invalid by using a Tombstone Announcement, or updated by using an Update Announcement. For example, if a user creates a Reaction Announcement, they may remove that reaction by creating a Tombstone Announcement.
Related Operations
Non-Normative
Duplicate Announcements
Due to the distributed nature of DSNP, duplicate Announcements are possible from time to time. These should be discarded and ignored.
Replay Attacks
Implementations typically restrict replay attacks by testing that the chain transaction sender is authorized (often via delegation) to publish an Announcement.
Announcement Ordering and Activity Content Published Timestamp
Activity Content has a published field that contains a user-generated timestamp. User-generated timestamps cannot be validated, but may be used to indicate ordering other than the network order for Announcements (which are not time dependent.)
Announcement Reference Ordering
Some Announcements contain references to other Announcements via the inReplyTo
field.
Due to the distributed nature of DSNP, the canonical order can have an Announcement that refers to another Announcement appearing later in the network order.
For display purposes, these messages should be considered to have occurred after the reference.
DSNP v1.0 Announcement Signatures
In DSNP v1.0, Announcements had individual signatures, producing Batch Publications that were generic and disconnected from the user. Announcements could be submitted to the chain via anyone—not just delegates or users.
In DSNP v1.1, Announcement signatures were removed in favor of the implementation being responsible for the connection between the on-chain signature and the user. Implementations require that the transaction that produces a Batch be performed by the user or delegate directly. This creates batches that are delegate specific, but allows for faster testing of the validity of individual Announcements in a Batch.
For more information see DIP-145.
Tombstone Announcement
A Tombstone Announcement is a way to note that a previously announced content is invalid and the related Announcement should be considered reverted. It is NOT possible to revert a tombstone.
Fields
Field | Description | Data Type | Serialization | Parquet Type | Bloom Filter |
---|---|---|---|---|---|
announcementType | Announcement Type Enum (0 ) | enum | decimal | INT32 | no |
fromId | Id of the user creating the Announcement and the Tombstoned Announcement | 64-bit unsigned integer | decimal | UINT_64 | YES |
targetAnnouncementType | target tombstoned Announcement type | enum | decimal | INT32 | no |
targetContentHash | target contentHash of the original Announcement to tombstone | UTF-8 | base32 multibase | UTF8 | YES |
Field Requirements
announcementType
- MUST be fixed to
0
fromId
- MUST be a DSNP User Id
- MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation
targetAnnouncementType
- MUST be the Announcement Type of the target Announcement
- MUST ONLY be a Tombstone allowed Announcement Type
Tombstone Allowed Announcement Types
Value | Name |
---|---|
2 | Broadcast |
3 | Reply |
8 | User Attribute Set |
9 | DSNP Content Attribute Set |
10 | External Content Attribute Set |
targetContentHash
- MUST be the
contentHash
of a previous Announcement of an Allowed Announcement Type with the samefromId
as the Tombstone Announcement
Broadcast Announcement
A Broadcast Announcement is a way to send a public message to everyone.
Fields
Field | Description | Data Type | Serialization | Parquet Type | Bloom Filter |
---|---|---|---|---|---|
announcementType | Announcement Type Enum (2 ) | enum | decimal | INT32 | no |
contentHash | DSNP Content Hash of content | UTF-8 | base32 multibase | UTF8 | YES |
fromId | id of the user creating the Announcement | 64-bit unsigned integer | decimal | UINT_64 | YES |
url | content URL | UTF-8 | UTF-8 | UTF8 | no |
Field Requirements
announcementType
- MUST be fixed to
2
contentHash
- MUST be a valid DSNP Content Hash
fromId
- MUST be a DSNP User Id
- MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation
url
- MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
- Resource MUST be one of the supported Activity Content Types
- MUST use one of the supported URL Schemes
Supported URL Schemes
Scheme | Description | Reference | DSNP Version Added |
---|---|---|---|
HTTPS | Hypertext Transfer Protocol Secure | RFC2818 | 1.0 |
Reply Announcement
A Reply Announcement is the same as a Broadcast Announcement,
but includes an inReplyTo
field for noting it as a reply to a given DSNP Content URI.
Fields
Field | Description | Data Type | Serialization | Parquet Type | Bloom Filter |
---|---|---|---|---|---|
announcementType | Announcement Type Enum (3 ) | enum | decimal | INT32 | no |
contentHash | DSNP Content Hash of content | UTF-8 | base32 multibase | UTF8 | YES |
fromId | id of the user creating the Announcement | 64-bit unsigned integer | decimal | UINT_64 | YES |
inReplyTo | Target DSNP Content URI | UTF-8 | UTF-8 | UTF8 | YES |
url | content URL | UTF-8 | UTF-8 | UTF8 | no |
Field Requirements
announcementType
- MUST be fixed to
3
contentHash
- MUST be a valid DSNP Content Hash
fromId
- MUST be a DSNP User Id
- MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation
inReplyTo
- MUST be a DSNP Content URI
url
- MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
- Resource MUST be one of the supported Activity Content Types
- MUST use one of the supported URL Schemes
Supported URL Schemes
Scheme | Description | Reference | DSNP Version Added |
---|---|---|---|
HTTPS | Hypertext Transfer Protocol Secure | RFC2818 | 1.0 |
Reaction Announcement
A Reaction Announcement is for publishing emoji reactions to anything with a DSNP Content URI.
Fields
Field | Description | Data Type | Serialization | Parquet Type | Bloom Filter |
---|---|---|---|---|---|
announcementType | Announcement Type Enum (4 ) | enum | decimal | INT32 | no |
emoji | the encoded reaction | UTF-8 | UTF-8 | UTF8 | YES |
apply | how to apply the reaction | 8-bit unsigned integer | decimal | UINT_8 | no |
fromId | id of the user creating the relationship | 64-bit unsigned integer | decimal | UINT_64 | YES |
inReplyTo | Target DSNP Content URI | UTF-8 | UTF-8 | UTF8 | YES |
Field Requirements
announcementType
- MUST be fixed to
4
emoji
- Emoji fields must not be empty
- Emoji fields must consist only of Unicode points from
U+2000
toU+2BFF
, fromU+E000
toU+FFFF
, or fromU+1F000
toU+10FFFF
Examples
All of the following should be considered valid emojis:
"😀", "🤌🏼", "👩🏻🎤", "🧑🏿🏫", "🏳️🌈", "🏳️⚧️", "⚛︎", "🃑", "♻︎"
None of the following should be considered valid:
"F", ":custom-emoji:", "<custom-emoji>", "ᚱ", "ᘐ", "״"
apply
- MUST be an UINT_8
- Indicates whether the emoji should be applied and if so, at what “strength”.
Potential uses:
- a single reaction
- ratings
- a range of responses, e.g. “strongly disagree” –> “strongly agree” = 1 –> 5 stars.
- recommendation engines
Apply Enums
Value | Name | Description |
---|---|---|
0 | retract | Remove the referenced emoji |
n | apply | Apply the referenced emoji N times |
fromId
- MUST be a DSNP User Id
- MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation
inReplyTo
- MUST be a DSNP Content URI
Non-Normative
Likes
Generic “likes” should default to the "❤️"
or Unicode U+FE0F
as the emoji in the reaction.
Update Announcement
An Update Announcement is a way to note intent to update previously announced content. If the original Broadcast/Reply is Tombstoned, subsequent Updates should be ignored.
Fields
Field | Description | Data Type | Serialization | Parquet Type | Bloom Filter |
---|---|---|---|---|---|
announcementType | Announcement Type Enum (6 ) | enum | decimal | INT32 | no |
fromId | id of the user creating the announcement | 64 bit unsigned integer | decimal | UINT_64 | YES |
contentHash | DSNP Content Hash of content | UTF-8 | base32 multibase | UTF8 | YES |
url | updated content URL | UTF-8 | UTF-8 | UTF8 | no |
targetAnnouncementType | target updated Announcement type | enum | decimal | INT32 | no |
targetContentHash | target contentHash of the original Announcement to update | UTF-8 | base32 multibase | UTF8 | YES |
Field Requirements
announcementType
- MUST be fixed to
6
fromId
- MUST be a DSNP User Id
contentHash
- MUST be a valid DSNP Content Hash
url
- MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
- Resource MUST be one of the supported Activity Content Types
- MUST use one of the supported URL Schemes
targetAnnouncementType
- MUST be the Announcement Type of the target Announcement
- MUST ONLY be an Update allowed Announcement Type
Update Allowed Announcement Types
targetContentHash
- MUST be the
contentHash
of a previous Announcement of an Allowed Announcement Type with the samefromId
as the Update Announcement
User Attribute Set Announcement
A User Attribute Set Announcement is a way to create an authenticated (and, optionally, attested) attribute set for a DSNP User.
Fields
Field | Description | Data Type | Serialization | Parquet Type | Bloom Filter |
---|---|---|---|---|---|
announcementType | Announcement Type Enum (8 ) | enum | decimal | INT32 | no |
fromId | Id of the user creating the announcement | 64 bit unsigned integer | decimal | UINT_64 | YES |
subject | DSNP User Id of the attribute set subject | 64 bit unsigned integer | decimal | UINT_64 | YES |
url | URL for the Verifiable Credential document | UTF-8 | UTF-8 | UTF8 | no |
contentHash | DSNP Content Hash of content | UTF-8 | base32 multibase | UTF8 | YES |
attributeSetType | Canonical name of Attribute Set Type | UTF-8 | UTF-8 | UTF8 | YES |
issuer | URI of issuer | UTF-8 | UTF-8 | UTF8 | YES |
Field Requirements
announcementType
- MUST be fixed to
8
fromId
- MUST be a DSNP User Id
- MUST have authorized the creation of the announcement, either directly or via a transparent chain of delegation
subject
- MUST be a DSNP User Id
url
- MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
- Resource MUST be a Verifiable Credential Document
- MUST use one of the supported URL Schemes
Supported URL Schemes
Scheme | Description | Reference | DSNP Version Added |
---|---|---|---|
HTTPS | Hypertext Transfer Protocol Secure | RFC2818 | 1.3 |
contentHash
- MUST be a valid DSNP Content Hash
attributeSetType
- MUST be a valid Attribute Set Type canonical name
issuer
- MUST be a valid URI
- MUST match the
issuer
value from the credential document retrieved fromurl
DSNP Content Attribute Set Announcement
A DSNP Content Attribute Set Announcement is a way to create an authenticated (and, optionally, attested) attribute set for a DSNP content item.
Fields
Field | Description | Data Type | Serialization | Parquet Type | Bloom Filter |
---|---|---|---|---|---|
announcementType | Announcement Type Enum (9 ) | enum | decimal | INT32 | no |
fromId | Id of the user creating the announcement | 64 bit unsigned integer | decimal | UINT_64 | YES |
subject | DSNP Content URI of the attribute set subject | DSNP Content URI | UTF-8 | UTF8 | YES |
url | URL for the Verifiable Credential document | UTF-8 | UTF-8 | UTF8 | no |
contentHash | DSNP Content Hash of content | UTF-8 | base32 multibase | UTF8 | YES |
attributeSetType | Canonical name of Attribute Set Type | UTF-8 | UTF-8 | UTF8 | YES |
issuer | URI of issuer | UTF-8 | UTF-8 | UTF8 | YES |
Field Requirements
announcementType
- MUST be fixed to
9
fromId
- MUST be a DSNP User Id
- MUST have authorized the creation of the announcement, either directly or via a transparent chain of delegation
subject
- MUST be a DSNP Content URI
url
- MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
- Resource MUST be a Verifiable Credential Document
- MUST use one of the supported URL Schemes
Supported URL Schemes
Scheme | Description | Reference | DSNP Version Added |
---|---|---|---|
HTTPS | Hypertext Transfer Protocol Secure | RFC2818 | 1.3 |
contentHash
- MUST be a valid DSNP Content Hash
attributeSetType
- MUST be a valid Attribute Set Type canonical name
issuer
- MUST be a valid URI
- MUST match the
issuer
value from the credential document retrieved fromurl
External Content Attribute Set Announcement
An External Content Attribute Set Announcement is a way to create an authenticated (and, optionally, attested) attribute set for content external to the DSNP system.
Fields
Field | Description | Data Type | Serialization | Parquet Type | Bloom Filter |
---|---|---|---|---|---|
announcementType | Announcement Type Enum (10 ) | enum | decimal | INT32 | no |
fromId | Id of the user creating the announcement | 64 bit unsigned integer | decimal | UINT_64 | YES |
subject | URL of the external content | URL | UTF-8 | UTF8 | no |
subjectContentHash | DSNP Content Hash of content at subject | UTF-8 | base32 multibase | UTF8 | YES |
url | URL for the Verifiable Credential document | UTF-8 | UTF-8 | UTF8 | YES |
contentHash | DSNP Content Hash of content at url | UTF-8 | base32 multibase | UTF8 | YES |
attributeSetType | Canonical name of Attribute Set Type | UTF-8 | UTF-8 | UTF8 | YES |
issuer | URI of issuer | UTF-8 | UTF-8 | UTF8 | YES |
Field Requirements
announcementType
- MUST be fixed to
10
fromId
- MUST be a DSNP User Id
- MUST have authorized the creation of the announcement, either directly or via a transparent chain of delegation
subject
Optional. If present,
- MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
- MUST use one of the supported URL Schemes
subjectContentHash
- MUST be a valid DSNP Content Hash
url
- MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
- Resource MUST be a Verifiable Credential Document
- MUST use one of the supported URL Schemes
contentHash
- MUST be a valid DSNP Content Hash
attributeSetType
- MUST be a valid Attribute Set Type canonical name
issuer
- MUST be a valid URI
- MUST match the
issuer
value from the credential document retrieved fromurl
Supported URL Schemes
Scheme | Description | Reference | DSNP Version Added |
---|---|---|---|
HTTPS | Hypertext Transfer Protocol Secure | RFC2818 | 1.3 |
Migrated Announcements
Graph Change Announcement
*Since DSNP version 1.2, social graph changes use [User Data](../UserData.md) operations as described in the [Graph](../Graph.md) section.*A Graph Change Announcement is for publishing relationship state changes for a user.
Fields
Field | Description | Data Type | Serialization | Parquet Type | Bloom Filter |
---|---|---|---|---|---|
announcementType | Announcement Type Enum (1 ) | enum | decimal | INT32 | no |
changeType | Type of relationship change | enum | decimal | INT32 | no |
fromId | Id of the user creating the relationship | 64-bit unsigned integer | decimal | UINT_64 | YES |
objectId | Id of the target of the relationship | 64-bit unsigned integer | decimal | UINT_64 | YES |
Field Requirements
announcementType
- MUST be fixed to
1
changeType
- MUST be one of the Change Type Enum
Change Type Enum
Different change types have different meanings.
Value | Name | Description |
---|---|---|
0 | Unfollow | Remove a Follow relationship |
1 | Follow | Create a Follow relationship |
fromId
- MUST be a DSNP User Id
- MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation
objectId
- MUST be a DSNP User Id
Non-Normative
Graph Retrieval, Ordering and Reading
Each Graph Change event represents a state transition for the graph. The state of the graph at any time is given by taking the state of the graph at a previous time and applying all Graph Change events not previously applied in the order specified by Announcement Ordering.
Once those Graph Change events are retrieved, they can be ordered to reflect the current graph state (i.e. Charlie has followed Bob, then he unfollowed him, and then followed him again. The graph state reflects that Charlie is following Bob.)
To retrieve the graph, do the following:
- Retrieve the events with announcementType matching the enum for Graph Change.
- Filter the events to a particular DSNP User Id to retrieve information about the respective graph.
- Order the retrieved data by Announcement Ordering.
Public Key Announcement
*Since DSNP version 1.3, public keys use [User Data](../UserData.md) operations.*A Public Key Announcement is a way to note a new cryptographic key that can be used in DSNP to secure and verify the authenticity of communications.
The most recently published key (if one exists) for a given key type should be treated as the active key of that key type.
Fields
Field | Description | Data Type | Serialization | Parquet Type | Bloom Filter |
---|---|---|---|---|---|
announcementType | Announcement Type Enum (7 ) | enum | decimal | INT32 | no |
fromId | id of the user creating the Announcement | 64 bit unsigned integer | decimal | UINT_64 | YES |
keyType | Key Type Enum | enum | decimal | INT32 | YES |
keyId | user-assigned identifier | 64 bit unsigned integer | decimal | UINT_64 | no |
publicKey | public key in multikey format | variable length byte array | UTF-8 | BYTE_ARRAY | no |
Field Requirements
announcementType
- MUST be fixed to
7
fromId
- MUST be a DSNP User Id
keyType
- MUST be an allowed Key Type value
Allowed Key Types
Value | Name | Allowed Algorithms (multicodec) | Purpose |
---|---|---|---|
1 | keyAgreement | x25519-pub | A Curve25519 public key that can be used in key exchange protocols to generate a shared secret |
2 | assertionMethod | ed25519-pub | A public key that can be used for verifying digital signature proofs of verifiable credentials |
keyId
- A user-assigned 64-bit identifier for the key.
The user may assign a new keyId
each time they announce a new key of a given keyType
.
A keyId
value is useful when invoking certain DSNP Operations in order to indicate which key was used to encrypt data.
It may also provide a hint to the user if they ever need to regenerate their private key (for example, many key derivation functions enable the use of a subkey identifier to deterministically create a subkey from a root key).
publicKey
- MUST be a public key of an allowed algorithm for
keyType
, encoded inmultikey
format
The multikey
encoding of public keys is described in the draft did:key Method specification.
The byte encoding consists of a multicodec key identifier (as a varint) followed by the public key’s binary data in the codec’s described format.
If serializing the multicodec
value as a string, base58btc
encoding is recommended.
For example, the string z6LStiZsmxiK4odS4Sb6JmdRFuJ6e1SYP157gtiCyJKfrYha
decodes as a Base58 string using the x25519-pub
multicodec value with a 32-byte raw key of 0xfd3384e132ad02a56c78f45547ee40038dc79002b90d29ed90e08eee762ae715
.
Profile Announcement
A Profile Announcement is a constrained version of a Broadcast Announcement. The reference content MUST be of profile type.
Fields
Field | Description | Data Type | Serialization | Parquet Type | Bloom Filter |
---|---|---|---|---|---|
announcementType | Announcement Type Enum (5 ) | enum | decimal | INT32 | no |
contentHash | multihash-encoded hash of content stored at URL | variable length byte array | hexadecimal | BYTE_ARRAY | YES |
fromId | id of the user creating the Announcement | 64-bit unsigned integer | decimal | UINT_64 | YES |
url | profile content URL | UTF-8 | UTF-8 | UTF8 | no |
Field Requirements
announcementType
- MUST be fixed to
5
contentHash
- MUST be a DSNP Content Hash
fromId
- MUST be a DSNP User Id
- MUST have authorized the creation of the Announcement, either directly or via a transparent chain of delegation
url
- MUST NOT refer to localhost or any reserved IP addresses as defined in RFC6890
- Resource MUST be a valid Profile Activity Content Type
- MUST use one of the supported URL Schemes
Supported URL Schemes
Scheme | Description | Reference | DSNP Version Added |
---|---|---|---|
HTTPS | Hypertext Transfer Protocol Secure | RFC2818 | 1.0 |
Non-Normative
Most Recent Profile
When displaying a DSNP user’s profile, the most recent profile should be considered the complete and correct version.
Previous Profile Announcements from the same fromId
may be disregarded.
Operations
DSNP implementations perform well-defined DSNP Operations and generate DSNP State Change Records.
Control Keys and Proofs
Each invocation of a DSNP Operation MUST have verifiable approval of the acting principal(s) via a Control Key Ownership Proof. The precise data type and representation of both the Control Key and the Control Key Ownership Proof MUST be defined by each DSNP implementation. For example, an implementation might use the public key of an asymmetric key pair as a control key, and provide a proof for each operation by producing a cryptographic signature of the user’s DSNP Identifier and some nonce value.
Where operations are listed as using control keys or ownership proofs as input parameters, this indicates that these keys or proofs should be provided in addition to those needed for invocation authentication.
Transaction Identifiers
Each invocation of a DSNP Operation should be associated with a Transaction Identifier. Transaction Identifiers are used to associate Operation invocations with asynchronously emitted State Change Records. It MUST be possible to associate a DSNP State Change Record with a Transaction Identifier from a particular DSNP Operation invocation. Transaction Identifiers MUST be unique within an implementation. Transaction Identifiers MUST be serializable as a string.
Failure Handling
Compliant implementations may respond to error conditions either synchronously, as a response to the invocation request, or asynchronously, by emitting a Failure Record.
List of Operations
Operation | Optional? | Principal(s) | Inputs | State Change Record or Output |
---|---|---|---|---|
Create Identifier | no | None | Control Key, Control Key Ownership Proof | Identifier Creation Record |
Retire Identifier | no | User | None | Identifier Retirement Record |
Define Delegation | no | User AND Delegate | User’s Identifier, Delegate’s Identifier, Set of Allowed Announcement Types, Set of Allowed User Data Types | Delegation Definition Record |
Revoke Delegation | no | User OR Delegate | User’s Identifier, Delegate’s Identifier | Delegation Revocation Record |
Add Control Key | YES | User | Key, Key Ownership Proof | Control Key Addition Record |
Remove Control Key | YES | User | Key | Control Key Removal Record |
Publish Announcement | no* | User OR Delegate | Announcement | Announcement Published Record |
Publish Batch | no* | User OR Delegate | Announcement Type, Batch Publication URL, Batch Publication Content Hash | Batch Published Record |
Get User Data | no | Any | User’s Identifier, Set of Requested User Data Types | Map of User Data Types to Data Chunks with optional key identifiers of encryption keys for each chunk |
Replace User Data | no | User OR Delegate | User’s Identifier, Key Identifier, Map of User Data Types to Data Chunks | User Data Replaced Record |
* For each Announcement Type, an implementation may support one or both of these operations. Implementations MUST document which of the operations is available for each Announcement Type.
State Change Records
State Change Records constitute the observable output of a DSNP system. Implementations MUST specify how applications can translate implementation-specific output into State Change Records.
Each Record is associated with a Transaction Identifier, which allows Operations to be asynchronously associated with their results.
Records consists of one or more fields.
Record Type | Field | Field Data Type |
---|---|---|
Identifier Creation Record | Control Key | Implementation dependent |
Control Key Ownership Proof | Implementation dependent | |
Identifier Retirement Record | User's Identifier | DSNP User Id |
Delegation Definition Record | User Id | DSNP User Id |
Delegate Id | DSNP User Id | |
Allowed Announcement Types | List of enum values | |
Allowed User Data Types | List of enum values | |
Delegation Revocation Record | User Id | DSNP User Id |
Delegate Id | DSNP User Id | |
Control Key Addition Record | User Id | DSNP User Id |
Key | Implementation dependent | |
Key Ownership Proof | Implementation dependent | |
Control Key Removal Record | User Id | DSNP User Id |
Key | Implementation dependent | |
Announcement Published Record | Announcement | One of the types described in Announcements |
Batch Published Record | From Id | DSNP User Id |
Announcement Type | Enum value | |
URL | String | |
Content Hash | DSNP Content Hash | |
Failure Record | Message | String |
User Data Replaced Record | User Id | DSNP User Id |
User Data Types | Set of User Data Types |
Serializations
Serialization is how the value should be stringified for signing and for transfer between systems. Most serializations use outside standards, but some requiring additional clarifications are provided here.
decimal
Used to represent integers. Strings are used to avoid issues with different implementations of numbers.
- MUST use 0-9 representation
- MUST NOT have spaces or separators
- MUST be a string
Invalid | Why | Valid |
---|---|---|
"0x123" | Must be decimal | "291" |
291 | Must be a string | "291" |
"291n" | BigInt(291) serialization appends an n | "291" |
base32 multibase
Used to represent bytes.
A base32 multibase string is self-identifying and always begins with the b
character.
The Multibase Table describes this encoding as “RFC4648 case-insensitive - no padding”.
- MUST use RFC4648 §6 alphabet
abcdefghijklmnopqrstuvwxyz234567
- MUST be lowercase
- MUST be prefixed with
b
- MUST NOT have spaces or separators
- MUST NOT end with or contain padding characters (
=
)
Invalid | Why | Valid |
---|---|---|
BDYQDUA4T | Must user lowercase | bdyqdua4t |
dyqdua4t | Missing b prefix | bdyqdua4t |
b3og3k0sj | Wrong alphabet (base32hex was used) | bdyqdua4t |
bdyqdua4t= | Must not have padding characters | bdyqdua4t |
DSNP Systems
- Frequency (Official)
DSNP Over Frequency
Language and Framework
Frequency is a Polkadot Parachain written in Rust, using the Substrate framework.
- Frequency Website
- Frequency GitHub
- Frequency Documentation
- Frequency Rust Documentation
- Helpful Links
DSNP Over Frequency Schemas
Official schemas may be found in GitHub.
Name | Schema Name | Schema Id Mainnet | Schema Id Testnet (Paseo) |
---|---|---|---|
Tombstone | dsnp.tombstone | 16 | 16 |
Broadcast | dsnp.broadcast | 17 | 17 |
Reply | dsnp.reply | 18 | 18 |
Reaction | dsnp.reaction | 4 | 4 |
Update | dsnp.update | 19 | 19 |
Key Agreement Public Keys | dsnp.public-key-key-agreement | 7 | 7 |
Public Follows | dsnp.public-follows | 8 | 8 |
Private Follows | dsnp.private-follows | 9 | 9 |
Private Connections | dsnp.private-connections | 10 | 10 |
Assertion Method Public Keys | dsnp.public-key-assertion-method | 14 | 14 |
Profile Resources | dsnp.profile-resources | 15 | 15 |
User Attribute Set | dsnp.user-attribute-set | 20 | 20 |
DSNP Content Attribute Set | dsnp.dsnp-content-attribute-set | 12 | 12 |
External Content Attribute Set | dsnp.ext-content-attribute-set | 13 | 13 |
Last Update Date | Frequency Release | DSNP Version |
---|---|---|
2024-09-20 | 1.13.0+ | 1.3.0 |
Frequency Identity
Identity
- Name: Message Source Account or MSA
- Docs:
MSA Pallet
- Representation:
The following data that constitute a Message Source Account are stored in the MSA pallet:
- The DSNP User Id associated with this MSA
- Delegation relationships to Providers
- Schema permissions granted to Providers
- MSA user state (Profile, Graph, etc…)
- Control keys
DSNP User Identifier
- Name: Message Source Account Identifier, or MSA Id.
- Data Type:
uint64
- Docs:
MessageSourceId
- Mapping: The MSA Id is able to be used directly as the DSNP User Id
- Description: At least one public key MUST be associated with an MSA Id for it to be considered active.
Control Keys
- Name: Referred to as:
public_key
,provider_key
, ordelegator_key
- Data Type:
AccountId
, Schnorrkel/Ristretto X25519 (“sr25519”) derived cryptographic public key - Docs:
AccountId
- Description: See Cryptography on Polkadot and Polkadot Protocol Specification. A public key CANNOT be associated with more than one MSA at a time.
Delegation
- Name:
delegation
- Docs:
Delegation
- Representation:
The following data storage relates necessary information for retrieving and validating delegations:
- Provider registry
- Delegations
- Schema permissions granted to Providers
User
- Name: Delegator
- Representation: MSA Id
Delegate
- Name: Provider
- Representation: MSA Id
- Description:
A Provider MUST already have an MSA Id (via
msa::create()
) and be approved as a Provider (viamsa::propose_to_be_provider()
).
Related Operations
- Create Identifier
- Retire Identifier
- Define Delegation
- Revoke Delegation
- Add Control Key
- Remove Control Key
Announcement Publishing
On Frequency, Announcements are mapped to Schemas which in turn publish Frequency Messages. Frequency Messages are either individual Announcements from a particular user, or a Batch Publication with a multitude of possible users.
Enum | Announcement | Type | Deployed Schema Id | Frequency Model Type | Frequency Payload Location |
---|---|---|---|---|---|
0 | Tombstone | Batched | Mainnet: 16 Testnet (Paseo): 16 | ||
Parquet | |||||
IPFS | |||||
2 | Broadcast | Batched | Mainnet: 17 Testnet (Paseo): 17 | ||
Parquet | |||||
IPFS | |||||
3 | Reply | Batched | Mainnet: 18 Testnet (Paseo): 18 | ||
Parquet | |||||
IPFS | |||||
4 | Reaction | Batched | Mainnet: 4 Testnet (Paseo): 4 | ||
Parquet | |||||
IPFS | |||||
6 | Update | Batched | Mainnet: 19 Testnet (Paseo): 19 | ||
Parquet | |||||
IPFS | |||||
8 | User Attribute Set | Batched | Mainnet: 20 Testnet (Paseo): 20 | ||
Parquet | |||||
IPFS | |||||
9 | DSNP Content Attribute Set | Batched | Mainnet: 12 Testnet (Paseo): 12 | ||
Parquet | |||||
IPFS | |||||
10 | External Content Attribute Set | Batched | Mainnet: 13 Testnet (Paseo): 13 | ||
Parquet | |||||
IPFS |
Source code for each schema is located in the LibertyDSNP/schemas repository.
Batch Publications
Frequency uses DSNP Batch Publications for some types of Announcements. Parquet files are stored on IPFS, but are discovered through Frequency Messages.
DSNP Batch Publications MUST be validated upon fetching to ensure data and permission integrity.
Announcement Validation
DSNP Announcements are validated differently depending on the type of Announcement. Non-batched Announcements are on chain, are validated at write time, and do not need to be re-validated at read time. Batched Announcements are off chain and MUST be validated at read time (See: Validation).
Ordering Announcements
Frequency Messages are well ordered:
- Frequency: Block number ascending
- Frequency: Block index ascending (unique)
- DSNP Standard: Order Announcements in a Batch Publication File by row appearance order.
Human Order
Due to the asynchronous nature of networks and batching, it is possible that the canonical ordering of Announcements is wrong from a human viewpoint. With dependent Announcements, where one Announcement refers to another Announcement, the order may be inferred differently than the canonical ordering. It is left to user interfaces to handle these situations.
Retrieval
Frequency nodes provide an RPC interface
messages.getBySchemaId()
with paginated responses that differ based on the Schema.
Frequency nodes can provide a websocket interface that will emit an event for each block that has one or more messages of a given schema in that block.
The
messages::MessagesStored
event can be used to know when to call the RPC interface to retrieve the messages.
See the Frequency Documentation for more details on Message retrieval.
Batch Publication Validation
The Frequency Message for a Batch Publication has several important fields for validation:
Field | Description |
---|---|
provider_msa_id | MSA Id of the provider sending the message |
cid | The Content IDentifier v1 for IPFS content |
payload_length | Expected length of the content from IPFS |
block_number | Block number that the message was recorded on the chain. |
File Validation
- Retrieve the file from the IPFS network using the
cid
. - Verify the file hash by comparing it to the hash included in the
cid
. (Required for non-trusted IPFS nodes.) - Verify that the byte length of the retrieved file matches the
payload_length
.
Publication Announcements Validation
- Collect the unique set of
fromId
values. - Use the Custom RPC
msa.checkDelegations()
with thefromId
values as thedelegator_msa_ids
and theprovider_msa_id
at theblock_number
. - The
fromId
values thatmsa.checkDelegations()
verifies as having a delegation atblock_number
are valid Announcements. - Set the
schema_id
parameter to the Schema Id used in the Frequency Message
Announcement Duplicates
Duplicate Announcements MUST be rejected or ignored.
Operations
Method of execution
DSNP Operations are executed on Frequency via on-chain transactions, also known as extrinsic calls. An extrinsic is a type of function defined in a Substrate Pallet. A Pallet is a Substrate runtime module and also a Cargo crate.
MSA Pallet
Responsible for DSNP Identity Operations and Delegation management.
Schema Pallet
Responsible for managing the data structures for DSNP Announcements and User Data.
Messages Pallet
Responsible for most DSNP Announcement Operations (see Publishing).
Stateful Storage Pallet
Responsible for DSNP User Data (see User Data) and select Announcement Operations (see Publishing).
Documentation Links
Principals
Every Frequency transaction for DSNP is accompanied by an Schnorrkel/Ristretto X25519 (“sr25519”) derived cryptographic signature and associated public key. When the transaction occurs, the signature is validated and the MSA Id is retrieved.
DSNP Term | Frequency Term |
---|---|
User | User / Delegator |
Delegate | Provider |
Failure Handling
Frequency has a variety of errors that fall into these classes:
- Timeout: A transaction with an unknown status after a set time.
- Node Rejection: Public nodes may reject transactions from unknown parties.
- Node Validation Error: A node will reject malformed or invalid transactions when possible.
- Execution Error: A transaction that was included in a block but failed upon execution.
Transactions are not automatically resubmitted. Check with the Frequency Documentation for more information regarding errors.
List of Operations
Write operations are via Transactions (also called Extrinsics): pallet::extrinsic()
Operation | Principal(s) | Pallet::Extrinsic | State Change Record |
---|---|---|---|
Create Identifier | None | ||
msa::create() , | |||
msa::create_sponsored_account_with_delegation() | Identifier Creation Record | ||
Retire Identifier | User | ||
msa::retire_msa() | Identifier Retirement Record | ||
Define Delegation | User AND Delegate | ||
msa::grant_delegation() , | |||
msa::create_sponsored_account_with_delegation() | Delegation Definition Record | ||
Revoke Delegation | User | ||
msa::revoke_delegation_by_delegator() | Delegation Revocation Record | ||
Revoke Delegation | Delegate | ||
msa::revoke_delegation_by_provider() | Delegation Revocation Record | ||
Add Control Key | User | ||
msa::add_public_key_to_msa() | Control Key Addition Record | ||
Remove Control Key | User | ||
msa::delete_msa_public_key() | Control Key Removal Record | ||
Publish Announcement | User OR Delegate | ||
messages::add_onchain_message() | Announcement Published Record | ||
Publish Batch | User OR Delegate | ||
messages::add_ipfs_message() | Batch Published Record | ||
Get User Data | Any | See User Data: Read Operation Mapping | - |
Replace User Data | User OR Delegate | See User Data: Write Operation Mapping | User Data Replaced Record |
State Change Records
State Change Records constitute the observable output of a DSNP system. Frequency uses a combination of on chain data to store and Events to notify for Records.
Frequency Transaction Identifier
Frequency uses the hash of the Operation transaction to then request a node report the transaction status and any Events related to that hash. (Client libraries usually have this built in.)
Frequency Records
Frequency produces three types of data that map or point to DSNP Records.
Events
Events are generated on each block and may contain a pointer to the Record data instead of the entire Record.
Events are referenced by pallet::EventName
.
Messages
Frequency Messages store Announcements or Batch Publication Records. It uses the Messages Pallet.
Messages are retrieved via state queries (pallet.stateQuery
) or RPC calls (pallet.rpcCall()
).
State
Frequency State stores data associated with an Identity. It can also be used to look up the state of prior Records related to an Identity.
State data is retrieved via state queries (pallet.stateQuery
) or RPC calls (pallet.rpcCall()
).
Record Mappings
Record Type | Record Pointer/Location |
---|---|
Identifier Creation Record | Event: |
msa::MsaCreated State: | |
msa.publicKeyToMsaId | |
Identifier Retirement Record | Event: |
msa::MsaRetired | |
Delegation Definition Record | Event: |
msa::DelegationGranted , | |
msa::DelegationUpdated RPC: | |
msa.checkDelegations() , | |
msa.getGrantedSchemasByMsaId() | |
Delegation Revocation Record | Event: |
msa::DelegationRevoked RPC: | |
msa.checkDelegations() | |
Control Key Addition Record | Event: |
msa::PublicKeyAdded State: | |
msa.publicKeyToMsaId | |
Control Key Removal Record | Event: |
msa::PublicKeyDeleted State: | |
msa.publicKeyToMsaId | |
Announcement Published Record | Event: |
messages::MessagesStored RPC: | |
messages.getBySchemaId() | |
Batch Published Record | Event: |
messages::MessagesStored RPC: | |
messages.getBySchemaId() | |
Failure Record | See section on Failure Handling |
User Data Replaced Record | Event: |
stateful_storage::ItemizedPageUpdated , | |
stateful_storage::ItemizedPageDeleted , | |
stateful_storage::PaginatedPageUpdated , | |
stateful_storage::PaginatedPageDeleted RPC: | |
statefulStorage.getPaginatedStorage() , | |
statefulStorage.getItemizedStorage() See Also: Read User Data |
Frequency User Data
On Frequency, User Data is mapped to Schemas which use Stateful Storage for storage and retrieval of the data.
User Data Sets
User Data Set | Deployed Schema Ids | Frequency Model Type | Frequency Payload Location | Settings |
---|---|---|---|---|
Public Key (Key Agreement) | Mainnet: 7 Testnet (Paseo): 7 | |||
AvroBinary | ||||
Itemized | Append Only, Signature Required | |||
Public Follows | Mainnet: 8 Testnet (Paseo): 8 | |||
AvroBinary | ||||
Paginated | None | |||
Private Follows | Mainnet: 9 Testnet (Paseo): 9 | |||
AvroBinary | ||||
Paginated | None | |||
Private Connections | Mainnet: 10 Testnet (Paseo): 10 | |||
AvroBinary | ||||
Paginated | None | |||
Public Key (Assertion Method) | Mainnet: 14 Testnet (Paseo): 14 | |||
AvroBinary | ||||
Itemized | Signature Required | |||
Profile Resources | Mainnet: 15 Testnet (Paseo): 15 | |||
AvroBinary | ||||
Itemized | None |
Private Connection Pseudonymous Relationship Identifiers (PRIds) are stored alongside Private Connections in the same Stateful Storage page.
For historical reasons, bytes for the Public Key (Key Agreement) schema are serialized to the corresponding Avro bytes
field without the leading multicodec identifier.
Readers should prepend or assume leading 0xec 0x01
bytes (indicating the x25519-pub
multicodec).
Writers should omit these leading bytes when inserting new items.
Source code for each schema is located in the LibertyDSNP/schemas repository.
Read Operation Mapping
Stateful data is retrieved via state queries (pallet.stateQuery
) or RPC calls (pallet.rpcCall()
).
Model Type | DSNP Properties | Query/Call |
---|---|---|
Itemized | Entity Tag | |
stateful_storage.getItemizedStorage() | ||
Paginated | Entity Tag, Chunked | |
stateful_storage.getPaginatedStorage() |
Write Operation Mapping
Write operations are via Transactions (also called Extrinsics): pallet::extrinsic()
Entity Tags
Frequency requires the hash of current state for any Stateful Storage change.
Activity Content Specification
Version 1.3.0
Content references shared via the DSNP consist of URLs pointing to documents containing Activity Streams JSON objects. For the purposes of the DSNP, restrictions are placed on the Activity Streams 2.0 specification.
JSON-LD and Activity Streams
All DSNP Activity Content is compatible with the Activity Streams 2.0 specification.
While there are some DSNP extensions, they are guaranteed to use non-colliding terms.
Therefore, the JSON-LD @context
field is set to https://www.w3.org/ns/activitystreams
according to Activity Streams 2.0 §2.1.
Core Activity Content Types
DSNP uses only the following content types at the root level:
Name | Description | DSNP Announcements |
---|---|---|
Note | standard user content | Broadcast, Reply, Update |
Profile | user profile content | Profile |
Associated Types
Name | Description | Specification |
---|---|---|
Location | add a location to content | Activity Vocabulary |
Tag | add a tag to content | Activity Vocabulary |
Attachments | supported attachment types | Activity Vocabulary |
Hash | content validation hash | DSNP Extension |
Supported URL Schema
URLs in DSNP-compatible Activity Content MUST use one of the following URL schemes:
Scheme | Description | Reference | DSNP Version Added |
---|---|---|---|
HTTPS | Hypertext Transfer Protocol Secure | RFC2818 | 1.0 |
HTTP | Hypertext Transfer Protocol | RFC2616 | 1.0 |
Libraries
Name | Language(s) |
---|---|
LibertyDSNP/activity-content | JavaScript/TypeScript |
LibertyDSNP/activity-content-java | Java/Kotlin |
LibertyDSNP/activity-content-swift | Swift |
Releases
Version | Description | Release Date | Changelog |
---|---|---|---|
1.3.0 | DIP-257 | 2024-09-17 | Changelog |
1.2.0 | DIP-210 | 2023-04-11 | Changelog |
1.1.0 | DIP-158 | 2022-05-05 | Changelog |
1.0.0 | Initial Release | 2021-09-09 | Changelog |
Non-Normative
Additional Fields
Implementers may choose to support more of the Activity Streams standard as long as it does not conflict with this specification, but should note that other implementations may not recognize those additions. Implementers who extend their support for Activity Streams objects beyond the subset defined here do so at their own risk.
Activity Stream Type: Note
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
@context | Activity Streams 2.0 | YES | JSON-LD @context | MUST be set to https://www.w3.org/ns/activitystreams |
type | Activity Vocabulary 2.0 | YES | Identifies the type of the object | MUST be set to Note |
content | Activity Vocabulary 2.0 | YES | Text content of the note | |
mediaType | Activity Vocabulary 2.0 | YES | MIME type for the content field | MUST be set to a supported MIME type |
published | Activity Vocabulary 2.0 | YES | The time of publishing | MUST be ISO8601 |
name | Activity Vocabulary 2.0 | no | The display name for the note | |
attachment | Activity Vocabulary 2.0 | no | Array of attached links or media | MUST be one of the Supported Attachments |
tag | Activity Vocabulary 2.0 | no | Array of tags/mentions | MUST follow Tag Type |
location | Activity Vocabulary 2.0 | no | For location | MUST follow Location Type |
Supported Content MIME Types
Format | MIME Type | Specification(s) |
---|---|---|
Plain | text/plain |
Examples
{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Note",
"content": "Hello world!",
"mediaType": "text/plain",
"published": "1970-01-01T00:00:00+00:00"
}
{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Note",
"content": "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.",
"mediaType": "text/plain",
"attachment": [
{
"type": "Link",
"href": "https://en.wikipedia.org/wiki/Citation_needed"
}
],
"published": "1970-01-01T00:00:00+00:00"
}
Activity Stream Type: Profile
Profiles are used to provide additional user information display.
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
@context | Activity Streams 2.0 | YES | JSON-LD @context | MUST be set to https://www.w3.org/ns/activitystreams |
type | Activity Vocabulary 2.0 | YES | Identifies the type of the object | MUST be set to Profile |
name | Activity Vocabulary 2.0 | no | The display name for the profile | |
icon | Activity Vocabulary 2.0 | no | An array of avatars of the profile | MUST follow Image Link Type |
summary | Activity Vocabulary 2.0 | no | Used as a plain text biography of the profile | |
published | Activity Vocabulary 2.0 | no | The time of publishing | MUST be ISO8601 |
location | Activity Vocabulary 2.0 | no | For location | MUST follow Location Type |
tag | Activity Vocabulary 2.0 | no | For tags or mentions | MUST follow Tag Type |
Examples
{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Profile",
"name": "John Doe",
"summary": "John Doe is actually a small kitten. See pfp.",
"icon": [
{
"type": "Link",
"href": "https://upload.wikimedia.org/wikipedia/commons/thumb/1/1d/1-month-old_kittens_32.jpg/256px-1-month-old_kittens_32.jpg",
"mediaType": "image/jpeg",
"width": "256",
"height": "171",
"hash": [
"bdyqphnphmjdoumkxqbsuspribxvlsx2hx6525u3fh2dkr5bxnqritzi"
]
},
{
"type": "Link",
"href": "https://upload.wikimedia.org/wikipedia/commons/thumb/1/1d/1-month-old_kittens_32.jpg/64px-1-month-old_kittens_32.jpg",
"mediaType": "image/jpeg",
"width": "64",
"height": "43",
"hash": [
"bdyqktr7p5hc27bx4ernmngs6tj7uyukfb4atrtq44mdmx4yntuh2s5y"
]
}
],
"published": "1970-01-01T00:00:00+00:00"
}
Associated Type: Hash
NOT part of the Activity Streams 2.0 Vocabulary.
Activity objects linking to external content such as audio, image or video files must include a "hash"
field for users to validate linked content.
The value of this "hash"
field must be an array of strings.
Each item in the array MUST be a valid DSNP Content Hash for the content associated with the hash.
Example
This example gives SHA-256 and BLAKE3 hashes for the PDF version of the DSNP whitepaper.
{
"hash": [
"bciqdnu347gcfmxzbkhgoubiobphm6readngitfywktdtbdocgogop2q",
"bdyqhwoxp2mc6oyaqpqyd2fvaxralslk32ggazv6nxpp342iec6652tq"
]
}
Associated Type: Location
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
type | Activity Vocabulary 2.0 | YES | Identifies the type of the object | MUST be set to Place |
name | Activity Vocabulary 2.0 | YES | The display name for the location | |
accuracy | Activity Vocabulary 2.0 | no | The accuracy of the coordinates as a percentage. (e.g. “94.0” means “94.0% accurate”) | |
altitude | Activity Vocabulary 2.0 | no | The altitude of the location | |
latitude | Activity Vocabulary 2.0 | no | The latitude of the location | |
longitude | Activity Vocabulary 2.0 | no | The longitude of the location | |
radius | Activity Vocabulary 2.0 | no | The area around the given point that comprises the location | |
units | Activity Vocabulary 2.0 | no | The units for radius and altitude (defaults to meters) | MUST be one of these: cm , feet , inches , km , m , miles |
Warning
Location data may pose a privacy danger to users. Users should be warned before publishing location data.
Example
{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Note",
"content": "I'm in NYC!",
"mediaType": "text/plain",
"location": {
"type": "Place",
"name": "New York City, NY",
"latitude": "40.73",
"longitude": "-73.93",
"accuracy": "94.0"
},
"published": "1970-01-01T00:00:00+00:00"
}
Associated Type: Tag
Hashtag
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
name | Activity Vocabulary 2.0 | YES | The text of the tag |
Example
{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Note",
"content": "I love the #dsnp spec!",
"mediaType": "text/plain",
"tag": [
{
"name": "#dsnp"
}
],
"published": "1970-01-01T00:00:00+00:00"
}
Mention
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
name | Activity Vocabulary 2.0 | no | The text of the tag | |
type | Activity Vocabulary 2.0 | YES | Identifies the tag as type Mention | MUST be Mention |
id | Activity Vocabulary 2.0 | YES | Link to the user mentioned | MUST be a DSNP User URI |
Example
{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Note",
"content": "@sally should check out the #dsnp spec!",
"mediaType": "text/plain",
"tag": [
{
"name": "#dsnp"
},
{
"name": "@sally",
"type": "Mention",
"id": "dsnp://12345678"
}
],
"published": "1970-01-01T00:00:00+00:00"
}
Associated Attachments
Audio
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
type | Activity Vocabulary 2.0 | YES | Identifies the type of the object | MUST be set to Audio |
url | Activity Vocabulary 2.0 | YES | An array of links for given audio content in different formats | MUST be an Audio Link AND MUST have at least one supported audio MIME type |
name | Activity Vocabulary 2.0 | no | The display name for the audio file | |
duration | Activity Vocabulary 2.0 | no | Approximate duration of the audio |
Audio Link
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
type | Activity Vocabulary 2.0 | YES | Identifies the type of the object | MUST be set to Link |
href | Activity Vocabulary 2.0 | YES | The URL for the given audio content | MUST be a Supported URL Schema |
mediaType | Activity Vocabulary 2.0 | YES | MIME type of href content | MUST follow |
hash | DSNP 1.0 | YES | Array of hashes for linked content validation | MUST include at least one supported hash |
Supported Audio MIME Types
Format | MIME Type | Specification(s) |
---|---|---|
MP3 | audio/mpeg | RFC3003 |
OGG | audio/ogg | RFC5334 |
WebM | audio/webm | WebM standard |
Example
{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Note",
"attachments": [
{
"type": "Audio",
"name": "The Scream",
"url": [
{
"type": "Link",
"href": "https://upload.wikimedia.org/wikipedia/commons/d/d9/Wilhelm_Scream.ogg",
"mediaType": "audio/ogg",
"hash": [
"bdyqbcji3okmzxobvaqgduz5prixmumyndzopyufultmslndi4pdebii"
]
}
],
"duration": "1S"
}
],
"published": "1970-01-01T00:00:00+00:00"
}
Image
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
type | Activity Vocabulary 2.0 | YES | Identifies the type of the object | MUST be set to Image |
url | Activity Vocabulary 2.0 | YES | An array of links for given image content in different formats | MUST be an Image Link AND MUST have at least one supported image MIME type |
name | Activity Vocabulary 2.0 | no | The display name or alt text for the image |
Image Link
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
type | Activity Vocabulary 2.0 | YES | Identifies the type of the object | MUST be set to Link |
href | Activity Vocabulary 2.0 | YES | The URL for the given image | MUST be a Supported URL Schema |
mediaType | Activity Vocabulary 2.0 | YES | MIME type of href content | |
hash | DSNP 1.0 | YES | Array of hashes for linked content validation | MUST include at least one supported hash |
height | Activity Vocabulary 2.0 | no | A hint as to the rendering height in device-independent pixels | |
width | Activity Vocabulary 2.0 | no | A hint as to the rendering width in device-independent pixels |
Supported Image MIME Types
Format | MIME Type | Specification(s) |
---|---|---|
JPEG | image/jpeg | RFC2045 |
PNG | image/png | W3C PNG Standard |
SVG | image/svg+xml | W3C SVG standard |
WebP | image/webp | WebP standard |
GIF | image/gif | RFC2045 |
Example
{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Note",
"attachments": [
{
"type": "Image",
"name": "One of the founders of DSNP",
"url": [
{
"type": "Link",
"href": "https://upload.wikimedia.org/wikipedia/commons/a/ae/Mccourt.jpg",
"width": 350,
"height": 228,
"mediaType": "image/jpg",
"hash": [
"bciqjiqcidmzuqpvrl5cocu3l4z2uhj22xqruht3d5kx7qijvfbnjlda",
"bdyqbjzt5drgji5w7xhsddsynusgx2vdmakcsrr4sfin5fyfkwlpup6q"
]
}
]
}
],
"published": "1970-01-01T00:00:00+00:00"
}
Video
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
type | Activity Vocabulary 2.0 | YES | Identifies the type of the object | MUST be set to Video |
url | Activity Vocabulary 2.0 | YES | An array of links for given video content in different formats | MUST be a Video Link AND MUST have at least one supported video MIME type |
name | Activity Vocabulary 2.0 | no | The display name for the video | |
duration | Activity Vocabulary 2.0 | no | Approximate duration of the video |
Video Link
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
type | Activity Vocabulary 2.0 | YES | Identifies the type of the object | MUST be set to Link |
href | Activity Vocabulary 2.0 | YES | The URL for the given content | MUST be a Supported URL Schema |
mediaType | Activity Vocabulary 2.0 | YES | MIME type of href content | |
hash | DSNP 1.0 | YES | Array of hashes for linked content validation | MUST include at least one supported hash |
height | Activity Vocabulary 2.0 | no | A hint as to the rendering height in device-independent pixels | |
width | Activity Vocabulary 2.0 | no | A hint as to the rendering width in device-independent pixels |
Supported Video MIME Types
Format | MIME Type | Specification(s) |
---|---|---|
MPEG | video/mpeg | RFC2045 |
OGG | video/ogg | RFC5334 |
WebM | video/webm | WebM standard |
H265 | video/H265 | RFC7798 |
MP4 | video/mp4 | RFC4337 |
Example
{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Note",
"attachments": [
{
"type": "Video",
"name": "One of the founders of DSNP",
"duration": "PT10M32S",
"url": [
{
"type": "Link",
"href": "https://upload.wikimedia.org/wikipedia/commons/c/c0/Big_Buck_Bunny_4K.webm",
"width": 4000,
"height": 2250,
"mediaType": "video/webm",
"hash": [
"bdyqed7dnok3batd7tr64trqmovfxam5tqsgxkiv2op5765pq43swtui"
]
}
]
}
],
"published": "1970-01-01T00:00:00+00:00"
}
Link
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
type | Activity Vocabulary 2.0 | YES | Identifies the type of the object | MUST be set to Link |
href | Activity Vocabulary 2.0 | YES | The URL for the given link | MUST be a Supported URL Schema |
name | Activity Vocabulary 2.0 | no | The display name for the link |
Examples
{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Note",
"content": "Check out the DSNP Website!",
"mediaType": "text/plain",
"attachment": [
{
"type": "Link",
"name": "DSNP Website",
"href": "https://dsnp.org"
}
],
"published": "1970-01-01T00:00:00+00:00"
}
Attestations
Attestation attachments are DSNP extensions to the Activity Content model that allow users to attach Verifiable Credentials corresponding to an Attribute Set Type with their DSNP profile or a social media post.
The Verifiable Credential found at the indicated URL must include the user’s DSNP User URI in its credentialSubject.id
field.
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
type | Activity Vocabulary 2.0 | YES | Identifies the type of the object | MUST be set to Attestation |
url | Activity Vocabulary 2.0 | YES | An array of links for the given credential | MUST be a Verifiable Credential Link |
name | Activity Vocabulary 2.0 | no | The display name for the attestation |
Verifiable Credential Link
Attestation attachments must contain a link to a Verifiable Credential document.
The link must contain a relationship identifier in the form of a DSNP Attribute Set Type (the rel
field) to allow applications to determine the type of credential expected.
Property | Base Spec | Required | Description | Restrictions |
---|---|---|---|---|
type | Activity Vocabulary 2.0 | YES | Identifies the type of the object | MUST be set to Link |
rel | Activity Vocabulary 2.0 | YES | The attestation’s attribute set type | MUST be a DSNP Attribute Set Type corresponding to the referenced credential document |
href | Activity Vocabulary 2.0 | YES | The URL for the associated Verifiable Credential | MUST be a Supported URL Schema |
mediaType | Activity Vocabulary 2.0 | YES | MIME type of href content | MUST be set to |
application/vc | ||||
hash | DSNP extension | YES | Array of hashes for linked content validation | MUST include at least one supported hash |
Example
{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Profile",
"content": "I am a doctor. You're doing this all wrong.",
"mediaType": "text/plain",
"attachments": [
{
"type": "Attestation",
"name": "My Degree",
"url": [
{
"type": "Link",
"rel": "did:dsnp:123456$AcmeMedicalAssociationPhysician",
"href": "https://acmemedicalassn.org/credentials/degree-123.json",
"mediaType": "application/vc",
"hash": [
"bciqdnu347gcfmxzbkhgoubiobphm6readngitfywktdtbdocgogop2q"
]
}
]
}
],
"published": "1970-01-01T00:00:00+00:00"
}
Verifiable Credentials Specification
Version 1.3.0
Attribute Sets and Attestation attachments shared via DSNP reference data documents containing Verifiable Credentials and related objects. For DSNP purposes, certain restrictions and extensions are applied to the base World Wide Web Consortium (W3C) specification documents noted below. When there are DSNP extensions, they are guaranteed to use non-colliding terms.
It is anticipated that these W3C specifications will evolve, and applications interacting with Verifiable Credentials, Schemas, and DIDs via DSNP are encouraged to be as flexible as possible in their syntax support as new versions and supporting software libraries become available. Therefore, compatibility with the following versions and specifications should be construed as the minimal level of support required to ensure interoperability between DSNP applications.
Compatibility
Current usage with DSNP relies on the following specifications:
Specification | Version/Status | Relevant JSON-LD @context Values |
---|---|---|
Verifiable Credentials Data Model | 1.1 (W3C Recommendation) 2.0 (W3C Candidate Recommendation Draft 15 September 2024) | https://www.w3.org/2018/credentials/v1 |
https://www.w3.org/ns/credentials/v2 | ||
Verifiable Credential Data Integrity | 1.0 (W3C Candidate Recommendation Draft 16 September 2024) | https://w3id.org/security/data-integrity/v2 |
https://w3id.org/security/multikey/v1 | ||
Verifiable Credentials JSON Schema | (W3C Candidate Recommendation Draft 12 September 2024) | |
Decentralized Identifiers (DIDs) | 1.0 (W3C Recommendation 19 July 2022) | https://www.w3.org/ns/did/v1 |
Cryptography
The Data Integrity specification provides a generic format for expressing cryptographic proofs, where the detailed representation of each data item is defined in individual cryptosuites.
DSNP compliant applications MUST support the following cryptosuites, which correspond to the allowed algorithms for the DSNP User Data item assertionMethodPublicKeys
:
Specification | Version/Status | Multikey codec |
---|---|---|
Data Integrity EdDSA Cryptosuites | 1.0 (W3C Candidate Recommendation Draft 16 September 2024) | ed25519-pub |
DSNP Usage Details
DSNP incorporates the following W3C JSON-LD types. See the individual pages for details of restrictions to and extensions on each type.
Name | Since DSNP Version |
---|---|
Verifiable Credential | 1.3 |
Verifiable Credential Schema | 1.3 |
DID | 1.3 |
Libraries
Name | Language(s) |
---|---|
LibertyDSNP/dsnp-verifiable-credentials | JavaScript/TypeScript |
LibertyDSNP/dsnp-did-resolver | JavaScript/TypeScript |
Releases
Version | Description | Release Date | Changelog |
---|---|---|---|
1.3.0 | DIP-257 | 2024-09-17 | Changelog |
Non-Normative
Additional Fields
DSNP application developers may choose to support more of the relevant JSON-LD vocabularies from the specifications above as long as doing so does not conflict with this specification, but should note that other conforming applications may not recognize those additions. Implementers who extend their support for Verifiable Credentials objects beyond the subset defined here do so at their own risk.
Verifiable Presentations
Verifiable Presentations combine one or more Verifiable Credentials in a single document. These might be existing Verifiable Credentials, whether or not previously published, or new Verifiable Credentials derived from non-public source credentials using methods like zero-knowledge proofs. At this time, Verifiable Presentations are not directly tied to DSNP content or announcements. However, application and service providers may be interested in implementing functionality using Verifiable Presentations with DSNP Verifiable Credentials. Applications are encouraged to follow the verification guidelines for DSNP Verifiable Credentials even when encountering those credentials as part of a Verifiable Presentation.
Verifiable Credential
Credentials should conform to the Verifiable Credentials Data Model 1.1 or Verifiable Credentials Data Model 2.0, expressed as JSON-LD. The fields noted below consist only of those with requirements or semantics that differ from the underlying specification.
JSON-LD Contexts
The following JSON-LD context values are valid for use with DSNP:
Object | @context |
---|---|
Verifiable Credential | https://www.w3.org/2018/credentials/v1 |
https://www.w3.org/ns/credentials/v2 |
Verifiable Credential Document
A DSNP Verifiable Credential JSON document specializes the following fields:
Property | Required | JSON Type | Description | Restrictions |
---|---|---|---|---|
@context | YES | Array of strings | JSON-LD @context | MUST include "https://www.w3.org/2018/credentials/v1" or "https://www.w3.org/ns/credentials/v2" |
type | YES | Array of strings | Type of credential | MUST contain "VerifiableCredential" per specification; if a credentialSchema is present, MUST also contain a string matching the referenced JSON Schema’s title . |
issuer | YES | String or Object | DID or object identifying credential issuer | See Issuer |
credentialSubject | YES | Object | Object identifying subject and claim | See Credential Subject |
credentialSchema | no | Object | Reference to schema for this credential | See Credential Schema |
proof | no | Object | Cryptographic proof of authorship by issuer | See Proof |
Issuer
The issuer
field is required, even for credentials that do not include a proof
.
Unsigned documents should still use the document creator’s DID.
The issuer
field may be a string that is the issuer’s DID, or an object with the issuer’s DID as the id
property.
In the object form, the issuer may optionally include references to its own credentials, which may be used by a verifier processing the DSNP trust extension.
Property | Required | JSON Type | Description | Restrictions |
---|---|---|---|---|
id | YES | String | DID of issuer | Must be a DSNP DID |
authority | no | Array | List of relevant credentials with issuer as subject | See Authority |
Authority
Objects in the issuer.authority
array MUST have the following properties:
Property | Required | JSON Type | Description | Restrictions |
---|---|---|---|---|
id | YES | String | URL of Verifiable Credential | MUST be a DSNP Verifiable Credential |
rel | YES | String | The linked credential’s attribute set type | MUST be a DSNP Attribute Set Type corresponding to the referenced credential document |
digestMultibase | YES | Array | Array of hashes for linked content validation | MUST include at least one supported hash |
Examples:
"issuer": "did:dsnp:86420"
"issuer": {
"id": "did:dsnp:86420",
"authority": [
{
"id": "https://mydsnpcreds.net/86420-fp",
"rel": "did:dsnp:123456$FairTradeProducer",
"digestMultibase": [
"bciqdnu347gcfmxzbkhgoubiobphm6readngitfywktdtbdocgogop2q"
]
},
{
"id": "https://mydsnpcreds.net/86420-ap",
"rel": "did:dsnp:123456$AppleProducer",
"digestMultibase": [
"bdyqhwoxp2mc6oyaqpqyd2fvaxralslk32ggazv6nxpp342iec6652tq"
]
}
]
}
Credential Subject
Property | Required | JSON Type | Description | Restrictions |
---|---|---|---|---|
id | YES | String | Subject of the claim data | If describing a DSNP User, MUST be a DSNP User URI; if describing DSNP content, MUST be a DSNP Content URI |
The remainder of the contents of the credentialSubject
value MUST conform to the JSON schema found via the credentialSchema
object.
Credential Schema
Property | Required | JSON Type | Description | Restrictions |
---|---|---|---|---|
id | YES | String | URL of schema document | If type is JsonSchemaCredential , MUST reference a DSNP Verifiable Credential Schema |
Proof
Property | Required | JSON Type | Description | Restrictions |
---|---|---|---|---|
type | YES | String | Proof type | MUST be DataIntegrityProof |
verificationMethod | YES | String | URI of public key | MUST reference a public key within a DSNP DID document |
cryptosuite | YES | String | Cryptographic algorithm identifier | MUST be eddsa-rdfc-2022 |
proofPurpose | YES | String | Context for assessing proof | MUST be assertionMethod |
proofValue | YES | String | Digital signature proof | MUST be a multibase-encoded signature using base58btc encoding |
Example
{
"@context": [
"https://www.w3.org/ns/credentials/v2"
],
"type": [
"VehicleOwner",
"VerifiableCredential"
],
"issuer": "did:dsnp:654321",
"issuanceDate": "2024-02-12T03:09:40.497Z",
"credentialSchema": {
"type": "JsonSchemaCredential",
"id": "https://dsnp.org/schema/examples/vehicle_owner.json"
},
"credentialSubject": {
"id": "dsnp://999999",
"make": "DeLorean",
"model": "DMC-12",
"year": 1981
},
"proof": {
"type": "DataIntegrityProof",
"created": "2024-02-12T03:09:44Z",
"verificationMethod": "did:dsnp:654321#z6Mkumvf8FpJybzi9byLX7qAhTPuKpqH7d5rWyqcrKJ9Mies",
"cryptosuite": "eddsa-rdfc-2022",
"proofPurpose": "assertionMethod",
"proofValue": "z2YLydotgaGsbRGRxPzmoscd7dH5CgGHydXLKXJXefcT2SJGExtxmkJxGfUGoe81Vm62JGEYrwcS6ht1ixEvuZF9c"
}
}
Verifiable Credential Schema
The schema for a DSNP Verifiable Credential MAY be defined using the format described in the Verifiable Credentials JSON Schema W3C Candidate Recommendation Draft. This format provides a credential wrapper around a JSON schema document.
Empty schemas (Verifiable Credential Schema Documents with "jsonSchema": {}
) are allowed; however, schemaless credentials may be preferred in this situation.
Empty schemas are useful in situations where no attribute data fields are relevant but the schema author wishes to assert authorship.
Verifiable Credential Schema Document
A Verifiable Credential Schema document is a JSON-LD document that is itself a Verifiable Credential with a claim that includes the schema and (optionally) the DSNP extension described below. A schema credential specializes the meaning of the following fields:
Property | Required | JSON Type | Description | Restrictions |
---|---|---|---|---|
type | YES | Array of strings | Type of credential | MUST contain the strings "VerifiableCredential" and "JsonSchemaCredential" |
credentialSubject | YES | Object | Object containing JSON schema and DSNP extensions | See Credential Subject |
credentialSchema | YES | Object | Metaschema defining JSON Schema types | See Credential Schema |
Credential Schema
The required credentialSchema
object MUST follow the specification and contain:
{
"id": "https://www.w3.org/2022/credentials/v2/json-schema-credential-schema.json",
"type": "JsonSchema",
"digestSRI": "sha384-S57yQDg1MTzF56Oi9DbSQ14u7jBy0RDdx0YbeV7shwhCS88G8SCXeFq82PafhCrW"
}
Credential Subject
A DSNP Verifiable Credential Schema document’s credentialSubject
object uses the following keys:
Property | Required | JSON Type | Description | Restrictions |
---|---|---|---|---|
type | YES | String | Type of subject matter | MUST be JsonSchema |
jsonSchema | YES | Object | Embedded JSON Schema object | See JSON Schema |
dsnp | no | Object | DSNP extension object | See DSNP Extensions |
JSON Schema
The JSON Schema object is formed as follows:
Property | Required | JSON Type | Description | Restrictions |
---|---|---|---|---|
$schema | YES | String | JSON Schema version identifier | MUST be a valid DSNP JSON Schema version |
title | YES | String | Title of the schema | MUST match a string within the referencing credential’s type array |
The remainder of the schema object should be interpreted as per the relevant JSON schema specification.
Valid DSNP JSON Schema Versions
DSNP applications MUST support the following JSON Schema versions:
$schema value | Specification |
---|---|
https://json-schema.org/draft/2020-12/schema | JSON Schema 2020-12 Update |
Other JSON Schema versions MAY be supported, but creators should be aware that not all DSNP applications will be able to correctly validate credentials against schema definitions that are not universally supported.
DSNP Extensions
An optional dsnp
object within the credentialSubject
provides additional semantics for DSNP applications and users interacting with the schema.
Display Extension
The optional display
key within the dsnp
extension object MUST contain a label
key that has an object value where the object is a map of one or more language tags to human-readable string values.
Language tags should follow BCP-47/RFC-5646 (as used in the HTTP Content-Language
header).
A content language key of "*"
indicates a wildcard or default value, as in HTTP.
Example:
"display": {
"label": {
"en-US": "Verified User",
"de-DE": "Verifizierter Benutzer"
}
}
A DSNP application MAY use the indicated values as a hint when displaying information about the subject of a Verifiable Credential utilizing this schema.
Trust Extension
The optional trust
key within the dsnp
extension object enables the author of a schema to describe the types of Verifiable Credentials that an issuer of credentials with this schema must have to be considered an accredited source and therefore trusted by an application.
These accreditations MUST be included by the issuer using the authority
key under issuer
in each relevant Verifiable Credential document they create.
The trust
object contains one or both of the oneOf
or allOf
keys, each containing an array of strings.
Each string MUST be a valid DSNP Attribute Set Type.
See Attribute Sets for a detailed discussion of Attribute Set Types and their derivation.
A trust
object containing both oneOf
and allOf
sections should be construed as requiring any of the oneOf
constraints as well as all of the allOf
constraints.
Example:
"trust": {
"oneOf": [
"did:dsnp:123456$AppleProducer",
"did:dsnp:123456$OrangeProducer"
],
"allOf": [
"did:dsnp:999999$FairTradeProducer"
]
}
The above example might be translated as “A credential conforming to this schema should only be trusted if its issuer is accredited as a FairTradeProducer
and is also either an AppleProducer
or an OrangeProducer
.”
Example
{
"@context": [
"https://www.w3.org/ns/credentials/v2"
],
"id": "https://dsnp.org/schema/examples/vehicle_owner.json",
"type": [
"VerifiableCredential",
"JsonSchemaCredential"
],
"issuer": "did:dsnp:123456",
"issuanceDate": "2024-02-12T03:09:40.497Z",
"expirationDate": "2099-01-01T00:00:00.000Z",
"credentialSchema": {
"id": "https://www.w3.org/2022/credentials/v2/json-schema-credential-schema.json",
"type": "JsonSchema",
"digestSRI": "sha384-S57yQDg1MTzF56Oi9DbSQ14u7jBy0RDdx0YbeV7shwhCS88G8SCXeFq82PafhCrW"
},
"credentialSubject": {
"type": "JsonSchema",
"jsonSchema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "VehicleOwner",
"type": "object",
"properties": {
"credentialSubject": {
"type": "object",
"properties": {
"make": {
"type": "string"
},
"model": {
"type": "string"
},
"year": {
"type": "number"
}
},
"required": [
"make",
"model",
"year"
]
}
}
},
"dsnp": {
"display": {
"label": {
"en-US": "Vehicle Owner"
}
},
"trust": {
"oneOf": [
"did:dsnp:123456$AuthorizedCarDealership",
"did:dsnp:123456$OfficialTaxOffice"
]
}
}
},
"proof": {
"type": "DataIntegrityProof",
"created": "2024-02-12T03:09:44Z",
"verificationMethod": "did:dsnp:123456#z6MkhFqFV1UD6GYbQ1V4HSF3pnGprovQceXbgwbLKrhxnbny",
"cryptosuite": "eddsa-rdfc-2022",
"proofPurpose": "assertionMethod",
"proofValue": "z3ENMBo7UyKvZkJSBMqvGFDB1uvGChP1QTEgiCsNzt23ciq4RTffk2xxz2noXxxHL6wPmN1Bp2fSvTuMnvHQFP9tp"
}
}
DIDs
DSNP Users are referenced in Verifiable Credentials documents via DIDs compliant with the Decentralized Identifiers v1.0 specification.
Applications that make use of Verifiable Credentials issued by DSNP Users MUST be able to resolve DSNP User DIDs to DID documents in order to verify that signatures were created with keys controlled by that user. A DSNP DID document is effectively a document aggregating DSNP User Data. Compliant DSNP systems are encouraged to provide their own DID resolver libraries.
DID Method Syntax
A DSNP DID uses dsnp
(lowercase) as the method name and the DSNP User Id as the method-specific identifier, as the following example illustrates:
did:dsnp:123456
This DID identifies the DSNP User with User Id 123456. In this format, the DSNP User Id is serialized in decimal form with no additional punctuation.
References to identifiers within a DID document are formed by appending URL fragments to a DID.
For example, a Verifiable Credential might reference the public key to be used to verify a document’s signature as did:dsnp:123456#key1
, assuming the document included a verificationMethod
with "id": "key1"
.
DID Document
A DSNP DID document is a JSON-LD document representing key material associated with a DSNP User.
Property | Required | JSON Type | Description | Restrictions |
---|---|---|---|---|
@context | YES | Array of strings | JSON-LD @context | MUST include "https://www.w3.org/ns/did/v1" |
id | YES | String | The DID described by this document | MUST be of the form did:dsnp:{userId} |
verificationMethod | no | Array of Verification Method objects | Set of public keys that may be referenced from assertionMethod , authentication , and keyAgreement arrays | |
assertionMethod | no | Array | Set of public keys used to generate digital signatures | MUST include or reference all relevant keys present in DSNP User Data assertionMethodPublicKeys |
authentication | no | Array | Set of public keys used as DSNP control keys | MAY include or reference any keys used as control keys |
keyAgreement | no | Array | Set of public keys used to generate shared secrets | MUST include or reference all relevant keys present in DSNP User Data keyAgreementPublicKeys |
Additional properties defined in the DID specification MAY be present.
Public Key Representation
As per the DID specification, each element of the assertionMethod
and keyAgreement
arrays may be a string reference to the object with a matching id
field in the verificationMethod
array, or the Verification Method object itself.
Each Verification Method describes a public key.
These keys MUST be taken from the DSNP User Data associated with the user referenced by the DID.
DSNP DID resolvers MUST serialize keys in the Multikey format, defined in Verifiable Credential Data Integrity 1.0.
Property | Required | JSON Type | Description | Restrictions |
---|---|---|---|---|
@context | YES | String | JSON-LD @context | MUST be https://w3id.org/security/multikey/v1 |
id | YES | String | The full URI of this key | MUST be of the form did:dsnp:{userId}#{identifier} ; MUST be unique within the document |
type | YES | String | The type of this verification method | MUST be Multikey |
controller | YES | String | The controller of this key | MUST be the DID of the enclosing document |
publicKeyMultibase | YES | String | The public key | MUST be a valid multicodec-prefixed public key in base58btc encoding |
Example
{
"@context": [
"https://www.w3.org/ns/did/v1"
],
"id": "did:dsnp:645313",
"authentication": [
{
"@context": "https://w3id.org/security/multikey/v1",
"id": "did:dsnp:645313#z6QP1gZa1xAGCtsPzZSc5mdTDtrGsWUyf12TmU6pSu15SXUr",
"type": "Multikey",
"controller": "did:dsnp:645313",
"publicKeyMultibase": "z6QP1gZa1xAGCtsPzZSc5mdTDtrGsWUyf12TmU6pSu15SXUr"
}
],
"assertionMethod": [],
"keyAgreement": [
{
"@context": "https://w3id.org/security/multikey/v1",
"id": "did:dsnp:645313#z6LSoYFtPRBEizFQ2zYXEXBPP96t9gNFWjZVJTfXWqzMhw9e",
"type": "Multikey",
"controller": "did:dsnp:645313",
"publicKeyMultibase": "z6LSoYFtPRBEizFQ2zYXEXBPP96t9gNFWjZVJTfXWqzMhw9e"
}
]
}
Glossary
- Activity Content - User-generated social networking content in a JSON format defined by the Activity Streams 2.0 specification with DSNP extensions. Some DSNP Announcements contain the URL and hash for an Activity Content document.
- Announcement - Content, or references to content, that communicate user activity to the network.
- Application - A computer program that helps a User interact with a DSNP system
- Batch Publication - A collection of Announcements bundled together into a single file using the Parquet format
- Consumer - An application or user who reads content from a DSNP system
- DSNP System - A consensus system (and its surrounding services) that enables DSNP operations and generates DSNP state change records
- Hash - A string of bytes generated from a hash function, a cryptographic function whose output is effectively unique for any given input without any information from the input being accessible in the output
- User - A human using a DSNP application. See article for an exception