Skip to main content
Version: 0.6

Storage Interface

Introduction​

The high-level account API takes care of publishing updates to an identity and storing secrets securely. It does the latter by using an implementation of the Storage interface. In this section, we will go into more depth of the interface, and how to implement that interface.

The key idea behind the interface is strongly inspired by the architecture of key management systems (KMS) or secure enclaves: once private keys are entered into the system, they can never be retrieved again. Instead, all operations using the key will have to go through that system. This approach is what allows Storage implementations to be architected more securely than simply storing and loading private keys from a regular database. Of course, the security is directly dependent on the concrete implementation, which is why we provide one such implementation using Stronghold, and strongly recommend using it. However, there are cases where one cannot use Stronghold or may want to integrate key management of identities into their own KMS or similar, which is why the Storage interface is an abstraction over such systems. Any implementation of that interface can then be used by the Account.

The storage interface has three major categories of functions. A brief overview of those functions:

  • DID Operations: Management of identities.
    • did_create: Based on a private key, or a generated one, creates a new DID.
    • did_list: List all DIDs in this Storage.
    • did_exists: Returns whether the given DID exists in this Storage.
    • did_purge: Wipes all data related to the given DID.
  • Key Operations: Various functionality to managing cryptographic keys.
    • key_generate: Generates a new key for the given DID.
    • key_insert: Inserts a pre-existing private key for the given DID location.
    • key_public: Calculates and returns the public key for the given location to a private key.
    • key_delete: Removes the key at the given location.
    • key_sign: Signs the given data with the key at the given location.
    • key_exists: Returns whether the key at the given location exists.
  • Data Operations: Used for keeping state persistent. Storages only need to serialize and store the data.
    • chain_state_get: Returns the ChainState data structure for the given DID.
    • chain_state_set: Sets the ChainState data structure for the given DID.
    • document_get: Returns the DID document for the given DID.
    • document_set: Sets the DID document for the given DID.

Storage Layout​

Identifiers​

There are two types of identifiers in the interface, DIDs and key locations. A DID identifies an identity, while a key location identifies a key. An implementation recommendation is to use the DID as a partition key. Everything related to a DID can be stored in a partition identified by that DID. Importantly, the location of a key is only guaranteed to be unique within the DID partition it belongs to. If no partitioning is used, then DID and key location should be combined (e.g. concatenated) to produce a single, globally unique (i.e. across all identities) identifier for a key in storage.

Representations​

A KeyLocation is a compound identifier based on the fragment of a verification method and the hash of a public key. The motivation for this design is that a KeyLocation can be derived given a DID document and one of its verification methods. Thus, no additional state is necessary.

Canonical string representations of the IotaDID and KeyLocation type can be obtained using the string representation of a DID and the canonical method on KeyLocation respectively. These representations are intended to be kept stable as much as possible.

Example layout​

This illustrates the recommended approach for partitioning the storage layout (where location -> key is a mapping from location to key):

  • did:iota:Ft3wA8Tv2nF25hij3aegR54Wvqju7t5zqW9xnCB5L3Wu
    • sign-0:16843234495045965331 -> 0xc6f0dbacd56156ff4c383d549ac61ada87f8aa69454f3bfae99f5fa9e093a5c3
    • kex-0:7560300328640998700 -> 0xe494e36164e0a760140f3a9ab7dfdad38edac698f93d5239655dbd7499194760
  • did:iota:DSvXWs7FUch9MQcaUKmrRFZyHYcHwt3t3pbjvKsQBfep
    • sign-0:16843234495045965331 -> 0xc6f0dbacd56156ff4c383d549ac61ada87f8aa69454f3bfae99f5fa9e093a5c3
    • kex-0:16546298247591944074 -> 0x8e1d037cd343f84276ab737b638da9095bcb6052f7fd9628d21d20f434f9959a
    • key:8559754420653090937 -> 0x4ef484a54aa16503878aa1ecaa6d73cb8254aefa3f80a569ed33ca685289d01e

Note how fragments (such as kex-0) can appear more than once, but the hash of the public key - calculated from the stored private key - makes the location unique in general.

caution

Although unlikely in practice, even the same private key can be used across different DIDs, which produces the same key location (here sign-0:16843234495045965331) and it's important that these are stored independently, so that deleting one does not accidentally delete the other. Hence why a key's full identifier in storage needs to be based on the DID and the key location.

That said, the following flattened structure also satisfies the requirements:

  • did:iota:Ft3wA8Tv2nF25hij3aegR54Wvqju7t5zqW9xnCB5L3Wu:sign-0:16843234495045965331 -> 0xc6f0dbacd56156ff4c383d549ac61ada87f8aa69454f3bfae99f5fa9e093a5c3
  • did:iota:Ft3wA8Tv2nF25hij3aegR54Wvqju7t5zqW9xnCB5L3Wu:kex-0:7560300328640998700 -> 0xe494e36164e0a760140f3a9ab7dfdad38edac698f93d5239655dbd7499194760
  • did:iota:DSvXWs7FUch9MQcaUKmrRFZyHYcHwt3t3pbjvKsQBfep:sign-0:16843234495045965331 -> 0xc6f0dbacd56156ff4c383d549ac61ada87f8aa69454f3bfae99f5fa9e093a5c3
  • did:iota:DSvXWs7FUch9MQcaUKmrRFZyHYcHwt3t3pbjvKsQBfep:kex-0:16546298247591944074 -> 0x8e1d037cd343f84276ab737b638da9095bcb6052f7fd9628d21d20f434f9959a
  • did:iota:DSvXWs7FUch9MQcaUKmrRFZyHYcHwt3t3pbjvKsQBfep:key:8559754420653090937 -> 0x4ef484a54aa16503878aa1ecaa6d73cb8254aefa3f80a569ed33ca685289d01e

The primary advantage of the partitioning is that it simplifies the implementation of the did_purge operation, which wipes all data belonging to a given DID. With partitioning, this operation can simply wipe the partition whereas a storage with a flattened layout will have to do more work.

Indexing​

The interface has two methods called did_list and did_exists. These return the list of stored DIDs, and whether a DID exists in storage, respectively. Implementations are thus expected to maintain a list or index of stored DIDs. An identity created with did_create is added to the index, while an identity deleted through did_purge is removed from the index.

If the storage implementation can be accessed concurrently, then access to the index needs to be synchronized, since it is unique per storage instance.

Implementation​

The IOTA Identity framework ships two implementations of Storage. The MemStore is an insecure in-memory implementation intended as an example implementation and for testing. The secure and recommended Storage is Stronghold. Stronghold may be interesting for implementers to look at, as it needs to deal with some challenges the in-memory version does not have.

This section will detail some common challenges and embeds the MemStore implementations in Rust and TypeScript.

Challenges​

The did_create method takes the fragment of the initial verification method, the name of a network in which the DID will eventually exist, and an optional private key. From these inputs, it either generates a key or uses the passed private key to calculate the public key and from that derive the DID. In case a key needs to be generated, the challenge is to obtain the location for the key to be stored at. Since the key location depends on the public key, but key generation likely needs a location for the key to be stored at, there is a circular dependency that needs to be resolved. This can be resolved in at least two ways.

  1. Generate the key at a random location, then derive the actual location and move the key there
  2. If moving a key is not possible, then an additional mapping from key location to some storage-internal location identifier can be maintained. Then it's possible to generate the key at some storage-internal location, calculate the key location and store the mapping.

Since this also needs to happen before the DID can be derived from the public key, similar approaches can be used to work around the not-yet available DID partition key. Storages may choose to have one statically identified partition where keys are generated initially, and then moved from there. Storages whose restrictions do not allow for this, may want to use the flattened storage layout described in example layout and use the mapping approach.

Storage Test Suite​

The StorageTestSuite can be used to test the basic functionality of storage implementations. See its documentation (Rust docs, Wasm docs) for more details.

Examples​

This section shows the Rust and TypeScript MemStore implementations, which are thoroughly commented.

v0.6/bindings/wasm/examples-account/src/custom_storage.ts
loading...