Crate pallet_bonsai

Expand description

Provides a decentralised authority for data storage.

In Totem we require an off-chain searchable database that may end up containing billions of records. IPFS is not a solution as the type of data to be stored may be queried, editied, and each time IPFS cannot overwrite or update existing datasets. Additionally IPFS may drop files that are not considered current, used or needed, which is not ideal for static records like invoices.

We wanted a solution where permission for storing an editing data should not be dependent on third-party authentication and access was global, recoverable and self-sovereign.

Bonsai is a simple protocol, for allowing independent databases to come to a consensus on content. It works by assuming that the data to be stored must be previously authenticated by its owner on-chain.

How it works

Firstly, a reference to the record is created either on-chain or offchain by an account which immediately becomes its owner. The reference is a hash (H256) with sufficient entropy to be unique per the record. A transaction is sent to the blockchain at some point associating the reference to an address for the first time. The reference is considered to be the key to some other data which is not suitable for onchain storage, but will be stored in an offchain database. The offchain database will only accept new or changing records, provided that it can a) find the reference hash onchain, and b) an associated data-hash which it also finds on chain with a hash of the incoming data. The data may be plaintext or encrypted, neither matters as long as the hash of this data matches onchain data-hash. As the on-chain transaction validates the signature, the off-chain database does not need to authenticate the client that communicates the insertion or change request as it has already been “pre-authorised” by the blockchain runtime. Totem believes there is a fee market for storage in this model.

Process

A third party database receives a request to store some data. The Database queries the blockchain to find out:

Does the reference hash exist on chain and of it does, then collect the associated data-hash also stored onchain;
Upon confirmation the reference hash exists, hashing the received data and compare the data-hash to the one found on chain. If it does not match, then do nothing (effectively rejecting the attempt to store the data), and if it does match then store the data using the reference hash as the key;
In the event that an reference hash already exists, the data-hash obtained from the blockchain is always king. Provided it matches, overwrite exiting data.