diff --git a/content/en/smart-contracts/developing-contracts/using-raas/index.md b/content/en/smart-contracts/developing-contracts/using-raas/index.md new file mode 100644 index 000000000..7f725475e --- /dev/null +++ b/content/en/smart-contracts/developing-contracts/using-raas/index.md @@ -0,0 +1,76 @@ +--- +title: "Using RaaS" +description: "This page explains how to use RaaS from a smart contract." +lead: "RaaS refers to replication, renew and repair of storage deals. This guide shows how to enable RaaS from a smart contract with the RaaS starter kit." +draft: false +images: [] +type: docs +menu: + smart-contracts: + parent: "smart-contracts-developing-contracts" + identifier: "Using RaaS" +weight: 120 +toc: true +--- + +This guide explains how to create a storage deal from a smart contract, including replication, renewal and repair storage requirements. This utilizes the [RaaS Starter Kit](https://github.com/filecoin-project/raas-starter-kit). + +## Interacting with the Smart Contract + +First, you need to: +- EITHER start an instance of the BaseInterface by deploying a contract that inherits from `IAggregatorOracle` (you can do so via. `yarn deploy` in the [RaaS Starter Kit](https://github.com/filecoin-project/raas-starter-kit)) +- OR use an existing instance of the FullInterface located at + - Calibration Testnet: `0x6ec8722e6543fB5976a547434c8644b51e24785b` + +Interact with the smart contract by submitting a CID of your choice to the `submit` function. This will create a new deal request that will be picked up by the RaaS services. + +```javascript +// contractInstance is the address of the contract you deployed or the FullInterface address above. +const dealStatus = await ethers.getContractAt("DealStatus", contractInstance); +// Submit the CID of the file you want to upload to the Filecoin network in the following way. +await dealStatus.submit(ethers.utils.toUtf8Bytes(newJob.cid)); +``` + +The [RaaS Starter Kit](https://github.com/filecoin-project/raas-starter-kit) actually provides you with a frontend that allows you to upload your file to Lighthouse, get a CID for the uploaded file, then seamlessly submit the CID to the smart contract (accessible via `yarn start`). + +First, you need to know how to register the various RaaS workers. Note that RaaS functionality will NOT function automatically if deals are only created using submit function. + +## Add Replication, Renewal, Repair Workers + +You can add workers to perform replication, renewal, and repair jobs by having them listen to the `SubmitAggregatorRequest`. The methods for doing so differ between the Base and Full interfaces. + +If you are running a base interface (specifically, the one in the [RaaS Starter Kit](https://github.com/filecoin-project/raas-starter-kit)), there's an event listener inside the RaaS service node that you can use to listen for new deal requests. +This event listener performs processing for each job submitted to the contract to add RaaS service workers and eventually to call `complete` on the contract. + +```javascript +// Initialize the listener for the Deal Creation event +async function initializeDealCreationListener() { + const dealStatus = await ethers.getContractAt(contractName, contractInstance); + + /// Logic for handling SubmitAggregatorRequest events + function handleEvent(transactionId, cid) { + console.log(`Received SubmitAggregatorRequest event: (Transaction ID: ${transactionId}, CID: ${cid})`); + // ... other code to handle the event emission + + (async () => { + // ... other code + + // After processing this event, reattach the event listener + if (dealStatus.listenerCount("SubmitAggregatorRequest") === 0) { + dealStatus.once("SubmitAggregatorRequest", handleEvent); + } + })(); + } + + // Start listening to the first event and recursively handle the next events + if (dealStatus.listenerCount("SubmitAggregatorRequest") === 0) { + dealStatus.once("SubmitAggregatorRequest", handleEvent); + } +} +``` + +To use this, simply do `yarn service` in the terminal and proceed through the frontend by uploading a file. Then, register the workers using the autocompleted CID that appears in the box. + +If you want to register the workers manually for a job that you didn't upload, paste in the known CID of your file into the box and register the jobs. + +Check out an implementation of this RaaS base interface, into a full interface, available at [Lighthouse.storage](https://docs.lighthouse.storage/lighthouse-1/filecoin-virtual-machine/fvm-contract-overview). diff --git a/content/en/smart-contracts/programmatic-storage/Interfaces/index.md b/content/en/smart-contracts/programmatic-storage/Interfaces/index.md new file mode 100644 index 000000000..92d14ee77 --- /dev/null +++ b/content/en/smart-contracts/programmatic-storage/Interfaces/index.md @@ -0,0 +1,96 @@ +--- +title: "RaaS base and full interfaces" +description: "This page explains the specifications for RaaS base and full interfaces." +lead: "RaaS has a base and full interface to enable replication, renew and repair of storage deals." +draft: false +images: [] +type: docs +menu: + smart-contracts: + parent: "smart-contracts-programmatic-storage" + identifier: "Interfaces" +weight: 120 +toc: true +--- + +## RaaS Base Interface + +RaaS refers to replication, renewal and repair as a service, for data stored in storage deals on Filecoin. Developers can leverage the RaaS base interface to provide RaaS features, within their storage solution, using the RaaS Starter Kit. + +The base interface requires 4 components: + +- Client who has data to upload + +- Aggregator platform (a type of storage solution) that receives the Client’s data and makes a storage deal on Filecoin + +- The RaaS node, hosted by the Aggregator, that checks if the storage deal requires replication, renewal and/or repair + +- The RaaS DealStatus smart contract that the RaaS node executes checks with + +### In the example of replication: + +1. The client requests the aggregator platform to store data and defines the number of replicas of their data to store (e.g. “This data needs to have a minimum of 10 copies on the network”) + +2. The aggregator takes the client’s data, generates a ```CID``` for the data and makes the storage deal onto Filecoin. + +3. The aggregator registers a replication job to the RaaS node and defines the number of replicas of their data to store, by calling the ```/register_jobs``` API on the RaaS node. + +4. The RaaS node periodically checks the client data’s ```CID``` for its deal status on Filecoin. + +5. The RaaS node requests deal status with the DealStatus smart contract, via ```getActiveDeals(CID)``` and checks if the client’s data is stored with the accurate number of replicas. + +6. The DealStatus smart contract returns the information of active deals to the RaaS node. + +7. If the number of replicas does not match the client’s requirements, the RaaS node will fetch the data via its ```CID``` and resubmit a request to create new storage deals (repeat step 2). + +### In the example of renewal: + +1. The client requests the aggregator platform to store data and defines the renewal threshold for the data’s storage deal (e.g. renew storage deals that are 1 month away from expiry) + +2. The aggregator platform takes the client’s data, generates a ```CID``` for the data and makes the storage deal onto Filecoin. + +3. The aggregator registers a renewal job to the RaaS node and defines the renewal threshold for the data’s storage deal, by calling the ```/register_jobs``` API on the RaaS node. + +4. The RaaS node periodically checks the client data’s ```CID``` for its deal status on Filecoin. + +5. The RaaS node requests deal status with the DealStatus smart contract, via ```getExpiringDeals(CID)``` and checks if any of its active deals is expiring. + +6. The DealStatus smart contract returns the information of expiring deals to the RaaS node. + +7. If deals with the client’s data are expiring, the RaaS node will fetch the data via its ```CID``` and resubmit a request to create new storage deals (repeat step 2). + +Below is a diagram for both renewal and replication: + + +shapes (9) + + +### In the example of repair: + +1. The client requests the aggregator platform to store data and defines the repair threshold for the data’s storage deal (e.g. “this deal needs repairing if it is not proven active for X epochs”) + +2. The aggregator platform takes the client’s data, generates a ```CID``` for the data and makes the storage deal onto Filecoin. + +3. The aggregator registers a repair job to the RaaS node and defines the repair threshold for the data’s storage deal, by calling the ```/register_jobs``` API on the RaaS node. + +4. The RaaS node periodically checks the Client data’s ```CID``` for its deal status on Filecoin. + +5. The RaaS node receives client data’s storage deal information from the aggregator including deal ID and miner ID via ```getAllDeals(CID)```. + +6. The RaaS node periodically checks with Lotus, if the deal ID and corresponding miner, is actively being proven by the miner on Filecoin. The node calls StateMarketStorageDeal, with provided deal and miner IDs. + +7. If the deal ID and corresponding deal sector are not being actively proven for X epochs, the deal will require repairing. + +8. To repair the deal, the RaaS node will fetch the data via its ```CID``` and resubmit a request to the aggregator, to create new storage deals (repeat step 2). + +Below is a diagram for repair: + + +shapes (10) + + +Check out the [RaaS starter kit](https://github.com/filecoin-project/raas-starter-kit) to build your own RaaS interface. + +## RaaS Full Interface + +The RaaS full interface refers to a solution that utilizes the base interface, hosts all the components (except for the Client) and provides a seamless interface to the Client. [Lighthouse.storage](https://www.lighthouse.storage/) is the first FVM project to integrate RaaS and provide a full interface, via its SDK. See its website and [docs](https://docs.lighthouse.storage/lighthouse-1/filecoin-virtual-machine/fvm-contract-overview). diff --git a/content/en/smart-contracts/programmatic-storage/RaaS/index.md b/content/en/smart-contracts/programmatic-storage/RaaS/index.md new file mode 100644 index 000000000..b946fb110 --- /dev/null +++ b/content/en/smart-contracts/programmatic-storage/RaaS/index.md @@ -0,0 +1,50 @@ +--- +title: "RaaS" +description: "This page introduces RaaS, which refers to replication, renew and repair of storage deals." +lead: "RaaS refers to replication, renew and repair of storage deals. It is a feature of programmatic storage on Filecoin, enabled by FVM." +draft: false +images: [] +type: docs +menu: + smart-contracts: + parent: "smart-contracts-programmatic-storage" + identifier: "RaaS" +weight: 120 +toc: true +--- + +## Introduction + +What does it mean to replicate, renew, or repair data storage deals through a Filecoin smart contract? RaaS refers to replication, renewal and repair as a service, for data stored in storage deals on Filecoin: + +- Replication refers to the option to store a user-defined number of replicas of your data. + +- Renewal refers to the option to automatically observe on-chain storage deals until the expiry of their deal term, and automatically renew the deal. + +- Repair refers to the automatic observation of storage deals, to ensure they are not in a faulted sector. If they are, these workers repair them automatically. + +- “As a service" refers to the opportunity to provide these RaaS features as services, to incentivize clients to use them and/or storage platforms to enable them. These are a part of the Filecoin programmable storage market with FVM. + +The motivation behind RaaS is to enable perpetual data storage on Filecoin, where deals and the data inside are perpetually maintained and stored. + +## Interfaces + +There are two interfaces available to use with RaaS, for different purposes - the base interface (self-hosted RaaS) and full interface (hosted RaaS). + +The base interface refers to a self-hosted RaaS service, where the Client runs all components. The full interface refers to a fully hosted RaaS service that is provided to the Client. + +At the base interface, any contract or storage platform that has the metadata of the stored data, can request the RaaS node to perform its functions. However, the RaaS node has to be self-hosted in order to function. Storage platforms have the opportunity to build a full interface, with the RaaS node, to provide hosted replication, renewal and repair services to clients. + +A RaaS node, which monitors deals done through aggregators on Filecoin, will take action if replication, renew, or repair requirements are not observed. Note that RaaS nodes are only able to monitor deals from aggregators, since the nodes listen to the SubmitAggregatorRequest event in order to pick these deals up. + +Here is a brief outline of the RaaS process: + +1. An Aggregator sends a CID to the RaaS node, requesting for replication, renewal and/or repair as needed. + +2. The RaaS smart contracts maintains the information of deals created by the RaaS, including deal_id and miner_id. + +3. The RaaS node checks the deal status on the Filecoin network, by interacting with the RaaS` smart contract. + +4. If the deal requires replication, renewal and/or repair, the RaaS node fetches data from the given deal ID (and provided data CID) and requests the aggregator to make a new deal. + +Read more about the interfaces and how to build with them here or get started with the RaaS starter kit [here](https://github.com/filecoin-project/raas-starter-kit). diff --git a/content/en/smart-contracts/programmatic-storage/_index.md b/content/en/smart-contracts/programmatic-storage/_index.md new file mode 100644 index 000000000..57f01d890 --- /dev/null +++ b/content/en/smart-contracts/programmatic-storage/_index.md @@ -0,0 +1,8 @@ +--- +title: "Programmatic storage" +description: "" +lead: "" +draft: false +images: [] +type: docs +---