Interplanetary File System - no longer need to copy to the network

The idea of ​​IPFS is good for everyone, but there was only one flaw with it. Data uploaded to the network were copied to the block storage by doubling the space they occupied. Moreover, the file was cut into blocks that are of little use for reuse.

There is an experimental option --nocopy , which eliminates this drawback. In order to use it, it is necessary to fulfill several conditions.

Also, a new type of identifiers. We will analyze it too.

Let me remind you: InterPlanetary File System is a new decentralized file sharing network (HTTP-server, Content Delivery Network ). About her, I began the story in the article "Interplanetary File System IPFS" .

--nocopy

This option causes IPFS to use source files as a source of blocks. Thus, files are not copied to the block storage and do not occupy 2 times more space.

To use this option, follow these steps:

1. need to enable filestore

   
   

    ipfs config --json Experimental.FilestoreEnabled true 

   
   

2. in the directory where the ".ipfs" directory is located (it is usually in the user directory) you need to make a link to the directory or file that needs to be downloaded to the network

   
   

   The file can be linked by hardlink:

   
   

    fsutil hardlink "[  .ipfs]\[ ]" "[  ]\[ ]" 

   
   

   or

   
   

    mklink /h "[  .ipfs]\[ ]" "[  ]\[ ]" 

   
   

   The catalog can be linked with a symbolic link:

   
   

    linkd "[  .ipfs]\[ ]" "[  ]\[ ]" 

   
   

   or

   
   

    mklink /j "[  .ipfs]\[ ]" "[  ]\[ ]" 

   
   

3. And now we add

   
   

    ipfs add -r -w --nocopy "[  .ipfs]\[   ]" 

   
   

   The -w wraps the target in a directory, preserving its name.

   
   

   The result will be the identifiers CIDv1 and CIDv0 (multihash)

   
   

Webseed

Now you can add WebSeed to the magnet or torrent .

For file:

 http://127.0.0.1:8080/ipfs/[]/[ ] http://gateway.ipfs.io/ipfs/[]/[ ] 

For the catalog:

 http://127.0.0.1:8080/ipfs/[]/ http://gateway.ipfs.io/ipfs/[]/ 

Example of a magnet with WebSeed

 magnet:?xt=urn:btih:953edbe75de612bc966194d2ee60099b3bc1a1aa&dn=Magnet-.txt&ws=http://gateway.ipfs.io/ipfs/QmfQCxNW9r2974xR5dXopXfQqsEvgexhza6aQgqTGL7Yh3/Magnet-.txt 

Content ID (CID)

In connection with these changes in the IPFS appeared RAW blocks. The --nocopy key automatically enables the use of RAW blocks. But you can enable this mode with the key --raw-leaves . In this regard, a new CID (Content IDentifier) ​​or in Russian "content identifier" has appeared.

Old id

Called CIDv0 and usually has a constant prefix "Qm".

CIDv0 is just a multi-cache in Base58

 [varint ID ][varint  ][] 

CIDv0 Example: QmPbs8syAxac39bcNuMLpHXnqjKUguqakCM8LN8sZVPD9R

New ID

RAW blocks have their own CID. It can be distinguished by the constant prefix "zb2rh".

CIDv1 contains more information.

 [ ][varint  CID][varint  ][varint ID ][varint  ][] 

CIDv1 Example: zb2rhe143L6sgu2Nba4TZgFMdPidGMA6hmWhK9wLUoVGWYsR7

Let's take it apart

1. z - base58 Bitcoin [ base prefix ]

   
   

   base58: b2rhe143L6sgu2Nba4TZgFMdPidGMA6hmWhK9wLUoVGWYsR7
   translate into
   HEX: 01 55 12 20 6D542257CBD1BE7FD0AE8914F42066BCBF1E79487EF67B959A86DBEE4670B386

   
   

2. 01 - v1 [ varint version CID ]

   
   

3. 55 - raw binary [varint content type ]

   
   

4. 12 - sha2-256 [varint Hash ID ]

   
   

5. 20 - 32 bytes [varint is long hash]

   
   

6. 6D542257CBD1BE7FD0AE8914F42066BCBF1E79487EF67B959A86DBEE4670B386 - sha2-256 digest [hash]

   
   

Conclusion

The option --nocopy helps a lot when you want to share a Wikipedia dump with the world. Or other useful but very large in volume arrays of information.

Links

Windows links, symbolic and not only
ipfs command reference
List of experimental IPFS features

Other parts

1. Interplanetary File System IPFS
2. We publish the site in the interplanetary file system IPFS
3. We host a site in the interplanetary IPFS file system under Windows
4. No need to copy to the network
5. Switch your site to localhost (local IPFS gateway)
6. Localize the global gateway or sites in IPFS