“It is the one obvious use case that billions of Internet users can be exposed to before the rest of web3.” Advocates have high hopes for decentralised file storage, which is already being used to ensure the integrity of documents from presidential handovers to NFT assets. Ekin Genç asks whether the sector is worth the hype.
There’s so much stuff online, found on almost 2 billion websites and more. But everything you see is hosted by a few centralised storage players like Amazon and Google. That’s a problem: when their servers go down, a big chunk of the Internet also goes kaputt. The underlying code and governance structures may be decentralised, but the continued reliance on centralised servers for the front shop of web3 projects hinders the purpose.
Could the massive realm of online data be stored decentrally, reducing this core dependency? A growing cluster of web3 sector around decentralised file storage solutions thinks so, and they’re building a use case that anyone on the internet can benefit from
Proponents value decentralisation because it eliminates single points of failure. If data is stored decentrally across several nodes running independent of each other – unlike a giant Google server – then the risks of complete failure are significantly reduced, if not completely eliminated.
“We see that permanent, decentralised file storage has found strong product market fit in ensuring that assets traded in web3 apps, like NFTs, do not disappear,” Sam Williams, founder and CEO of the decentralised file storage provider Arweave, tells Culture3.
From web3 dApps to NFT art, data is central to the web3 experience and goes hand in hand with the wider web3 ethos. But the online infrastructure is crucial to everyone’s experience. If decentralised file storage can ensure fast, safe, and secure data transfers online, then it presents a genuine use case for billions of online users around the world. It also makes possible an enormous scope of new opportunities built on a more secure way to store and share information on the internet.
Arweave powers what it calls the “permaweb,” a suite of applications and sites built on top of the blockchain, maintained by computers distributed around the world. Arweave also lets users store data permanently by paying a one-time storage fee, which varies by file size and network congestion at time of storage. Arweave maintains network security – guaranteeing that the files are indeed stored – through a cryptographic innovation called proof-of-access, an alternative consensus mechanism, like proof-of-stake, designed specifically for Arweave. In proof-of-access the nodes will receive a computational request to confirm that the latest block and a randomly-picked past block still store the data. Other chains using proof-of-access include Filecoin, 0Chain, and Skynet.
Although many pixelated NFT artwork like CryptoPunks or MoonCats with smaller file sizes are stored on-chain on Ethereum, the newly proof-of-stake chain isn’t designed to hold large amounts of data. Ethereum requires the whole chain to be accounted for when running a node, which makes the costs of data storage prohibitive and ever-increasing. As of this writing, Ethereum contains around 900gb of data, up from 480gb the year before, and more than twice the size of Bitcoin. It is a rapid increase, and bearing in mind that every node on the network has to store this data, one fraught with difficulties.
Most NFTs, or rather, the data files (like JPEGs) which the tokens point to, aren’t stored decentrally. Instead, an NFT today is usually a data file that points to an image stored offchain. It’s why purely on-chain work commands particular interest and value, because when it’s offchain, it’s more vulnerable. And it shows, because offchain NFTs keep disappearing: the total value of missing NFTs is estimated to be around $918 million, represented by more than a fifth of all NFTs ever created.
Daniel Norman, a developer advocate at InterPlanetary File System, one of the major providers in the sector, estimates that 75 zettabytes of data will be generated by 2025, all of which he believes should be stored onchain. That’s 75 trillion gigabytes, so twelve zeroes.
“Centralised storage is limited to a small number of providers, limiting pricing and efficiency, and very little interoperability, leading to consumer lock-in,” Daniel explains, adding that “storage is costly, and it is not always verifiable and immutable. This is why a decentralised, efficient, and robust foundation for storing information is needed.”
The impact of early providers can already be seen. Filecoin, which launched in October 2020, decentrally (but offchain) hosts the metadata and content of 80 million NFTs. It is also home to a large array of historically significant data, such as that provided by Starling Lab, a historical archive for journalists, lawyers, and academics. Founded by Stanford University and the USC Shoah Foundation, the research lab is using decentralised file storage to “meet the technical and ethical challenges of establishing trust in our most sensitive digital records.” To begin, Starling is preserving the testimonials of 55,000 genocide survivors, and has worked with Reuters to record contemporary events in Kurdistan and Myanmar.
In December 2022, they partnered with Rolling Stone Magazine to archive, authenticate, and draw attention to the Bijeljina massacre, which occured in Serbia in 1992. Preserving original photographs taken by photojournalist Ron Haviv, Starling ensures that this evidence, comprising photos, documents, and other sources, is not forgotten.
Marta Belcher, Chair of the Filecoin Foundation, tells Culture3 that this decentralised version of the Internet lets online data “stay up even if some nodes fail, and the availability of information is not dependent on any one server or company.” Other historical materials stored on Filecoin include video evidence of Russian war crimes in Ukraine, documents uploaded by nonprofits and newsrooms via MuckRock’s DocumentCloud, and the City of New York’s public datasets on climate and demographics. Belcher explains decentralised data storage “provides a more robust platform for humanity’s most important information.” Information that society shouldn’t risk ever forgetting.
“We’re innovating to meet challenges of establishing trust in our most sensitive digital records.”
— Starling Lab
Filecoin’s network capacity is provided by over 4,000 storage providers from across the world. They have contributed more than 18 exabytes (EiB) of storage capacity. To put that in perspective, 18 exabytes amounts to all of the written works of humankind – from the beginning of recorded history to today – in all languages, multiplied by 20 times. Or 50 million hours of 4K video recording. Filecoin currently spearheads the space, holding about 90% of the total data stored on decentralised storage.
Filecoin was developed by Protocol Labs, who also produced IPFS. Whilst Filecoin provides the infrastructure for data preservation and verification, such as coin rewards to storage miners for storing data and to retrieval miners for serving data to those who need it, IPFS functions as a protocol for routing content. Think of it like decentralised https service. Data is stored and served elsewhere much like a website is, by Filecoin and other organisations such as Arwearve, but to do so they rely on IPFS, which tells them where the data is.
“It is evident that decentralised storage networks are inherently complex,” Daniel acknowledges. “The ability to abstract that complexity so that users can be productive is a massive opportunity that depends on continuous feedback and evolution.” Today, that evolution is being driven by a “large network of contributors, communities, and companies” in the decentralised storage ecosystem, directed at building the technical infrastructure to make Filecoin a seamless solution for non-technical users.
One of the most anticipated developments in the decentralised data storage space is the forthcoming launch of the Filecoin Virtual Machine (FVM). FVM will introduce programmable storage features like auctions and storage bounties to drive down the cost of storage, cross-chain bridges to the likes of Ethereum and Solana to make storing files more accessible, and Layer-2 solutions like better reputation systems to make the process much more efficient.
Getting the average internet user to appreciate the complex data routing protocols and content addressing, and then use peer-to-peer storage networks to storage their data, feels like a big ask. Web2 is intuitive and heavily optimised for user experience. Indeed, entire careers are built around user experience design, working to make products more user-friendly. By contrast, web3 feels still geeky, like the early days of the Internet.
”Decentralised file storage is a very complicated problem to solve; it has been on the table since the very early Ethereum,” says an individual familiar with the decentralised storage provider Swarm, who preferred to remain anonymous. Swarm's hope is that this use case will be the tool to onboard new swathes of people into crypto.
With a goal “to take the best from web2 and replace the complex experience of web3 with a familiar one,” Swarm’s desktop app is consciously designed to feel like a web2 app, requiring no crypto knowledge from the user while affording the benefits of the underlying crypto tech stack without most users knowing it exists.
Another project trying to abstract away complexity and build a user-facing product is Fileverse. “There are plenty of privacy, security, censorship resistance, and decentralisation benefits that could attract a wide range of internet users to these networks — and away from centralised services like Google, AWS, DropBox, Wetransfer, and so on,” Fileverse’s co-founder miroyato told Culture3.
“It is the one familiar and obvious use case.”
— miroyato, co-founder of Fileverse
But to get to a world in which users are no longer tied to a platform like Google Drive because they can’t stomach moving across a lifetime of documents, things need to become much simpler, which is what decentralised storage organisations are trying to achieve. Fileverse offers an intuitive website for users to drop files and upload them to decentralised servers in two clicks. No account creation is required, and documents can be shared by a URL or token-gated.
Although crypto wallets are often associated with financial applications (after all, the term is wallet), Fileverse wants to expand that perception, treating the wallet as a piece of encryption software for your own digital inventory. “Non-financial use cases for blockchains are an important yet underappreciated piece of the next cycle of adoption in web3, NFTs, and blockchains more generally,” miroyato predicts, arguing that even people upset with the financial aspects of web3 can appreciate a use-case that makes their everyday internet experience more secure and private. “I think that consumer-facing applications like Fileverse for peer-to-peer storage will be the main adoption vehicle of web3 in the next years.”
“It is the one familiar and obvious use case that billions of Internet users can be exposed to before the rest of web3.” Who can be against their online data being safely secured and permanently available? For web3 proponents, decentralised file storage is an obvious poster case.
Ekin Genç is a writer based in the UK. He's been writing about all aspects of crypto and web3 since 2020 for the likes of CoinDesk, Decrypt, and VICE. Ekin actively participates in DeFi and NFTs, which informs his writing about an experimental industry that's also deeply experiential. He's a graduate of the University of Oxford and the London School of Economics.
%401x.svg){kind=icon}
