Huge portions of the US have been pummeled with violent storms lately, and, as a result, several datacenters in the region suffered severe damage and power outages. Since some of the biggest datacenters are used to power cloud computing platforms, the outages also led to disruptions in cloud services by providers of all sizes. So while the storms devastated just that region, outages with cloud service providers impacted businesses around the entire globe.
This is yet another rude reminder that we need to design systems and services that can’t be so easily impacted by nature’s wrath or other disasters. We all know we should implement strong disaster recovery architecture, but rarely do companies worry about the continued use of centralized infrastructure in by many cloud service providers. Building cloud services that are dependent on traditional, centralized datacenter resources and used by customers worldwide compounds the market and economic impact of such disasters.
The Internet, which forms the basis for cloud services, has been designed to be inherently resilient to such localized failures because of its decentralized architecture. While major routers in the eastern US may have been down, the rest of the world remained fairly connected and in business because Internet protocols automatically picked up different routes to transmit the traffic. Even look-up services such as DNS have inherent redundancy built into them.
Symform is using these same decentralization principles in the architecture of its Cloud Storage platform. All the raw storage that makes up Symform’s Cloud Storage platform is crowdsourced across the Internet. The network now encompasses users and devices in nearly 150 countries.
Rather than data going to a centralized datacenter, every block of data is encrypted, divided up, encoded and dispersed across hundreds of nodes in the global network in a way where only a subset of the pieces are needed to re-construct the original block. This allows Symform’s Cloud Storage to be highly resilient to such regional disasters as well as offer much better security, speed and economics.
Similar to DNS, Symform does centralize its network orchestration and meta-data service, which we call Cloud Control, to enable efficient look-up and routing of data storage and retrieval. However, because this centralized piece is such a small fraction of the overall storage in the network, Symform is able to keep geo-redundant copies of this meta-data across multiple datacenters. This means that if the primary instance goes down, we are able to fire up the alternate within minutes and get the service back up and available. But even if something happens to Cloud Control, that does not impact your data, which is geo-dispersed.
Cloud service providers need to strive harder to build services that are resilient to localized disasters and truly leverage the decentralized and geo-distributed power of the Internet.