Last month, I’ve had to opportunity to implement scality ring on Apollo hp server in our infrastructure, today I want to share with you what we have to know about it.
Scality is an object storage startup with a solid part of it’s DNA coming from French horizons (which, obviously, I like a lot as the impenitent French guy I am). They have been around, according to their website, since February 2010 with their current name of, well, Scality.
Scality’s mission is to “bridge the gap between application vendors and industry standard hardware providers to meet cost-effective storage scale, durability, cloud and performance requirements”. So how do they plan to achieve this?
First of all, Scality is a software-only company, with their RING software able to run on any commodity x86 hardware platform. It may make sense to consult with Scality to ensure that the selected hardware platform will deliver up to the expectations. In other words: commodity hardware doesn’t means cheap and/or underperforming (but you already knew this).
Unlike other SDS solutions, the RING operating system uses a specific method to communicate between nodes, which is different to the master/slave architecture. RING uses a peer-to-peer mesh communication model based on the CHORD protocol. No matter how much you scale your solution, each node is in contact with a specific set of other nodes, and the idea for this mesh architecture is to minimise the amount of traffic and to locate objects very quickly.
Onion RINGs Solution :
The Scality RING can be looked at like an onion: it is made of three layers, of which the outer one manages connectivity protocols (a.k.a. Connectors), the middle one takes care of data routing, high availability, self-healing and data reliability, while the inner one provides data placement (distributed storage) and abstraction between the RING and the actual physical storage.
The RING OS, now in version 6.0, offers file (NFS, SMB, Fuse) and object (S3, Native Scality REST API) capacities. It is also useable with OpenStack through Cinder, Glance and Swift. These access methods are defined at the Connector level (upper layer). This level provides a persistent shared cache and shared namespace across all connectors.
The system leverages SSD to store metadata in a logical ring, and HDD spindles to store actual objects in yet another logical ring. Most of the operations are performed at the metadata level, the goal being to touch the data residing on HDDs at the very last moment.
If we look at the middle layer, data protection is configurable up to the individual object level through to policies, with up to 6 replicas per object. In terms of data durability, I recommend to watch Giorgio Regni’s presentation at Storage Field Day 11. Finally, the RING is self-healing with rebuild processes & operations happening in the background. To avoid rebuilt performance penalties, data is served temporarily from a healthy node until the rebuild has completed.
The bottom layer, finally, takes care of distributing storage between nodes and of low level I/O operations.
To Note :
With the choice of being a pure software player in the Object Storage market, Scality is eliminating at the very least the tedious issue of maintaining a hardware supply chain with all its inconvenients: sourcing, lifecycle refreshes, logistics & customs, inventory, RMAs.
This software strategy is likely to pay off because in the object storage area commodity hardware is more or less the norm. Furthermore, the primary requirement of an object store is not performance, but data reliability (shall we add durability/retrievability) and the capacity to ingest huge amounts of data, at low prices and on standard 10 GbE network interfaces. A smart move is their open-source Scality S3 Server: it allows to get access to Scality’s S3 implementation in a docker container environment and is a good entry point to test the features of Scality.
In terms of differentiators, their Active Directory and AWS IAM integration is an important aspect. Why Active Directory and IAM matter, since we are here to talk storage? Building a system with integrated Identity and Access Management makes the solution work better in enterprise environments where AD integration is often a requirement.
In my opinion, Scality will face the fiercest competition from the other software players in the object storage space. I am tempted to say that appliance or hardware-backed object storage providers are likely to lose ground in this area. Software capabilities is where the money is made, and in object storage we can expect bring-your-own-hardware to become the standard.
With steady funding and a “more than just best friends” partnership with HP, Scality seems to be well equipped for success -and potentially also leadership- in the object storage world.