Chinese technology giant NetEase recently announced the open-sourcing of a high-performance distributed storage system called Curve, which NetEase claims can perform up to 1.84x of Ceph.
According to NetEase, Curve is positioned to provide a high-performance, low-latency storage dock, based on which users can build storage systems for different application scenarios, including block storage, object storage, and cloud-native databases.
According to the project documentation, Curve has three main features: high performance, high availability and autonomy.
High performance: Curve team referenced some of the industry's most efficient open source storage systems, designed a new architecture to achieve high performance and low latency.
It uses brpc to ensure high performance and low latency for network traffic, and braft-based bran to achieve high performance and low latency under multi-copy consistency.
On the disk IO side, Curve reduces IO collisions with more fine-grained hash of the address space. Increase IO concurrency and use chunkfilepool to reduce IO magnification to maximize Limit the performance of the hardware.
Net has published test data comparing the Curve and Ceph L versions. In a single-volume scenario, Curve's 4K random read IOPS performance is 1.84x that of Ceph and 1.58x that of Ceph's write performance. Curve's 4K random read IOPS performance is 1.84x that of Ceph, and its write performance is 1.58x that of Ceph.
High availability: Curve is designed to have core components that can all tolerate partial instance failures without affecting the availability of the entire cluster.
According to NetEase, Curve's client IO is unaffected by either a single storage failure or system sprawl.
The IO jitter is also very low with common exceptions such as hard disk docking and service process interruptions.
Of course, the failure recovery process will have no significant impact on the upper-level IO.
Autonomy: Curve can be deployed and upgraded with a single click, with minimal manual intervention from operations and maintenance.
It builds comprehensive metrics and alerting system based on open source projects such as bvar, Promethues, and Grafana.
At present, the high-performance block storage system based on Curve has been applied in some of NetEase's core business.
It supports snapshot cloning and recovery, supports QEMU virtual machine and physical machine NBD equipment two ways to mount.
NetEase introduced that the system has been online for more than 400 days, and there have been no data inconsistencies or data loss yet Major failures, with considerable reliability and maturity.
