Ceph is a Software-defined storage solution. Probably the most reliable and widely used, with the greatest scaling potential. Fully decentralized with no single point of failure. It can provide both block and object storage, as well as a filesystem.
Ceph can provide remote storage to several platforms, like Kubernetes, Openstack, and Incus. The key benefits of Ceph are its scalability and reliability. Although it can provide decent performance in large clusters, it generally offers less performance than solutions like Linstor.
A typical Ceph cluster is composed of the following components:
- Managers monitor the cluster and perform background tasks that require coordination of members
- Metadata servers store the metadata of the CephFS file system
- Object Storage Daemons (OSD) are the daemons that handle the actual data storage. Each OSD is responsible for managing a disk on the machine, so we usually have multiple OSDs for a given server, one for each disk
- Monitors keep track of the existing placement groups and their location across the cluster. Clients reach to the monitors to get a map of which OSDs replicate the data for a given storage pool
Ceph was originally designed as a distributed filesystem, and a key point of its architecture is that metadata and data can be scaled independently.
Ceph uses object storage at its core. It is capable of provide block devices and filesystems to clients by creating abstrations on top of the object storage layer. This is done via the CRUSH algorithm, which makes Ceph able to distribute, objects between OSDs and retrieve them without the need for a centralized index.
