Ferro's Gitbook
  • README
  • DevOps
    • Grafana_Cloud
  • OpenWrt
    • DHCP_DNS
    • GLiNet
    • boot
    • captive_portal
    • luci
    • mwan3
    • radius
    • theme
    • wireless
  • apps
    • web
  • BSD
    • Mac
  • Cloud
    • aws
    • azure
    • cf
    • gcp
    • github
    • ibm_bluemix
    • Pricing
  • container
    • docker
    • Kubernetes
    • podman
  • db
    • InfluxDB
    • loki
    • MySQL & MariaDB
    • Oracle
    • PostgreSQL
  • dev
    • AHK
    • BI
    • LBS
    • ML
    • android
    • editor
    • flutter_web
    • git
    • go
    • HTML5/BS
    • j2ee
    • js
    • js_grid
    • js_vue
    • jupyter
    • ocaml
    • powershell
    • py
    • py_GUI
    • Django
    • shell
    • snippets
    • uni
    • vba
    • wechat.zh
    • wechat_mp.zh
  • elec
    • 3D Printing
    • AC
    • MOSFET
    • battery
    • boost
    • bulk
    • metal
    • simulator
  • hw
    • GPU
    • PCI
    • arduino
    • Bluetooth
    • ent
    • Pinout
    • x86_AMD
    • x86_intel
  • linux
    • Test System
    • X
    • arch
    • fs
    • kernel
    • Memory
    • nw
    • Linux Services
    • Systemd
    • text
  • ms
    • vscode
    • windows
    • wsl
  • multimedia
    • Blender
    • audio
    • blender
    • graphics
    • home
  • nw
    • L3
    • L3_IPv6
    • SDN
    • VPN
    • dns
    • hw
    • Low Level
    • mikrotik
    • mwan
    • Openflow
    • OVS
    • pfsense
    • ppp
    • proxy
    • tsocks
    • pxe
    • Security
    • TCP
  • phone
    • Mi
    • android
  • Storage(SW)
  • vt
    • Intel GVT-g
    • PVE
    • QEMU
    • VDI
    • hyper-v
    • kube
    • libvirt
    • OpenStack
  • Web
    • IBM_MQ
    • IBM_Websphere
    • SSL
    • Apache/IBM_IHS
    • blockchain
    • caddy
    • j2ee
    • nginx
    • static_site
Powered by GitBook
On this page
  • FUSE
  • Compare
  • Gluster - C
  • Quickstart
  • Types of Volumes
  • RDMA(Remote direct memory access)
  • Snapshots
  • on ZFS
  • Ceph - C++
  • Setup
  • Docker
  • CACHE TIERING
  • Snapshot
  • UI - inkscope
  • Rook - Go
  • IPFS
  • Backup
  • duplicati

Was this helpful?

Edit on Git

Storage(SW)

PreviousandroidNextvt

Last updated 4 years ago

Was this helpful?

FUSE

Compare

HDFS

iRODS

Ceph

GlusterFS

Lustre

Arch

Central

Central

Distributed

Decentral

Central

Naming

Index

Database

CRUSH

EHA

Index

API

CLI, FUSE

CLI, FUSE

FUSE, mount

FUSE, mount

FUSE

REST

REST

REST

Fault-detect

Fully connect.

P2P

Fully connect.

Detected

Manually

sys-avail

No-failover

No-failover

High

High

Failover

data-aval

Replication

Replication

Replication

RAID-like

No

Placement

Auto

Manual

Auto

Manual

No

Replication

Async.

Sync.

Sync.

Sync.

RAID-like

Cache-cons

WORM, lease

Lock

Lock

No

Lock

Load-bal

Auto

Manual

Manual

Manual

No

Gluster - C

Quickstart

gluster peer probe server2  # from server1
gluster peer probe server1  # from server2
# Once this pool has been established, it must be probed from the pool.
gluster peer status

mkdir -p /data/brick1/gv0   # both server1 and server2
gluster volume create gv0 replica 2 server1:/data/brick1/gv0 server2:/data/brick1/gv0 # any server
gluster volume start gv0
gluster volume info

Types of Volumes

gluster volume create test-volume \
    server1:/exp1 server2:/exp2 server3:/exp3 server4:/exp4 # Distributed
    replica 2 transport tcp server1:/exp1 server2:/exp2     # Replicated, w/ data redundancy
    replica 2 transport tcp server1:/exp1 ... server4:/exp4 # R->D
    stripe 2 transport tcp server1:/exp1 server2:/exp2      # Striped
    stripe 4 transport tcp server1:/exp1 ... server8:/exp8  # S->D
    stripe 2 replica 2 transport tcp server1:/exp1 ... server4:/exp4   # R->S (*)
    stripe 2 replica 2 transport tcp server1:/exp1 ... server8:/exp8   # R->S->D (*)
   (*) only for Map Reduce workloads
    <Dispersed Volume Usable size> = <Brick size> * (#Bricks - Redundancy)
    bricks#(>=3) > 2 * redundancy

    [disperse [<count>]] [redundancy <count, default=auto>]

    disperse 4 server{1..4}:/bricks/test-volume # Dispersed
    disperse 3 server1:/brick{1..3} # Dispersed             (#)
    disperse 3 server1:/brick{1..6} # Distributed Dispersed (#)
    (#) on the same server, add `force`
    disperse-data 8 redundancy 4 transport tcp 
    server1:/exp/brick1 server1:/exp/brick2 server1:/exp/brick3 server1:/exp/brick4 
    server2:/exp/brick1 server2:/exp/brick2 server2:/exp/brick3 server2:/exp/brick4 
    server3:/exp/brick1 server3:/exp/brick2 server3:/exp/brick3 server3:/exp/brick4 
    server1:/exp/brick5 server1:/exp/brick6 server1:/exp/brick7 server1:/exp/brick8 
    server2:/exp/brick5 server2:/exp/brick6 server2:/exp/brick7 server2:/exp/brick8 
    server3:/exp/brick5 server3:/exp/brick6 server3:/exp/brick7 server3:/exp/brick8 
    server1:/exp/brick9 server1:/exp/brick10 server1:/exp/brick11 server1:/exp/brick12 
    server2:/exp/brick9 server2:/exp/brick10 server2:/exp/brick11 server2:/exp/brick12 
    server3:/exp/brick9 server3:/exp/brick10 server3:/exp/brick11 server3:/exp/brick12
gluster volume info test-volume
gluster volume start test-volume

RDMA(Remote direct memory access)

As of now only FUSE client and gNFS server would support RDMA transport.

Snapshots

on ZFS

Ceph - C++

Ceph uniquely delivers object, block, and file storage in one unified system. A Ceph Storage Cluster consists of two types of daemons:

  • Ceph Monitor: maintains a master copy of the cluster map

  • Ceph OSD Daemon: checks its own state and the state of other OSDs and reports back to monitors.

Setup

STEP 1: METADATA SERVER
STEP 2: MOUNT CEPHFS

Docker

CACHE TIERING

Snapshot

UI - inkscope

Rook - Go

IPFS

combines Kademlia + BitTorrent + Git
mountable filesystem (via FUSE)
http://ipfs.io/<path>

Backup

duplicati

CEPH: Provides a robust, highly scalable block, object, and filesystem storage platform for modern workloads, like cloud infrastructure and data analytics. Consistently ranked as preferred storage option by OpenStack® users. Gluster: Provides a scalable, reliable, and cost-effective data management platform, streamlining file and object access across physical, virtual, and cloud environments.

software defined distributed storage that can scale to several petabytes. It provides interfaces for object, block and file storage.

GlusterFS volume snapshot feature is based on thinly provisioned LVM snapshot.

Ceph supports many higher level interfaces, including QEMU, libvirt, OpenStack and CloudStack. Ceph supports the ability to create many copy-on-write (COW) clones of a block device shapshot. Snapshot layering enables Ceph block device clients to create images very quickly.

(with screenshots)

File, Block, and Object Storage Services for your Cloud-Native Environment

Store securely encrypted backups in the cloud! Amazon S3, OneDrive, Google Drive, Rackspace Cloud Files, HubiC, Backblaze (B2), Amazon Cloud Drive (AmzCD), Swift / OpenStack, WebDAV, SSH (SFTP), FTP, and more!

https://www.redhat.com/en/technologies/storage
http://cecs.wright.edu/~pmateti/Courses/7370/Lectures/DistFileSys/distributed-fs.html
http://www.youritgoeslinux.com/impl/storage/glustervsceph
http://docs.gluster.org/en/latest/Quick-Start-Guide/Architecture/
http://docs.gluster.org/en/latest/Quick-Start-Guide/Quickstart/
https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3/html/administration_guide/creating_distributed_striped_replicated_volumes
https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.1/html/administration_guide/chap-red_hat_storage_volumes-creating_dispersed_volumes_1
https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.1/html/administration_guide/chap-recommended-configuration_dispersed
http://docs.gluster.org/en/latest/Administrator Guide/RDMA Transport/
http://docs.gluster.org/en/latest/Administrator Guide/Managing Snapshots/
http://docs.gluster.org/en/latest/Administrator Guide/Gluster On ZFS/
http://docs.ceph.com/docs/master/cephfs/
http://hub.docker.com/r/ceph/
http://docs.ceph.com/docs/master/rbd/rbd-snapshot/
https://github.com/ipfs/ipfs
https://github.com/duplicati/duplicati
https://www.duplicati.com/screenshots/
FUSE
Compare
Gluster - C
Quickstart
Types of Volumes
RDMA(Remote direct memory access)
Snapshots
on ZFS
Ceph - C++
Setup
Docker
CACHE TIERING
Snapshot
UI - inkscope
Rook - Go
IPFS
Backup
duplicati
https://github.com/inkscope/inkscope
https://rook.github.io/docs/rook/master/kubernetes.html