3 physical servers (node1-osd, node2-osd, node3-Ceph-client) , model HOST-32H
-2x 2 TB SATA3 + 2 x480GB SSD (no hardware raid)
-10Gbps private network OVH vrack
- 1Cloud OVH VPS https://www.ovh.ie/vps/vps-cloud.xml as monitor( for the moment OVH doesn’t support private networking between dedicated and cloud VPS. So, even is incorrect, for testing purposes, the ceph-monitor will be KVM on top of node2)
sda, sdb - 2X 480GB SSD, CentOS 7
sda1/sdb1 raid1 16G swap
sda2/sdb2 raid1 1G /boot
sda3/sdb3 raid1 300G / ( needed for libvirt OpenShift KVM’s)
sda4/sdb4 noraid, no filesystem, OSD ( optimal size to be established)
sda5/sdb5 noraid, no filesystem, journal disk ( optimal size to be established)
sdc, sdd 2X2 TB SATA3 no raid, no partition, direct OSD ( separate cluster, for performance reasons, don’t want to mix ssd with sata)
NOT INCLUDED IN THIS JOB
-Openshift nodes will be setup later as KVM’s ( on top of node2/3). For testing purposes, in this job we shall setup ceph client with docker installation on KVM1 and also on node3-ceph-client, bare metal.
-Additional nodes will be added later. After more than 5 -10 node, we consider switching to OpenStack, settings up OpenShift on top of it. So, for the moment, we exclude OpenStack, for maintenance burden and performance penalty. by including another layer.
-Validate current startup architecture.
Details: http://docs.ceph.com/docs/master/start/quick-rbd/ recommends “You may use a virtual machine for your ceph-client node, but do not execute the following procedures on the same physical node as your Ceph Storage Cluster nodes (unless you use a VM).”
In this respect, I can’t setup Openshift nodes directly on bare metal ( needs KVM on top of node2/node3) , because Kubernetes persistent volume rbd client could conflict with ceph server libraries.
After architecture validation, I shall do myself the basic setup of the servers and http://docs.ceph.com/docs/master/start/quick-start-preflight/
-Setup ceph clusters
- Test high availability ( shutdown node1... Afterwards, OSD node2, ceph-mon KVM on node2 and node3-ceph-client should continue serving the testing web app)
-Performance analysis of the OVH setup
OHV built-in OS installer, with custom kernel, performs network transfer with 589.9MB/s, but doesn’t have KVM, Ceph included.
Centos 7 installation from USB iso performs network transfer only with 294.9MB/s
Our workaround was to use OVH default installer and then to replace their custom kernel with Centos7 stock kernel, obtaining speeds of 589.9MB/s. It would be nice to identify the OVH installer optimization tricks.
-Benchmark web application (vanilla drupal install) ( setup block devices for web files on SATA drives, and mysql on SSD drives)
-Performance tuning & benchmark ceph tools
- Workflow via Skype share screen with voice ( chat only is excluded) , root access with screen -rx ,with static ip firewall.