Install Ceph cluster on CentOS8 virtual machines. The following steps will install a ceph deployer node , a ceph monitor node and three ceph osd nodes
Networks
ceph-adm
CPU: 1
Memory: 1GB
Nics (virtio)
Disk: 15GB
ceph-mon
CPU: 2
Memory: 2GB
Nics (virtio)
Disk: 15GB
ceph-osd1/2/3
CPU: 2
Memory: 4GB
Nics (virtio)
Disks
ceph-adm
CentOS 8 default install
Set the hostname
Network setup
NIC1 / ifcfg-ens3
TYPE=Ethernet NAME=ens3 DEVICE=ens3 ONBOOT=yes IPADDR=192.168.126.248 NETMASK=255.255.255.0
NIC2 / ifcfg-ens4
TYPE=Ethernet NAME=ens4 DEVICE=ens4 ONBOOT=yes IPADDR=192.168.122.3 NETMASK=255.255.255.0 GATEWAY=192.168.122.1 DNS1=192.168.122.1
Restart network
Edit /etc/hosts to add
192.168.126.248 ceph-adm 192.168.126.249 ceph-mon 192.168.126.250 ceph-osd1 192.168.126.251 ceph-osd2 192.168.126.252 ceph-osd3
Edit /root/.ssh/config to add
Host ceph-mon Hostname ceph-mon User cephadm Host ceph-osd1 Hostname ceph-osd1 User cephadm Host ceph-osd2 Hostname ceph-osd2 User cephadm Host ceph-osd3 Hostname ceph-osd3 User cephadm
Install ceph repo
Set timezone
Add user cephadm
Disable SELINUX
After all ceph nodes are installed (refer bottom sections) , as root
ssh-copy-id cephadm@ceph-mon
ssh-copy-id cephadm@ceph-osd1
ssh-copy-id cephadm@ceph-osd2
ssh-copy-id cephadm@ceph-osd3
EPEL install
Update
Change to user ceph-adm
Install required packages
Clone ceph-ansible and configure
Prepare ceph config yml files
dummy: ceph_release_num: 15 cluster: ceph mon_group_name: mons osd_group_name: osds rgw_group_name: rgws mds_group_name: mdss nfs_group_name: nfss rbdmirror_group_name: rbdmirrors client_group_name: clients iscsi_gw_group_name: iscsigws mgr_group_name: mgrs rgwloadbalancer_group_name: rgwloadbalancers monitoring_group_name: monitoring grafana_server_group_name: grafana-server configure_firewall: True ntp_service_enabled: true ntp_daemon_type: chronyd ceph_origin: repository ceph_repository: community ceph_stable_release: octopus ceph_repository_type: cdn monitor_interface: ens3 radosgw_interface: ens3 dashboard_enabled: True dashboard_protocol: http dashboard_admin_user: admin dashboard_admin_password: 123456 grafana_admin_user: admin grafana_admin_password: 123456
cp group_vars/osds.yml.sample group_vars/osds.yml and edit
dummy: devices:
cp site.yml.sample site.yml
Edit hosts
# Ceph admin user for SSH and Sudo
[all:vars]
ansible_ssh_user=cephadm
ansible_become=true
ansible_become_method=sudo
ansible_become_user=root
# Ceph Monitor Nodes
[mons]
ceph-mon
# MDS Nodes
[mdss]
ceph-mon
# RGW
[rgws]
ceph-mon
# Manager Daemon Nodes
[mgrs]
ceph-mon
# set OSD (Object Storage Daemon) Node
[osds]
ceph-osd1
ceph-osd2
ceph-osd3
# Grafana server
[grafana-server]
ceph-mon
Deploy the cluster
sudo ansible-playbook -i hosts site.yml
Status should look like
INSTALLER STATUS *************************************************************************
Install Ceph Monitor : Complete (0:00:30)
Install Ceph Manager : Complete (0:00:34)
Install Ceph OSD : Complete (0:01:03)
Install Ceph MDS : Complete (0:00:37)
Install Ceph RGW : Complete (0:00:27)
Verify the cluster
On ceph-mon
Ceph dashboard
on ceph-mon
access URL
Ceph-mon
CentOS8 default install
Hostname
Time
User
Update
OS update
Networks
ifcfg-ens3
TYPE=Ethernet NAME=ens3 DEVICE=ens3 ONBOOT=yes IPADDR=192.168.126.249 NETMASK=255.255.255.0
Ifcfg-ens4
TYPE=Ethernet NAME=ens4 DEVICE=ens4 ONBOOT=yes IPADDR=192.168.122.4 NETMASK=255.255.255.0 GATEWAY=192.168.122.1 DNS1=192.168.122.1
Restart network
Edit /etc/hosts to add
192.168.126.248 ceph-adm 192.168.126.249 ceph-mon 192.168.126.250 ceph-osd1 192.168.126.251 ceph-osd2 192.168.126.252 ceph-osd3
Ceph-ods1
CentOS 8 default install
Hostname
Time
User
Update
OS update
Networks
ifcfg-ens3
TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=ens3 DEVICE=ens3 ONBOOT=yes IPADDR=192.168.126.250 NETMASK=255.255.255.0
ifcfg-ens
TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=ens4 DEVICE=ens4 ONBOOT=yes IPADDR=192.168.122.5 NETMASK=255.255.255.0 GATEWAY=192.168.122.1 DNS=192.168.122.1
Restart network
Edit /etc/hosts to add
192.168.126.248 ceph-adm 192.168.126.249 ceph-mon 192.168.126.250 ceph-osd1 192.168.126.251 ceph-osd2 192.168.126.252 ceph-osd3
Ceph-ods2
CentOS 8 default install
Hostname
Time
User
Update
OS update
Networks
ifcfg-ens3
TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=ens3 DEVICE=ens3 ONBOOT=yes IPADDR=192.168.126.251 NETMASK=255.255.255.0
Ifcfg-ens4
TYPE=Ethernet BOOTPROTO=none NAME=ens4 DEVICE=ens4 ONBOOT=yes IPADDR=192.168.122.6 NETMASK=255.255.255.0 GATEWAY=192.168.122.1 DNS1=192.168.122.1
Restart network
Edit /etc/hosts to add
192.168.126.248 ceph-adm 192.168.126.249 ceph-mon 192.168.126.250 ceph-osd1 192.168.126.251 ceph-osd2 192.168.126.252 ceph-osd3
Ceph-ods3
CentOS 8 default install
Hostname
Time
User
Update
OS update
Networks
ifcfg-ens3
TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=ens3 DEVICE=ens3 ONBOOT=yes IPADDR=192.168.126.252 NETMASK=255.255.255.0
Ifcfg-ens4
TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=none NAME=ens4 DEVICE=ens4 ONBOOT=yes IPADDR=192.168.122.7 NETMASK=255.255.255.0 GATEWAY=192.168.122.1 DNS1=192.168.122.1
Restart network
Edit /etc/hosts to add
192.168.126.248 ceph-adm 192.168.126.249 ceph-mon 192.168.126.250 ceph-osd1 192.168.126.251 ceph-osd2 192.168.126.252 ceph-osd3