How to add glusterfs to ovirt (rhev)?
I have 10 node trusted storage pool. I created a distributed replicated volume on it called emre-vol1 with replica count 2
[root@gluster01 ~]# gluster volume status emre-vol1 Status of volume: emre-vol1 Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick gluster01.peakox.com:/gluster/bric k_ssd/brick 49152 0 Y 172197 Brick gluster02.peakox.com:/gluster/bric k_ssd/brick 49152 0 Y 103826 Brick gluster03.peakox.com:/gluster/bric k_ssd/brick 49152 0 Y 107674 Brick gluster04.peakox.com:/gluster/bric k_ssd/brick 49152 0 Y 190228 Brick gluster05.peakox.com:/gluster/bric k_ssd/brick 49152 0 Y 106392 Brick gluster06.peakox.com:/gluster/bric k_ssd/brick 49152 0 Y 189540 Brick gluster07.peakox.com:/gluster/bric k_ssd/brick 49152 0 Y 183288 Brick gluster08.peakox.com:/gluster/bric k_ssd/brick 49152 0 Y 171855 Brick gluster09.peakox.com:/gluster/bric k_ssd/brick 49152 0 Y 186912 Brick gluster10.peakox.com:/gluster/bric k_ssd/brick 49152 0 Y 183638 NFS Server on localhost 2049 0 Y 172217 Self-heal Daemon on localhost N/A N/A Y 172225 NFS Server on gluster16.peakox.com 2049 0 Y 182317 Self-heal Daemon on gluster16.peakox.com N/A N/A Y 182325 NFS Server on gluster14.peakox.com 2049 0 Y 178787 Self-heal Daemon on gluster14.peakox.com N/A N/A Y 178795 NFS Server on gluster07.peakox.com 2049 0 Y 183308 Self-heal Daemon on gluster07.peakox.com N/A N/A Y 183316 NFS Server on gluster10.peakox.com 2049 0 Y 183658 Self-heal Daemon on gluster10.peakox.com N/A N/A Y 183666 NFS Server on gluster09.peakox.com 2049 0 Y 186932 Self-heal Daemon on gluster09.peakox.com N/A N/A Y 186940 NFS Server on gluster13.peakox.com 2049 0 Y 184300 Self-heal Daemon on gluster13.peakox.com N/A N/A Y 184308 NFS Server on gluster12.peakox.com 2049 0 Y 180419 Self-heal Daemon on gluster12.peakox.com N/A N/A Y 180427 NFS Server on gluster15.peakox.com 2049 0 Y 179405 Self-heal Daemon on gluster15.peakox.com N/A N/A Y 179413 NFS Server on gluster11.peakox.com 2049 0 Y 180841 Self-heal Daemon on gluster11.peakox.com N/A N/A Y 180849 NFS Server on gluster08.peakox.com 2049 0 Y 171875 Self-heal Daemon on gluster08.peakox.com N/A N/A Y 171883 NFS Server on gluster04.peakox.com 2049 0 Y 190248 Self-heal Daemon on gluster04.peakox.com N/A N/A Y 190256 NFS Server on gluster02.peakox.com 2049 0 Y 103846 Self-heal Daemon on gluster02.peakox.com N/A N/A Y 103854 NFS Server on gluster05.peakox.com 2049 0 Y 106412 Self-heal Daemon on gluster05.peakox.com N/A N/A Y 106420 NFS Server on gluster03.peakox.com 2049 0 Y 107694 Self-heal Daemon on gluster03.peakox.com N/A N/A Y 107702 NFS Server on gluster06.peakox.com 2049 0 Y 189560 Self-heal Daemon on gluster06.peakox.com N/A N/A Y 189568 NFS Server on gluster-tb1.peakox.com 2049 0 Y 12047 Self-heal Daemon on gluster-tb1.peakox.com N/A N/A Y 12055 |
When I was trying to create new domain in oVirt environment I was first getting errors below
“Unsupported Gluster Volume Replica Count”
Which made surprised since oVirt supports replica 1 and replica 3. I wanted to setup with replica count 2 and started looking solutions on how to get oVirt
to accept replica count 2.
On a virtualization node you can add following lines to vdsm.conf and then oVirt will be happy to use a glusterfs volume with replica 2.
Under /etc/vdsm/vdsm.conf add
[gluster] allowed_replica_counts = 1,2,3 |
After this setting added error above was resolved and I can add replica count 2 system to oVirt as my storage. This did ended up with another error on oVirt.
“Permission settings on the specified path do not allow access to the storage. Verify permission settings on the specified storage path.”
Doing some googling found that some volume setting needed to be done.
[root@gluster01 ~]# gluster volume set emre-vol1 group virt volume set: success [root@gluster01 ~]# gluster volume set emre-vol1 storage.owner-uid 36 volume set: success [root@gluster01 ~]# gluster volume set emre-vol1 storage.owner-gid 36 volume set: success |
36 is uid of vdsm and gid of kvm.
After those I was able to create storage for gluster and added my VM to use glusterfs for VM disk storage.