[Feature request] Use `ceph-volume` to manage OSDs

[spacekitteh]

So, I'm converting my existing Ceph cluster to use Skyflake. One issue that is stopping me from using it to deploy OSDs with, however, is that I require full disk encryption for each OSD.

I deployed my OSDs via ceph-volume create, with the --dmcrypt flag. This sets up each OSD to be encrypted via LUKS. At boot, ceph-volume activate grabs the LUKS keys from a Ceph monitor, passes them to dm-crypt, and then proceeds as normal in activating the OSDs.

Existing OSDs can be easily converted to use ceph-volume; so is it possible to change the OSD management logic to use ceph-volume instead? I suspect it would greatly simplify the implementation.

[astro]

IIRC I didn't want that because it forced an extra LVM layer, duplicating what I was doing myself already.

Caution: don't put production data into Ceph on NixOS before there are good and safe ways to upgrade that cluster to future Ceph versions.

[spacekitteh]

ceph-volume has two modes; one which uses LVM, the other doesn't :)

Caution: don't put production data into Ceph on NixOS before there are good and safe ways to upgrade that cluster to future Ceph versions.

I've already done an upgrade cycle with it! From quincy to reef. It was relatively painless; just upgraded one monitor at a time, then the OSDs :)

[spacekitteh]

Also, recall that there is a services.ceph.<daemon>.package option, thanks to @SuperSandro2000 :)