Supported by the National Science Foundation Collaborator: University of Michigan Collaborator: Michigan State University Collaborator: Wayne State University Collaborator: Indiana University
17 July 2020

The OSiRIS team updated the ceph cluster from Nautilus 14.2.9 to Octopus 15.2.4 which is the latest release of Ceph as of August 2020 and it is the fourth release of the Ceph Octopus stable release series.

Major Changes from Nautilus

  • A new deployment tool called cephadm has been introduced that integrates Ceph daemon deployment and management via containers into the orchestration layer. For more information see Cephadm
  • Health alerts can now be muted, either temporarily or permanently.
  • Health alerts are now raised for recent Ceph daemons crashes.

CentOS 7 Dependencies

The dashboard, prometheus, and restful manager modules do not work on CentOS 7 build due to Python 3 module dependencies that are missing in CentOS 7. To resolve the dependency issue and make sure the dashboard, prometheus and restful manager modules function as desinged in Octopus we installed three python3 rpm packages in our repo server by adding them to um-repo. The 3 packages are i) python3-cherrypy ii) python3-routes iii) python3-jwt. Please install these python3 packages in CentOS7 before proceeding with upgrade to Ceph Octopus if you use the ceph-mgr-dashboard, prometheus and restful manager modules.

Upgrade Instructions

  1. Make sure your cluster is stable and healthy (no down or recovering OSDs)
  2. ceph health detail
  3. Set the noout flag for the duration of the upgrade
  4. ceph osd set noout
  5. Upgrade monitors by installing the new packages and restarting the monitor daemons
  6. systemctl restart
    Once all monitors are up, verify that the monitor upgrade is complete by looking for the octopus string in the mon map
    ceph mon dump | grep min_mon_release
    The output should show :
    min_mon_release 15 (octopus)
  7. Upgrade ceph-mgr daemons by installing the new packages and restarting all manager daemons
  8. systemctl restart
    Verify that ceph-mgr deamons are running by executing
    ceph -s 
  9. Upgrade all OSDs by installing the new packages and restarting the ceph-osd daemons on all OSD hosts
  10. systemctl restart
    The progress of the OSD upgrades can be monitored using the either of the commands
    ceph versions 
    ceph osd versions
  11. Upgrade all CephFS MDS daemons

  12. Upgrade all radosgw daemons by upgrading packages and restarting daemons on all hosts
  13. systemctl restart
  14. Complete the upgrade by disallowing pre-Octopus OSDs and enabling all new Octopus-only functionality
  15. ceph osd require-osd-release octopus
  16. If noout was set at the beginning be sure to clear it with
  17. ceph osd unset noout
  18. Verify the cluster is healthy by running
  19. ceph health

For the complete information about upgrading Ceph from Nautilus to Octopus including new features, prerequisties and compatibility deatils , please refer to the Octopus upgrade documentation on the ceph website