A few years ago we started the cgmanager project to address two issues:
- Simplify code in callers.
- Support safe delegation of cgroups to nested container managers.
Historically, advice on where and how to mount cgroups was varied. As a result, programs which wanted to manipulate cgroups had quite a bit of work to do to find cgroup mountpoints for specific controllers and calculate the full paths to use. Cgmanager simplified this code by doing that work for callers. With the ‘cgm’ command, it was also greatly simplified for users. Today, with the advent of the cgroup-lite package, now the cgroupfs-mount package, and systemd, all of which agree on mounting each controller separately under /sys/fs/cgroup/$controller, container manager code can be greatly simplified. This can be evidenced by comparing the older ‘cgfs’ cgroup driver in lxc to the newer ‘cgfsng’ cgroup driver which benefits from assuming the simpler layout (falling back to the more complicated driver if needed).
The other core motivating factor for cgmanager, safer support of cgroup delegation, is now – at last! – deprecated by the availability of cgroup namespaces.
A few lessons?
When starting actual coding for cgmanager, one open question was what kind of communication it should support. We decided on using a dbus interface, implemented using libnih-dbus. I had strong reservations about that. In retrospect, it worked better at the time than I had expected, but had some severe problems. First, performance was pretty horrific. Every dbus connection required 4 round trips to get started. For the simple upstart based systems of the time this was ok, and we kept it from becoming worse by having a cgmanager proxy in a multiply-nested container talk straight to the host cgmanager. However, as Ubuntu switched to systemd, which made very heavy use of cgroups, we started seeing huge performance impacts. It also failed to satisfy the requirements of google folks who otherwise may have been more interested.
Secondly, as Ubuntu switched from systemd to upstart, and upstart – and libnih – became unsupported, this started affecting cgmanager.
So, if cgmanager were still needed today, I would strongly consider rewriting it to have a simple interface over a unix socket. However, as described above, cgmanager has happily become unnecessary. A different sort of cgroup manager may in fact be needed – a modern day ‘libcgroup’ for simple administration of service resources on systems without systemd. However, that is not cgmanager’s role.
So, with that, the cgmanager project is effectively closed. It will continue to be supported by the lxc project (us) on legacy systems. But on the whole, migrating systems to a kernel supporting cgroup namespaces (or at least lxcfs-provided cgroup filesystems) is highly recommended.
PS
By the same token, the cgroup part of lxcfs is also effectively deprecated. I will probably move it under a deprecation build flag soon. lxcfs will continue to be used for procfs virtualization, and likely expanded to support some /sys information.
S3hh's Blog 