Every Linux admin knows LVM, the Logical Volume Manager. This abstraction over storage can do software RAID, snapshots, resizing of logical volumes, and other magic. In this short note I summarise my recent experience with two advanced and interesting topics: cloning LVs and caching an LV on an SSD.

If PV, VG, and LV are unfamiliar, start with an introductory LVM tutorial. Briefly:

  • VG, or volume group, groups physical devices (PV, physical volumes).
  • Inside a VG, logical volumes (LVs) exist.
  • LVM can then create configurations such as mirrors and striping, create and remove LVs and snapshots, resize LVs, and so on.

This note returns to two interesting LVM operations I had recently needed.

Task 1: Cloning an LV

The problem was simple. I had a 500 GB SSD in a laptop and wanted to reinstall it, but first I wanted a complete backup of that SSD stored on a 500 GB hard drive. Several variants were possible.

The most low-level option was a raw bit copy from SSD to HDD. I rejected it because it felt too low-level.

The most high-level option was creating a filesystem on the HDD and copying files, using helpers such as tar or preferably cpio. I rejected that too, because it felt too high-level.

Both variants also have trouble creating a consistent copy while the system is heavily used. Snapshots could solve that, but that already adds work.

The simple solution, found in a discussion under this post, was:

  • Add the HDD to the same VG as the SSD.
  • Mirror all LVs that need to be copied from SSD to HDD. The mirror will be placed on the new PV if the VG originally had only one.
  • Split the mirrored LV into two independent LVs.
  • Split the VG so that the HDD becomes a separate VG.

The commands are:

vgextend vg /dev/sdX
lvconvert -m 1vg/lv1
lvconvert -m 1vg/lv2 # optional
...
lvconvert --splitmirrors 1 -n lv1_clone vg/lv1
lvconvert --splitmirrors 1 -n lv2_clone vg/lv2
...
vgsplit vg vg_clone /dev/sdX

Task 2: Caching an HDD on an SSD

The second problem is also simple: we have a machine with a hard drive, or several drives in a nice RAID, plus one or more SSDs, and we want to use the SSD to speed up operations with spinning storage.

Several solutions exist, such as bcache or dm-cache. Why not use LVM, which in recent versions supports using the kernel dm_cache module for caching? The inspiration was again a blog post.

The steps are:

  • Add the SSD to the same VG as the HDDs.
  • Create two LVs for cache: one for metadata and one for data. The metadata LV can be about 1000 times smaller than the data LV.
  • Convert those two LVs into a cache pool.
  • Attach the cache pool to an LV.

The advantage is that the cache can be created, attached, and removed while running. The disadvantage is that each LV has its own cache pool, so if /home and / are separate, each must be cached separately.

Again, the command sequence is only illustrative. The example uses /dev/sdX to cache the LV vg/home:

vgextend vg /dev/sdX # SSD cache

# create metadata cache LV on sdX
lvcreate -L 1G -n meta vg /dev/sdX
# create data cache LV on sdX
lvcreate -L 229G -n data vg /dev/sdX

# convert to cache pool
lvconvert --type cache-pool --poolmetadata vg/meta vg/data

# attach cache pool to LV vg/home
lvconvert --type cache --cachepool vg/data vg/home

Removing the SSD Cache

The lvmcache manual page is helpful here. Removing cache without removing the LV is mentioned directly: remove the vg/data LV.

lvremove vg/data

SSD Cache Statistics

Once the cache exists, simple statistics can be obtained with dmsetup status:

dmsetup status /dev/mapper/vg-home
# prints:
0 1677721600 cache 8 4160/17408 128 1306202/2097152 87122544 9943501 25634508 20958276 0 1306202 0 1 writeback 2 migration_threshold 2048 mq 10 random_threshold 4 sequential_threshold 512 discard_promote_adjustment 1 read_promote_adjustment 4 write_promote_adjustment 8

The meaning of the fields can be searched for, but it is better to look directly into the documentation:

<metadata block size> <#used metadata blocks>/<#total metadata blocks> <cache block size> <#used cache blocks>/<#total cache blocks> <#read hits> <#read misses> <#write hits> <#write misses> <#demotions> <#promotions> <#dirty> <#features> <features>* <#core args> <core args>* <policy name> <#policy args> <policy args>*

The output shows that my cache was 62% full and hit 89% of reads: read_hits / (read_hits+read_misses). For writes it hit 55%.

Notes

According to the lvmcache documentation, a cache pool can be created with one command. It is also possible to create a mirrored cache pool for higher robustness. Be careful: an LV with attached cache cannot be attached separately.

Conclusion

If you have tips for similar advanced LVM techniques, share them in the discussion under the article. I will also be glad for corrections and additions.