Johtizen

software engineering craftmanship blog
Home / About

View on GitHub
3 August 2022

Keep your C++ dependencies up-to-date with Renovate & CPM

by JohT

Looking for new versions of libraries and updating them can be a tedious task, especially if you have to deal with multiple dependencies and many repositories. Renovate is a tool that can scan your dependencies and update them to the latest version. But can this also be utilized in a C++ project with CPM.cmake?

Table of Contents

  1. Prerequisites
  2. External Dependencies
    1. What is CPM.cmake [2]?
    2. Why not just use Git Submodules [11]?
    3. How to use CPM.cmake?
    4. Why CPM.cmake?
    5. CPM Package Lock [15]
  3. Version Updates
    1. What is Renovate [1]?
    2. How to use Renovate?
  4. How to use Renovate with CPM.cmake?
    1. Renovate Regex Manager for CPM Package Lock VERSION
    2. Renovate Regex Manager for CPM Package Lock GIT_TAG
    3. Renovate Regex Manager for CPM Package Lock GIT_TAG commit SHA
    4. Renovate Regex Manager for GitHub download links in CMake
    5. Renovate Regex Manager for GitHub downloads with a version variable in CMake
    6. Real world example
  5. Summary
  6. References

Prerequisites

External Dependencies

What is CPM.cmake [2]?

cpm is …

CMake’s missing package manager. A small CMake script for setup-free, cross-platform, reproducible dependency management.

A key difference to other package managers is:

Any downloadable project or resource can be added as a version-controlled dependency though CPM.

Why not just use Git Submodules [11]?

A common practice to include external sources in C++ project is to use Git Submodules. This is a good solution for many projects:

But there are also some drawbacks:

In the end it is a choice of what suits the project best. Automated Dependency Updates for Git Submodules [12] goes further into details on how to setup automatic update for Git Submodules. Otherwise continue reading if you are interested in what CPM.cmake can do.

How to use CPM.cmake?

These few lines in CMake’s CMakeLists.txt are all you need to setup CPM.cmake:

set(CPM_DOWNLOAD_VERSION 0.27.2) 
set(CPM_DOWNLOAD_LOCATION "${CMAKE_BINARY_DIR}/cmake/CPM_${CPM_DOWNLOAD_VERSION}.cmake")

if(NOT (EXISTS ${CPM_DOWNLOAD_LOCATION}))
    message(STATUS "Downloading CPM.cmake")
    file(DOWNLOAD https://github.com/TheLartians/CPM.cmake/releases/download/v${CPM_DOWNLOAD_VERSION}/CPM.cmake ${CPM_DOWNLOAD_LOCATION})
endif()

include(${CPM_DOWNLOAD_LOCATION})

As an example, adding Catch2 for unit testing looks like this (shorthand syntax):

CPMAddPackage("gh:catchorg/Catch2@3.1.0")

For more information have a look at this article: CPM: An Awesome Dependency Manager for C++ with CMake [13]

Why CPM.cmake?

CPM.cmake can be used like Git Submodules to include sources from other Git Repositories. It can furthermore be used to download already built packages and download sources that are not available in Git. It can work with git tags and commit references. CPM.cmake wraps CMake’s FetchContent [14] to simplify downloading external dependencies. It doesn’t depend on a centralized package repository nor does it require any metadata to support CPM.cmake.

CPM Package Lock [15]

CPM.cmake provides a way to list all dependencies in one package-lock.cmake file. These can then easily be referenced in CMakeLists.txt by their name where needed. This mimics other package managers and has several advantages. It is easier to read, simplifies the process of updating dependencies and integrates much better with other tools.

The resulting package-lock.cmake may then look for example like this:

# CPM Package Lock
# This file should be committed to version control

# Catch2
CPMDeclarePackage(Catch2
  VERSION 3.1.0
  GITHUB_REPOSITORY catchorg/Catch2
  EXCLUDE_FROM_ALL YES
)

The library is then referenced by name within CMakeLists.txt like this:

CPMGetPackage(Catch2)

Version Updates

It is a tedious and time consuming task to update dependencies to their latest version manually. Tools like Renovate automate this process by scanning your dependencies and updating them to the latest version. Staying up-to-date is essential when it comes to security updates and bug fixes. It is also easier to continuously update in small steps. Last but not least an update might come in handy when there are useful new features.

What is Renovate [1]?

Renovate is a …

universal dependency update tool that fits into your workflows

… and let you …

get automated Pull Requests to update your dependencies

How to use Renovate?

If your repository is on GitHub [6], Renovate GitHub App [4] is free to install. Installing and onboarding Renovate into repositories [5] gives you more information about that.

The configuration for Renovate is usually found in the file renovate.json. For further options see Renovate Configuration Options [7]. The most basic configuration looks like this:

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": [
    "config:base"
  ]
}

How to use Renovate with CPM.cmake?

CPM.cmake [2] isn’t supported by Renovate yet (August 2022). Fortunately, there is a workaround for that. As described in Custom Manager Support using Regex [8], Regular Expressions [10] can be used to identify dependencies and then find and replace their versions.

ⓘ Note that it is recommended to use a tool like regex101.com to get immediate feedback when developing Regular Expressions.

Renovate Regex Manager for CPM Package Lock VERSION

The following package-lock.cmake entry shows a dependency to a GitHub repository with a specific version:

CPMDeclarePackage(Catch2
  VERSION 3.1.0
  GITHUB_REPOSITORY catchorg/Catch2
  EXCLUDE_FROM_ALL YES
)

Use this renovate.json to update the dependency’s version with a pull request whenever there is a new release:

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": [
    "config:base",
  ],
  "regexManagers": [
    {
      "fileMatch": ["(^|/)package-lock\\.cmake$"],
      "matchStrings": [
        "CPMDeclarePackage\\s*\\(\\s*\\w+\\s*.*VERSION\\s+\"?(?<currentValue>.*?)\"?\\s+GITHUB_REPOSITORY\\s+\"?(?<depName>.*?)\"?[\\s\\)]",
        "CPMDeclarePackage\\s*\\(\\s*\\w+\\s*.*GITHUB_REPOSITORY\\s+\"?(?<depName>.*?)\"?\\s+VERSION\\s+\"?(?<currentValue>.*?)\"?[\\s\\)]"
      ],
      "datasourceTemplate": "github-releases",
      "extractVersionTemplate": "^v?(?<version>.*?)$"
    }
  ]
}

ⓘ Note that there are two similar Regular Expressions to be independent from the order of VERSION and GITHUB_REPOSITORY.

ⓘ Note that CMake values can be given with or without surrounding double quotes. This is reflected within the Regular Expression by \"?.

ⓘ Note that extractVersionTemplate is necessary to extract the version including the “v” prefix from the GitHub release tag whereas it is given without the prefix.

ⓘ Note that it is also possible to observe all tags and not only those published as a release using "datasourceTemplate": "github-tags".

Renovate Regex Manager for CPM Package Lock GIT_TAG

The following package-lock.cmake entry shows a dependency to a GitHub repository with a specific git tag (without the “v” prefix):

CPMDeclarePackage(JUCE
  GIT_TAG 7.0.1
  GITHUB_REPOSITORY juce-framework/JUCE
  EXCLUDE_FROM_ALL YES
)

Use this regexManager in renovate.json to update the dependency’s git tag with a pull request whenever there is a new release:

{
  "fileMatch": ["(^|/)package-lock\\.cmake$"],
  "matchStrings": [
    "CPMDeclarePackage\\s*\\(\\s*\\w+\\s*.*GIT_TAG\\s+\"?(?<currentValue>.*?[^0-9a-f\\s]+.*|.{1,4}?)\"?\\s+GITHUB_REPOSITORY\\s+\"?(?<depName>.*?)\"?\\s+",
    "CPMDeclarePackage\\s*\\(\\s*\\w+\\s*.*GITHUB_REPOSITORY\\s+\"?(?<depName>.*?[^0-9a-f\\s]+.*|.{1,4}?)\"?\\s+GIT_TAG\\s+\"?(?<currentValue>.*?)\"?\\s+"
  ],
  "datasourceTemplate": "github-releases"
}

ⓘ Note that the GIT_TAG value needs to contain at least one character that distinguishes it from a Git SHA. Also values with less than 4 characters are considered Non-SHA. This is reflected within the Regular Expression by .*?[^0-9a-f\\s]+.*|.{1,4}. GIT_TAG supports both tags and commits. They need to be distinguished because they need to be updated differently.

Renovate Regex Manager for CPM Package Lock GIT_TAG commit SHA

The following package-lock.cmake entry shows a dependency to a GitHub repository with a specific git commit:

CPMDeclarePackage(span
 GIT_TAG 836dc6a0efd9849cb194e88e4aa2387436bb079b
 GITHUB_REPOSITORY tcbrindle/span
 EXCLUDE_FROM_ALL YES
)

Use this regexManager in renovate.json to update the dependency’s git SHA with a pull request whenever there is a new commit in the “master” branch:

{
  "fileMatch": ["(^|/)package-lock\\.cmake$"],
  "matchStrings": [
    "CPMDeclarePackage\\s*\\(\\s*\\w+\\s*.*GIT_TAG\\s+\"?(?<currentDigest>[0-9a-f]{5,40}?)\"?\\s+GITHUB_REPOSITORY\\s+?\"?(?<depName>.*?)\"?\\s+",
    "CPMDeclarePackage\\s*\\(\\s*\\w+\\s*.*GITHUB_REPOSITORY\\s+\"?(?<depName>[0-9a-f]{5,40}?)\"?\\s+GIT_TAG\\s+?\"?(?<currentDigest>.*?)\"?\\s+"
  ],
  "datasourceTemplate": "git-refs",
  "depNameTemplate": "https://github.com/{{{depName}}}.git",
  "currentValueTemplate": "master"
}

ⓘ Note that the GIT_TAG value needs to be a Git SHA. This is reflected within the Regular Expression by [0-9a-f]{5,40}. GIT_TAG supports both tags and commits. They need to be distinguished because they need to be updated differently.

ⓘ Note that currentDigest is used instead of currentValue to reflect the fact that it is a unique SHA and not a version (that could contain e.g. wildcards). currentValue won’t work here.

ⓘ Note that depNameTemplate is used to construct the dependency’s Git URL since Renovate doesn’t support a dataSource like “github-refs” yet. Nevertheless this can be very useful when a custom Git URL is needed.

ⓘ Note that currentValueTemplate needs to be changed to “main” if this is the branch you want to observe.

Before it can be used, CPM.cmake needs to be downloaded. This can be done at build time with CMake. The following CMakeLists.txt line shows a simple uncached solution to download CPM.cmake from a specific GitHub release. A cached solution with variable is shown further below.

file(DOWNLOAD https://github.com/cpm-cmake/CPM.cmake/releases/download/v0.35.4/CPM.cmake ${CMAKE_BINARY_DIR}/cmake/CPM.cmake)

Use this regexManager in renovate.json to update CPM.cmake with a pull request whenever there is a new release:

{
  "fileMatch": ["(^|/)\\w+\\.cmake$", "(^|/)CMakeLists\\.cmake$"],
  "matchStrings": [
    "https:\\/\\/github\\.com\\/(?<depName>[^{}]*?)\\/releases\\/download\\/(?<currentValue>[^{}]*?)\\/"
  ],
  "datasourceTemplate": "github-releases"
}

ⓘ Note that this Regular Expression can be used to update any GitHub release download link, not just CPM.cmake.

ⓘ Note that this doesn’t work when the version is taken from a CMake variable. This is assured within the Regular Expression with [^{}]*.

Renovate Regex Manager for GitHub downloads with a version variable in CMake

To download CPM.cmake with the version specified in a CMake variable as shown above in How to use CPM.cmake?, there need to be some sort of convention to be able to update it with Renovate.

The following regexManager in renovate.json shows one possible solution to update the version variable for the download with a pull request whenever there is a new release:

{
  "fileMatch": ["(^|/)\\w+\\.cmake$", "(^|/)CMakeLists\\.cmake$"],
  "matchStrings": [
    "set\\s*\\(\\s*\\w+VERSION\\s+\"?(?<currentValue>[^v$][^${}]*?)\"?\\s*\\)[\\S\\s]*https:\\/\\/github.com\\/(?<depName>.*?)\\/releases\\/download\\/v\\$\\{\\w+VERSION\\}"
  ],
  "datasourceTemplate": "github-releases",
  "extractVersionTemplate": "^v?(?<version>.*?)$"
}

ⓘ Note that the variable containing the version needs to have a name ending with “VERSION”. It also needs to be defined right before the download.

ⓘ Note that since Regular Expression RE2 doesn’t support backreferences, the name of the version variable is only checked to end with “VERSION”, it isn’t checked to be actual equal to its definition.

Real world example

The repository speclet shows a real world example with all the above mentioned snippets in these files:

Summary

To answer the question of the introduction: C++ projects can of course also benefit from a modern tool like Renovate. Even if package managers like CPM.cmake aren’t supported yet, automatic version updates can still be customized using Renovate Regex Managers.

renovate.json shows a real world example with the whole configuration. It relies on package-lock.cmake containing the description of all dependencies.

It should also be possible to extend the configuration for projects that don’t use CPM Package Lock as well as for those that use CPM.cmake’s shorthand syntax in future.


References

tags: renovate - cpm - dependency - version

Hint: If you want to reach out to me without leaving a comment below, open a new discussion on GitHub.