Johtizen

software engineering craftmanship blog
Home / About

View on GitHub
20 January 2022

Most effective ways to push within GitHub Actions

by JohT

Continuous Integration [1] goes far beyond testing and building code nowadays. Test code coverage and other reports might get created, documentation might get updated and metrics and statistics might get refreshed. This article shows how these results can be pushed back into the repository using GitHub Actions [3].

Table of Contents

  1. Prerequisites
  2. Introduction
    1. Strive for fast feedback
    2. GitHub Events
      1. What you can do with it:
      2. What you need to be aware of:
    3. GIT commands
    4. GIT conflict resolution
    5. GitHub commit user email address
  3. All-in-one solutions
    1. Pros
    2. Cons
  4. The easiest solution
    1. Example 1
    2. Example 2
  5. Run some steps on auto commit
    1. Auto commit environment variable
    2. Skip push on auto commit
    3. Example 3
  6. GIT commit within a pull request
    1. Adapt Checkout
    2. Detect commit message and author using git log
    3. Example 4
  7. Conclusion
  8. Examples
  9. Updates
  10. References

Prerequisites

Introduction

Strive for fast feedback

Be prepared that it will likely need a couple of attempts to get a GitHub Actions Workflow to work as intended. Here are some ideas that might help:

GitHub Events

Basic workflows simply use on: [push] to get triggered on every push regardless of the branch. The following slight enhancement is also widely used and adds some useful features.

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

ⓘ Note that the default branch is named main [6] here.

ⓘ Note that it might seem reasonable to use

on:
  pull_request:
    types: [opened, synchronize, reopened, closed]

as described in Github Actions workflow for merged/closed PRs [17]. However, git commands won’t work if the feature branch gets deleted right after the pull request merge.

What you can do with it:

What you need to be aware of:

📖 Further reading: Events that trigger workflows [7]
📖 Further reading: GitHub Actions: A deep dive into “pull_request”[8]

GIT commands

Push to origin from GitHub Action [10] shows an easy way to commit and push changed files to the repository. Here is an example using environment variables for the commit author and message:

- name: GIT commit and push all changed files
  env: 
    CI_COMMIT_MESSAGE: Continuous Integration Build Artifacts
    CI_COMMIT_AUTHOR: Continuous Integration
  run: |
    git config --global user.name "${{ env.CI_COMMIT_AUTHOR }}"
    git config --global user.email "username@users.noreply.github.com"
    git commit -a -m "${{ env.CI_COMMIT_MESSAGE }}"
    git push

The following example shows how to only commit changed files in the docs folder:

- name: GIT commit and push docs
  env: 
    CI_COMMIT_MESSAGE: Continuous Integration Build Artifacts
    CI_COMMIT_AUTHOR: Continuous Integration
  run: |
    git config --global user.name "${{ env.CI_COMMIT_AUTHOR }}"
    git config --global user.email "username@users.noreply.github.com"
    git add docs
    git commit -m "${{ env.CI_COMMIT_MESSAGE }}"
    git push

ⓘ Note that the username in username@users.noreply.github.com needs to be replaced.

GIT conflict resolution

(Added in June 2024)

Since pipelines can run in parallel, it is possible that merge conflicts arise when new commits are pushed into the main branch. Normally, you would go through them manually and decide how to merge them. This can be automated when the merge strategy is known in advance.

Considering that automated commits are usually used for generated reports, documentation and alike, it makes sense to pick the “latest and greatest” by prioritizing local changes and overriding conflicting files in the main branch [25]. This can be done by defining the conflict resolution strategy “theirs” as the following example shows:

- name: GIT commit and push docs overriding conflicts with local changes (verbose)
  env: 
    CI_COMMIT_MESSAGE: Continuous Integration Build Artifacts
    CI_COMMIT_AUTHOR: Continuous Integration
  run: |
    git config --global user.name "${{ env.CI_COMMIT_AUTHOR }}"
    git config --global user.email "username@users.noreply.github.com"
    git add docs
    git commit -m "${{ env.CI_COMMIT_MESSAGE }}"
    git fetch origin
    git rebase --strategy-option=theirs origin/main
    git push

ⓘ Note that git fetch origin is needed before rebasing to be up-to-date with the remote repository.

⚠ Note that git add docs is used here to only stage changes in the docs directory e.g. for automatic documentation generation. In practice it is highly advisable to only add the files you really want to commit and assure that no unintended changes are added, especially because every conflicting file in the main branch will be overwritten with the chosen strategy.

⚠ Note that the strategy “theirs” will overwrite conflicting files in the main branch with the newly committed content regardless on how up-to-date the local branch is.

Here is a variation of the example above with verbose output and status prints in between to get a better intuition of what is going on in detail and for troubleshooting:

- name: GIT commit and push docs overriding conflicts with local changes (verbose)
  env: 
    CI_COMMIT_MESSAGE: Continuous Integration Build Artifacts
    CI_COMMIT_AUTHOR: Continuous Integration
  run: |
    git config --global user.name "${{ env.CI_COMMIT_AUTHOR }}"
    git config --global user.email "username@users.noreply.github.com"
    git add docs
    git status
    git commit --verbose -m "${{ env.CI_COMMIT_MESSAGE }}"
    git status
    git fetch origin
    git rebase --strategy-option=theirs origin/main --verbose
    git push --verbose

To keep the examples below easy on the eye, automatic conflict resolution will be left out.

GitHub commit user email address

(Added in June 2024)

If you want to use your GitHub user email address for git commits, have a look at this GitHub documentation page: Setting your commit email address [26]

If you want to see the commits as if they came from the GitHub bot including its special annotation, then have a look at this checkout action pull request: README: Suggest user.email to be 41898282+github-actions[bot]@users.noreply.github.com [27]

All-in-one solutions

Already existing all-in-one solutions provide a good starting point for many use cases. On the downside, their parameters and their behaviour might change over time and won’t be as stable as git commands. As soon as more control or flexibility is needed, the examples below might be a better fit. If simplicity is key, also have a look at example 1.

Pros

Cons

The easiest solution

If the automatically generated and updated files should only be pushed into the repository on a push into the main branch e.g. after a pull request is merged, then the following example shows how this can be achieved. Even though it shows a JavaScript Node.js workflow, the build steps can easily be replaced by others.

Example 1

name: Continuous Integration
on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
jobs:
  build:
    runs-on: ubuntu-latest
    env: 
      CI_COMMIT_MESSAGE: Continuous Integration Build Artifacts
      CI_COMMIT_AUTHOR: Continuous Integration
    steps:
    - uses: actions/checkout@v3

    # Build steps
    - uses: actions/setup-node@v3
      with:
        node-version: '12' 
    - name: Node Install
      run: npm ci
    - name: Node Build (lint, test, coverage, doc, build, package)
      run: npm run package

    # Commit and push all changed files.
    - name: GIT Commit Build Artifacts (coverage, dist, devdist, docs)
      # Only run on main branch push (e.g. after pull request merge).
      if: github.event_name == 'push'
      run: |
        git config --global user.name "${{ env.CI_COMMIT_AUTHOR }}"
        git config --global user.email "username@users.noreply.github.com"
        git commit -a -m "${{ env.CI_COMMIT_MESSAGE }}"
        git push

ⓘ Note that it isn’t necessary to prevent the workflow from being triggered again by the automatically executed push. Triggering a workflow from a workflow [7] states that “events triggered by the GITHUB_TOKEN will not create a new workflow run”.

ⓘ Note that if you use a personal access token for actions/checkout [14], the workflow will trigger itself again resulting in an endless loop. The next example shows how to solve this.

⚠ Be aware that this won’t work when the main branch is protected. One way to overcome this is to create a personal access token as an administrator and use this as secret token as shown in the following example.

Example 2

This variation of example 1 uses a personal access token for the checkout and therefore needs to assure that the pipeline won’t run again on the automatically created commit/push.

This can be used to overcome the issue with protected branches that won’t allow a push otherwise as discussed in How to push protected branches [23]. The personal access token needs to be created from an administrator or a technical user that is selected from the organisation as an exception within the settings of the protected branch rules [24]. The personal access token only needs to be granted for public_repo (public repositories).

name: Continuous Integration
on:
  push:
    branches:
      - main
    # Ignore changes in folders that are affected by the auto commit. (Node.js project)
    paths-ignore: 
      - 'coverage/**'
      - 'devdist/**'
      - 'dist/**'
      - 'docs/**'
  pull_request:
    branches:
      - main
jobs:
  build:
    runs-on: ubuntu-latest
    env: 
      CI_COMMIT_MESSAGE: Continuous Integration Build Artifacts
      CI_COMMIT_AUTHOR: Continuous Integration
    steps:
    - uses: actions/checkout@v3
      with:
        token: ${{ secrets.WORKFLOW_GIT_ACCESS_TOKEN }}

    # Build steps
    - uses: actions/setup-node@v3
      with:
        node-version: '12' 
    - name: Node Install
      run: npm ci
    - name: Node Build (lint, test, coverage, doc, build, package)
      run: npm run package

    # Commit and push all changed files. 
    # Must only affect files that are listed in "paths-ignore".
    - name: GIT Commit Build Artifacts (coverage, dist, devdist, docs)
      # Only run on main branch push (e.g. pull request merge).
      if: github.event_name == 'push'
      run: |
        git config --global user.name "${{ env.CI_COMMIT_AUTHOR }}"
        git config --global user.email "username@users.noreply.github.com"
        git add coverage devdist dist docs
        git commit -m "${{ env.CI_COMMIT_MESSAGE }}"
        git push

ⓘ Note that the folders in paths-ignore and git add need to be the same to prevent the workflow from being triggered by itself.

📖 Further reading: Creating a personal access token [19]

📖 Working example: data-restructor-js [22]

Run some steps on auto commit

If some of the steps in the workflow should also be executed for the automatically generated commit/push, the commit name and author can be compared to detect the auto commit run. At least the step that creates the auto commit needs to be skipped when the auto commit triggered the workflow. Furthermore, a personal access token needs to be created, otherwise the whole workflow would be skipped for the automatically created commit.

Auto commit environment variable

The following workflow step shows how the automatically created commit can be detected and how the result can be written into an environment variable.

- name: Set environment variable "is-auto-commit"
  if: github.event.commits[0].message == env.CI_COMMIT_MESSAGE && github.event.commits[0].author.name == env.CI_COMMIT_AUTHOR
  run: echo "is-auto-commit=true" >> $GITHUB_ENV

ⓘ Note that this will only work for the GitHub event “push”. The variables will be empty when triggered by a “pull_request” event. Thus, “is-auto-commit” would always be false. GIT commit within a pull request shows how to solve this.

Add optional display steps for debugging and maintenance purposes as you like:

- name: Display Github event variable "github.event.commits[0].message"
  run: echo "last commit message = ${{ github.event.commits[0].message }}" 
- name: Display Github event variable "github.event.commits[0].author.name"
  run: echo "last commit author = ${{ github.event.commits[0].author.name }}" 
- name: Display environment variable "is-auto-commit"
  run: echo "is-auto-commit=${{ env.is-auto-commit }}"

Skip push on auto commit

The environment variable defined above can then be used to skip selected steps. At least the step that pushes the changed files into the repository needs to be skipped.

- name: Commit build artifacts (dist, devdist, docs, coverage)
  # Only run on main branch push (e.g. pull request merge). 
  # Don't run again on an already pushed auto commit.
  if: github.event_name == 'push' && env.is-auto-commit == false
  run: |
    git config --global user.name "${{ env.CI_COMMIT_AUTHOR }}"
    git config --global user.email "username@users.noreply.github.com"
    git commit -a -m "${{ env.CI_COMMIT_MESSAGE }}"
    git push

Any other steps can be skipped for the auto commit, but don’t have to.

Example 3

name: Continuous Integration
on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
jobs:
  build:
    runs-on: ubuntu-latest
    env: 
      CI_COMMIT_MESSAGE: Continuous Integration Build Artifacts
      CI_COMMIT_AUTHOR: ${{ github.event.repository.name }} Continuous Integration
    steps:
    - uses: actions/checkout@v3
      with:
        token: ${{ secrets.WORKFLOW_GIT_ACCESS_TOKEN }}

    # Set environment variable "is-auto-commit" 
    - name: Set environment variable "is-auto-commit"
      if: github.event.commits[0].message == env.CI_COMMIT_MESSAGE && github.event.commits[0].author.name == env.CI_COMMIT_AUTHOR
      run: echo "is-auto-commit=true" >> $GITHUB_ENV

    # Display variables for debugging
    - name: Display Github event variable "github.event.commits[0].message"
      run: echo "last commit message = ${{ github.event.commits[0].message }}" 
    - name: Display Github event variable "github.event.commits[0].author.name"
      run: echo "last commit author = ${{ github.event.commits[0].author.name }}" 
    - name: Display environment variable "is-auto-commit"
      run: echo "is-auto-commit=${{ env.is-auto-commit }}"

    # Build (will also run on auto commit)
    - uses: actions/setup-node@v3
      with:
        node-version: '12' 
    - name: Install node packages
      run: npm ci
    - name: Build package (lint, test, build, package, merge)
      run: npm run package

    # Commit and push all changed files.
    - name: Display event name 
      run: echo "github.event_name=${{ github.event_name }}"
    - name: Commit build artifacts (dist, devdist, docs, coverage)
      # Don't run again on already pushed auto commit. Don't run on pull request events.
      if: env.is-auto-commit == false && github.event_name != 'pull_request'
      run: |
        git config --global user.name "${{ env.CI_COMMIT_AUTHOR }}"
        git config --global user.email "joht@users.noreply.github.com"
        git commit -a -m "${{ env.CI_COMMIT_MESSAGE }}"
        git push

ⓘ Note that a personal access token is needed as mentioned above.

GIT commit within a pull request

If it is mandatory that automatically changed files need to be pushed within a pull request, e.g. to be able to review them, then there are a couple of things that need to be taken into account:

Adapt Checkout

The ref parameter of actions/checkout [14] needs to be set to the head reference of the pull request [8] to checkout the feature branch and be able to get the last commit of it:

- uses: actions/checkout@v3
  with:
    ref: ${{ github.event.pull_request.head.ref }}

ⓘ Note that this also works when the workflow is triggered by a push event. The reason is that ${{ github.event.pull_request.head.ref }} will be empty which will be replaced by the default value.

ⓘ Note that ${{ github.event.pull_request.head.sha }} won’t work as checkout ref. It will lead to an error message like fatal: You are not currently on a branch. when the automatically changed files are pushed.

Detect commit message and author using git log

The command git log [15] shows the commit history. The output can be limited to only display the last commit and formatted to only show the commit message or author.

git --no-pager log -1 --pretty=format:'%s' # prints the message of the last commit
git --no-pager log -2 --pretty=format:'%an' # prints the author name of the last commit

ⓘ Note that the git command option –no-pager [20] is used to print the result directly to the console. This is not necessary when the command is used within an echo command.

The following workflow steps show how to put commit message and author into environment variables in a linux or unix shell:

- name: Set environment variable "commit-message"
  run: echo "commit-message=$(git log -1 --pretty=format:'%s')" >> $GITHUB_ENV
- name: Set environment variable "commit-author"
  run: echo "commit-author=$(git log -1 --pretty=format:'%an')" >> $GITHUB_ENV

Add optional display steps for debugging and maintenance purposes as you like:

- name: Display environment variable "commit-message"
  run: echo "commit-message=${{ env.commit-message }}"
- name: Display environment variable "commit-author"
  run: echo "commit-author=${{ env.commit-author }}"

📖 Further reading: GIT pretty format [16]

Example 4

name: Continuous Integration
on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
jobs:
  build:
    runs-on: ubuntu-latest
    env: 
      CI_COMMIT_MESSAGE: Continuous Integration Build Artifacts
      CI_COMMIT_AUTHOR: ${{ github.event.repository.name }} Continuous Integration
    steps:

    # Checkout that works with "push" and "pull_request" trigger event
    - uses: actions/checkout@v3
      with:
        ref: ${{ github.event.pull_request.head.ref }}
        token: ${{ secrets.WORKFLOW_GIT_ACCESS_TOKEN }}

    # Set environment variables based on the last commit
    - name: Set environment variable "commit-message"
      run: echo "commit-message=$(git log -1 --pretty=format:'%s')" >> $GITHUB_ENV
    - name: Display environment variable "commit-message"
      run: echo "commit-message=${{ env.commit-message }}"

    - name: Set environment variable "commit-author"
      run: echo "commit-author=$(git log -1 --pretty=format:'%an')" >> $GITHUB_ENV
    - name: Display environment variable "commit-author"
      run: echo "commit-author=${{ env.commit-author }}"

    - name: Set environment variable "is-auto-commit"
      if: env.commit-message == env.CI_COMMIT_MESSAGE && env.commit-author == env.CI_COMMIT_AUTHOR
      run: echo "is-auto-commit=true" >> $GITHUB_ENV
    - name: Display environment variable "is-auto-commit"
      run: echo "is-auto-commit=${{ env.is-auto-commit }}"

    # Build
    - uses: actions/setup-node@v3
      if: env.is-auto-commit == false
      with:
        node-version: '12'
    - name: (Main) Install nodes packages
      if: env.is-auto-commit == false
      run: npm ci
    - name: (Main) Build package (lint, test, doc, build, package)
      if: env.is-auto-commit == false
      run: npm run package
    
    # Commit generated and commit files
    - name: Display event name 
      run: echo "github.event_name=${{ github.event_name }}"
    - name: Commit build artifacts (dist, devdist, docs, coverage)
      # Don't run again on already pushed auto commit. Don't run on pull request events.
      if: env.is-auto-commit == false && github.event_name != 'pull_request'
      run: |
        git config --global user.name "${{ env.CI_COMMIT_AUTHOR }}"
        git config --global user.email "username@users.noreply.github.com"
        git commit -a -m "${{ env.CI_COMMIT_MESSAGE }}"
        git push

Conclusion

The easiest solution shown in example 1 pushes all changed files automatically when a pull request is merged or a commit is directly pushed into the main branch. This will most likely cover common use cases like JavaScript distribution bundles, static site generation and test coverage reports.

Example 2 shows how to overcome a protected main branch using a personal access token.

Preventing the workflow from being triggered by itself is essential to avoid an endless loop. This isn’t an issue when using the build-in GITHUB_TOKEN. As soon as a personal access token is used, it needs to be assured, that the auto commit is only executed once.

Whilst example 2 skips the whole workflow for the automatically created commit using paths-ignore [11], example 3 shows how this can be configured for every single step detecting the auto commit by its message and author.

To be able to review an automatically pushed commit it is required that it is executed within a pull request. For this some extra effort is needed as shown in example 4.

Examples


Updates

References

tags: continuous - integration - github - actions - automate - git

Hint: If you want to reach out to me without leaving a comment below, open a new discussion on GitHub.