Johtizen

software engineering craftmanship blog
Home / About

View on GitHub
21 February 2021

Continuous Integration for JavaScript with npm

by JohT

Whereas there are numerous guidelines on how to setup specific tools like npm, combining all components to a build assembly line can be a tedious task. This article is intended to be a starting point, that gets you up and running with selected tools and references for vanilla JavaScript development. You can take it from there, exchange parts, add some more and dig into their configurations.

Addition 2021-09: Setting up an angular project shows that there are frameworks, that provide an easy way to set everything up. For them, only a few steps (4, 9 and 10) might be helpful.

Table of Contents

  1. Prerequisites
  2. 10 Steps to continuous integration
    1. 1. Package with npm
    2. 2. Bundle with Parcel
    3. 3. Unit Tests with Jasmine
    4. 4. Unit Test coverage measurement with nyc
    5. 5. Include Code Coverage Badge in README.md
    6. 6. Static code analysis with ESLint
    7. 7. Documentation generation with JSDoc
      1. Command line parameters explained
    8. 8. Run the whole chain within a single command
    9. 9. Continuous Integration with GitHub Actions
    10. 10. Publishing to npm
  3. Summary
    1. Setup Commands
    2. Script Commands
    3. Script Configuration in package.json
    4. Files and Directories
  4. Updates
  5. References

Prerequisites

10 Steps to continuous integration

These 10 steps cover right enough to get started. They don’t cover everything and are for sure not the right choice for every project. Hopefully, the following steps help to save some time. This living example was the original trigger and can be taken as reference too.

1. Package with npm

npm manages dependencies and provides a registry to publish packaged code. After installing npm, use cd to change into the directory of your repository and use the following command to create the package.json file:

npm init

Initializing npm within an already existing GIT repository automatically fills in all repository related package fields.

2. Bundle with Parcel

Parcel is an easy to use bundler, that copies multiple files into one, minifies them, transforms them,… As described here, parcel is added using the following command:

npm install parcel-bundler --save-dev

This adds parcel-bundler as development dependency inside your package.json file. If it hadn’t already been there, the directory nodes_modules will show up, which should be added to the .gitignore file:

# Dependency directories
node_modules/
# Optional npm cache directory
.npm

To run parcel for development and production, these two script commands need to be added in the package.json file:

{
  "scripts": {
    "dev": "parcel --out-dir devdist ./src/js/*.js",
    "build": "parcel build ./src/js/*.js",
  }
}

You can execute them by their name:

npm run dev
npm run build

Parcel puts all build results into the dist folder. Building for development leads to different files, that shouldn’t get published or shouldn’t even appear inside the repository. With --out-dir devdist all development build files get written into their own folder (here devdist) that can be ignored using .gitignore.

To assure that the dist folder gets cleaned up right before the build,
add the following script as described here. prebuild will automatically run when build is started.

{
  "scripts": {
    "prebuild": "rm -rf dist",
  }
}

If you have one entry point, like index.html or index.js, exchange ./src/js/*.js with it. ./src/js/*.js can be a good match for libraries, that publish all their javascript sources in different files or different variations (e.g. with/without IE compatibility).

If you accounter “command not found” problems while building in the pipeline, check if the first command in the chain is npm ci. If it is still an issue, try to prefix the commands with $(npm bin)/, e.g. $(npm bin)/parcel....

3. Unit Tests with Jasmine

Jasmine is a unit test framework for JavaScript that strongly encourages Behavior Driven Development (BDD). These two commands add Jasmine as a development dependency and initialize it by creating the file spec/support/jasmine.json:

npm install jasmine --save-dev
npx jasmine init

The script command inside package.json might look like this:

{
  "scripts": {
    "test": "jasmine --config=./test/js/jasmine.json",
  }
}

If you prefer having all tests and their configuration inside the folder test/js/, you can move jasmine.json there and refer to it by specifying --config=./test/js/jasmine.json in the command, as in the example above.

You can run the tests with:

npm test

The following example shows a test configuration in jasmine.json that reads all files ending with Test instead of Spec inside the folder test/js instead of spec. This sacrifices a bit of the Behavior Driven Development (BDD) philosophy in favour of a more commonly known and used structure:

{
  "spec_dir": "test/js",
  "spec_files": [
    "**/*Test.js"
  ],
  "helpers": [
    "polyfills/**/*.js",
    "**/*Data.js",
  ],
  "stopSpecOnExpectationFailure": false,
  "random": true
}

4. Unit Test coverage measurement with nyc

nyc measures test code coverage and extends the functionality of the formerly known Istanbul. Like all other development dependencies it is installed by the following command:

npm install nyc --save-dev

The script command inside package.json looks like this:

{
  "scripts": {
    "coverage": "nyc npm run test",
  }
}

Use the following command to run the tests including code coverage measurement:

npm run coverage

To assure that nyc fails when code coverage doesn’t meet the expectations, put a .nycrc configuration file into your project root. Here is an example including, among others, file name settings:

{
    "all": true,
    "include": [
        "src/**/*.js"
    ],
    "exclude": [
        "src/**/*-ie.js"
    ],
    "reporter": [
        "html",
        "text",
        "json-summary"
    ],
    "check-coverage": true,
    "branches": 90,
    "lines": 80,
    "functions": 80,
    "statements": 80
}

5. Include Code Coverage Badge in README.md

istanbul-badges-readme provides an easy way to dynamically add the test code coverage inside README.md.

Install it with:

npm install istanbul-badges-readme --save-dev

Add it as script command:

{
  "scripts": {
    "coverage-badge": "istanbul-badges-readme",
  }
}

Assure that nyc outputs the report additionally as json by adding "json-summary" as reporter inside the configuration file .nycrc:

"reporter": ["json-summary"]

Add the following line to your README.md to show the current branch coverage:

![Branches](https://img.shields.io/badge/Coverage-91.45%25-brightgreen.svg)

More examples can be found here. Finally run the following command after measuring the test coverage to update your README.md:

npm run coverage-badge

6. Static code analysis with ESLint

ESLint is a static code analyzer (aka “linter”) that detects typos and bugs inside the code. Use the following command to install it and initialize its configuration file .eslintrc.json:

npm install eslint --save-dev
npx eslint --init

Add the following script command inside package.json:

{
  "scripts": {
    "lint": "eslint \"./src/js/**\""
  }
}

Adapt the command if your JavaScript files are located somewhere else. Use the following command to run static code analysis:

npm run lint

7. Documentation generation with JSDoc

JSDoc is a code documentation generator based on code comments. Use the following command to install it:

npm install jsdoc --save-dev

Add the following script command inside package.json:

{
  "scripts": {
    "doc": "jsdoc -d doc --configure ./docs/jsdoc.json --readme ./README.md ./src/js/*.js"
  }
}

Command line parameters explained

To use markdown inside comments, the configuration file jsdoc.json will look like this:

{
    "plugins": [
        "plugins/markdown"
    ]
}

Use the following command to run documentation generation:

npm run doc

8. Run the whole chain within a single command

Running all previously mentioned commands in the right order for every change would be a tedious task. This script command shows how to put them into a chain and run them in sequence:

{
  "scripts": {
    "package": "npm run lint && npm run coverage && npm run coverage-badge && npm run doc && npm run build"
  }
}

The above example focuses on the raw sources and uses the build as finishing step. If lint fails, the chain fails early. coverage generates results for coverage-badge which changes the README.md that is included in doc generation.

Using & instead of && can be used to run tasks in parallel. Parentheses could be used to organize task groups. The chain depends on which tools are used and if the build needs to run first. Don’t hesitate to rearrange it as you like.

The whole chain can be started using the following command:

npm run package

9. Continuous Integration with GitHub Actions

GitHub Actions are triggered by events like git push and run predefined jobs like continuous integration pipelines. Create the directory .github/workflows/ and a .yaml file inside it, for example .github/workflows/action.yaml:

name: Node CI

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-node@v2
      with:
        node-version: '12'
    - name: Install nodes packages
      run: npm ci
    - name: Run linter
      run: npm run lint
    - name: Run tests
      run: npm test
    - name: Measure test coverage
      run: npm run coverage
    - name: Generate documentation
      run: npm run doc
    - name: Build
      run: npm run build

This action will run the previously introduced chain of commands in separate, traceable named steps. npm ci needs to be the first command. It loads all dependencies that are described inside package.json. Without it you may encounter “command not found” error messages.

coverage-badge is missing by intention, since it updates README.md. This is not practical, since the result would need another commit and push.

It is much easier to run npm run package before a git commit and push. You can even skip npm run doc and npm run build in the GitHub Actions job, because their results should already be checked in. On the other hand it can be helpful to see (e.g. on pull requests) if all steps succeeded.

10. Publishing to npm

Even if publishing new versions could be automated, it is often preferred to do it manually by intention, especially if it doesn’t happen that often and could imply to communicate breaking changes.

“How to publish packages to npm” covers this topic very well.


Summary

Here is a list of commands, configurations and files in a nutshell. This living example can also be taken as reference.

Setup Commands

Script Commands

Script Configuration in package.json

{
  "scripts": {
    "prebuild": "rm -rf dist",
    "lint": "eslint \"./src/js/**\"",
    "test": "jasmine --config=./test/js/jasmine.json",
    "coverage": "nyc npm run test",
    "coverage-badge": "istanbul-badges-readme",
    "doc": "jsdoc -d doc --configure ./docs/jsdoc.json --readme ./README.md ./src/js/*.js",
    "dev": "parcel --out-dir devdist ./src/js/*.js",
    "build": "parcel build ./src/js/*.js",
    "package": "npm run lint && npm run coverage && npm run coverage-badge && npm run doc && npm run build"
  }
}

Files and Directories


Updates

References

tags: continuous - integration - javascript - github - npm - parcel - bundler - jasmine - eslint - jsdoc

Hint: If you want to reach out to me without leaving a comment below, open a new discussion on GitHub.