MegaLinter with Azure DevOps

What is Linting?

In the context of software development and continuous integration/continuous delivery (CI/CD) pipelines, a linter is a tool that analyzes source code for potential errors, style violations, and other issues. The term “linter” is derived from the name of the original lint program, which was developed in the early 1970s to analyze C code.

Linters are widely used in modern software development to ensure code quality, maintainability, and adherence to coding standards. They can catch common programming errors, such as syntax mistakes, undefined variables, and other issues that might lead to bugs or difficulties in maintaining the codebase.

In a CI/CD pipeline, incorporating a linter as one of the stages helps to identify and address code issues early in the development process. This early feedback loop can significantly reduce the chances of introducing bugs and improve overall code quality. Linters are often integrated into the build and testing phases of a pipeline, allowing developers to receive immediate feedback on their code changes.

Popular programming languages have specific linters tailored to their syntax and conventions. For example, ESLint is a popular linter for JavaScript, Pylint for Python, and RuboCop for Ruby. In the context of a CI/CD pipeline, the linter stage is typically followed by other stages such as unit testing, integration testing, and deployment.

What is MegaLinter?

MegaLinter is an open-source, multi-language linter tool designed to support a wide range of programming languages and code formats in a single configuration. It aims to provide a comprehensive code analysis solution by integrating and running multiple linters for various languages in a unified manner. MegaLinter is often used in CI/CD pipelines to ensure consistent code quality across projects with diverse tech stacks.

Key features of MegaLinter include:

1. Multi-Language Support: MegaLinter supports a large number of programming languages and file formats. It can analyze code written in languages like JavaScript, Python, Java, Ruby, Bicep, Terraform, HTML, CSS, and many more.
2. Consistent Configuration: MegaLinter allows you to configure linting rules in a centralized way, making it easier to maintain and enforce coding standards consistently across different languages.
3. Unified Output: MegaLinter provides a unified output format, making it easier to interpret and act upon the results. This can be particularly useful when integrating MegaLinter into CI/CD pipelines.
4. Integration with CI/CD: MegaLinter is designed to be easily integrated into continuous integration and continuous deployment (CI/CD) pipelines. This means that it can automatically analyze code changes and provide feedback early in the development process.
5. Extensibility: MegaLinter supports adding or removing linters based on your project’s requirements. You can customize the set of linters to be run depending on the languages and formats used in your codebase.

How to templatize usage of linter in Azure DevOps and pipelines?

The below template can be stored in its own YAML file called megalinter_template.yaml:

steps:
    - checkout: self

    - script: docker pull oxsecurity/megalinter:v7
      displayName: Pull MegaLinter

    - script: |
        docker run -v $(System.DefaultWorkingDirectory):/tmp/lint \
            --env-file <(env | grep -e SYSTEM_ -e BUILD_ -e TF_ -e AGENT_) \
            -e SYSTEM_ACCESSTOKEN=$(System.AccessToken) \
            -e GIT_AUTHORIZATION_BEARER=$(System.AccessToekn) \
            -e MEGALINTER_CONFIG=.mega-linter.yml \
            oxsecurity/megalinter:v7
       displayName: Run MegaLinter

You can then call the template within your actual pipeline configuration file. In the below example, my pipeline triggers on Pull Requests:

trigger: none

pr:
    branches:
        include:
        - main

stages:

- stage: linter
  displayName: Linter
  jobs:
  - job: megalinter
    displayName: MegaLinter
    pool:
        vmImage: 'ubuntu-latest'
    steps:
    - template: megalinter_template.yaml

    # Upload MegaLinter reports
    - task: PublishPipelineArtifact@1
      condition: succeededOrFailed()
      displayName: Upload MegaLinter reports
      inputs:
        targetPath: "$(System.DefaultWorkingDirectory)/megalinter-reports/"
        artifactName: MegaLinterReport

MegaLinter Configuration File

# Configuration file for MegaLinter
# See all available variables at https://megalinter.io/configuration/ and in linters documentation

# APPLY_FIXES: all
ENABLE: # If you use ENABLE variable, all other languages/formats/tooling formats will be disabled by default
  - ARM
  - JSON
  - MARKDOWN
  - POWERSHELL
  - YAML
  - BICEP
ENABLE_LINTERS: # If you use ENABLE_LINTERS variable, all other linters will be disabled by default
  - ARM_ARM_TTK
  - BICEP_BICEP_LINTER
  - JSON_JSONLINT
  - MARKDOWN_MARKDOWNLINT
  - MARKDOWN_MARKDOWN_LINK_CHECK
  - MARKDOWN_MARKDOWN_TABLE_FORMATTER
  - POWERSHELL_POWERSHELL
  - POWERSHELL_POWERSHELL_FORMATTER
  - YAML_YAMLLINT
  - "TERRAFORM_TFLINT"
  - "TERRAFORM_TERRASCAN"
  - "TERRAFORM_TERRAFORM_FMT"
  - "YAML_PRETTIER"
# DISABLE:
#   - CLOUDFORMATION
#   - COPYPASTE # Uncomment to disable checks of excessive copy-pastes
#   - CSS
#   - EDITORCONFIG
#   - SPELL # Uncomment to disable checks of spelling mistakes
#   - JAVASCRIPT
#   - TEKTON
# DISABLE_LINTERS:
#   - JSON_V8R
#   - JSON_PRETTIER
#   - REPOSITORY_GITLEAKS
#   - REPOSITORY_KICS
#   - SPELL_PROSELINT
#   - SPELL_MISSPELL
#   - SPELL_VALE
#   - REPOSITORY_TRUFFLEHOG
# DISABLE_ERRORS_LINTERS:
#   - PYTHON_BANDIT
#   - PYTHON_PYRIGHT
#   - REPOSITORY_DEVSKIM
#   - REPOSITORY_GRYPE
#   - REPOSITORY_SEMGREP
#   - SPELL_VALE
VALIDATE_ALL_CODEBASE: true
PRINT_ALL_FILES: true
FILTER_REGEX_INCLUDE: '(CheckMe/)'
FILTER_REGEX_EXCLUDE: '(\.azuredevops|\.vscode|\.git|\.automation/test|\.automation/generated|\.github/workflows|docs/javascripts|docs/overrides|docs/json-schemas|flavors|TEMPLATES)'
JSON_JSONLINT_FILTER_REGEX_EXCLUDE: '(\.vscode/|bicepconfig.json)'
YAML_YAMLLINT_FILTER_REGEX_EXCLUDE: '(templates/\.mega-linter\.yml)'
# YAML_PRETTIER_FILTER_REGEX_EXCLUDE: '(templates/\.mega-linter\.yml|mkdocs\.yml)'
# YAML_V8R_FILTER_REGEX_EXCLUDE: '(descriptors|templates/\.mega-linter\.yml|\.codecov\.yml)'
BASH_FILTER_REGEX_EXCLUDE: "(lib)"
MARKDOWN_FILTER_REGEX_EXCLUDE: '(license\.md|docs/licenses)'
MARKDOWN_MARKDOWN_LINK_CHECK_DISABLE_ERRORS: true
SHOW_ELAPSED_TIME: true
FLAVOR_SUGGESTIONS: false
EMAIL_REPORTER: false
FILEIO_REPORTER: false
JSON_REPORTER: true
GITHUB_STATUS_REPORTER: false
GITLAB_COMMENT_REPORTER: false
AZURE_COMMENT_REPORTER: true
# PLUGINS:
#   - https://raw.githubusercontent.com/oxsecurity/megalinter/main/.automation/test/mega-linter-plugin-test/test.megalinter-descriptor.yml
# PRE_COMMANDS:
#   - command: echo "This is MegaLinter PRE_COMMAND on own MegaLinter ! :)"
#     cwd: "root"
# POST_COMMANDS:
#   - command: echo "This is MegaLinter POST_COMMAND on own MegaLinter ! :)"
#     cwd: "workspace"

The above is an example of a basic configuration file where I have specified curtain linters to be enabled and enabled reports to be submitted as a comment in the Azure DevOps Pull Request. All the linters will use their default configuration to analyze the code.

Running the pipeline configured with either of these .mega-linter.yaml configuration files will post to the Azure DevOps Pull Request. If you don’t want the report to post as a comment, you can set the AZURE_COMMENT_REPORTER to false in the configuration file. Without the report being commented on the PR, you will be able to visually see from the output logs a table containing the results of the analyses.

Attached report in the Azure DevOps pipeline PR checks, shown below

Pipeline run with error and recomendations