How to test a core pipeline repo against another, dependent repo with Azure DevOps PR build validation?

Context

In my Azure DevOps project, I have multiple project repos using a common Azure DevOps pipeline. I have accomplished this by defining the pipeline in a core pipeline repo and then referring to it from each project repo.

Repo: core-pipeline

File: pipeline.yml

jobs:
  - job: exampleJob
    displayName: Example job
    steps:
      - checkout: core_pipeline # Not magic; see azure-pipelines.yml below.
        path: $(PIPELINE_DIR_RELATIVE)
      - checkout: self
        path: $(PROJECT_DIR_RELATIVE)
      - task: ShellScript@2
        displayName: Show project repo info
        inputs:
          scriptPath: $(PIPELINE_DIR)/showProjectInfo.sh
          cwd: $(PROJECT_DIR)

variables:
  __SOURCES_DIR__: s
  PIPELINE_DIR_RELATIVE: $(__SOURCES_DIR__)/core-pipeline
  PIPELINE_DIR: $(Pipeline.Workspace)/$(PIPELINE_DIR_RELATIVE)
  PROJECT_DIR_RELATIVE: $(__SOURCES_DIR__)/$(Build.Repository.Name)
  PROJECT_DIR: $(Pipeline.Workspace)/$(PROJECT_DIR_RELATIVE)

File: showProjectInfo.sh

#!/bin/bash

pwd
ls -al

Repo: example-project

File: azure-pipelines.yml

resources:
  repositories:
    - repository: core_pipeline # Must match `checkout:` in core pipeline repo.
      type: git
      name: MyAzureDevOpsProject/core-pipeline
      ref: master

trigger:
  - master

extends:
  template: pipeline.yml@core_pipeline # Must match `repository:` above.

(This is the only piece of code duplicated across project repos.)

Problem

The setup described above works well. For example, I can have build validation for pull requests in example-project, and I can trigger pipeline runs manually from the Azure DevOps web GUI. In the latter case, I can optionally select another branch for the core_pipeline repository resource, which is useful for testing changes to the core-pipeline repo before merging them.

However, there is nothing preventing me from merging PRs in core-pipeline without having tested them whatsoever. I have to remember to manually test each project repo against the core-pipeline PR branch before merging the PR, which is somewhat tedious and, above all, very error-prone. There is effectively no protection against making arbitrarily bad changes to the core pipeline, breaking the workflow of all project repos depending on it.

I can add example-project‘s pipeline to the build validation policies for PRs in core-pipeline, but then core-pipeline‘s master branch is used for the validation build, which is useless; the point is to use the PR branch.

Goal

I would like to have a build validation policy for PRs in the core-pipeline repo such that example-project must pass a pipeline run using that PR branch of core-pipeline before the PR can me merged.

It is not necessary that all project repos using the core pipeline repo automatically be covered by said build validation; I’d be perfectly happy if I had to manually select (once) which project repos should be included.

Important note

The core pipeline consists of two equally important parts:

  • the YAML template – processed at template compile time, before the pipeline run is started
  • the Bash script – cloned at runtime in the checkout task

Checking out a specific ref using inline syntax, e.g.

- checkout: 'git://MyAzureDevOpsProject/core_pipeline@${{ parameters.ref }}'
  path: $(PIPELINE_DIR_RELATIVE)

is not a (complete) solution, because even if parameters.ref can be populated with the name of the PR branch, only the Bash script will be affected; the YAML template will still be from the master branch, not the PR branch.

Go to Source
Author: Simon Alling

Azure Pipeline PowerShell@2 Task can’t run ‘invoke-build’ command as inline script (build PowerShell Module)

I’m trying to build a powershell module in an Azure pipeline (dev.azure.com).

I’ve setup an initial pipleline and then building it up incrementally in small steps, because inexplicable errors are encountered far too easily on Azure.

I’ve added a ‘Prepare’ stage:

name: $(Build.DefinitionName)_$(Date:yyyyMMdd))

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'

variables:
  major: 0
  minor: 0
  patch: $(Build.BuildID)
  buildVersion: $(major).$(minor).$(patch)

stages:
- stage: Prepare
  jobs:
    - job: Prepare
      steps:
      - powershell: .bootstrap.ps1
        displayName: 'Install pre-requisites'

This is based on the original yaml that you get by default when you start a new Pipeline, so the part invoking the bootstrap file is what I got generated for free (This part works ok).

The bootstrap uses a dependencies file into which you can declare your dependencies and these are subsequently imported. One of those dependencies is InvokeBuild which contains the Invoke-Build command. So I would expect to be able to use the Invoke-Build command in subsequent stages.

The next part of the yaml pipeline is as follows:

- stage: Build
  jobs:
    - job: Build
      steps:
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'invoke-build build'

The invoke-build build, in itself is ok valid and works on a local host.

The Build job fails with the following error:

/usr/bin/pwsh -NoLogo -NoProfile -NonInteractive -Command . '/home/vsts/work/_temp/edb3758b-8e16-448d-bea2-9ba18acb9693.ps1'
Invoke-Build: /home/vsts/work/1/s/do-build.ps1:3
Line |
   3 |  Invoke-Build build
     |  ~~~~~~~~~~~~
     | The term 'Invoke-Build' is not recognized as the name of a
     | cmdlet, function, script file, or operable program. Check the
     | spelling of the name, or if a path was included, verify that
     | the path is correct and try again.

I discovered there is another way to invoke the build with the PowerShell@2 task using the filePath input:

- stage: Build
  jobs:
    - job: Build
      steps:
        - task: PowerShell@2
          inputs:
            filePath: '$(System.DefaultWorkingDirectory)/do-build.ps1'

the do-build.ps1 script is defined as:

Set-Location .Elizium.FakeBuddy
Invoke-Build build

(Elizium.FakeBuddy is the name of my PowerShell test module being built and that directory contains the InvokeBuild build script)

but this fails with the exact same error.

So why is the invoke-build command not available in the powershell session at the build stage, even though the Prepare stage has already imported the module.

The only thing I can think of is that different stages somehow don’t share the same powershell session, but I don’t know if this is correct, this seems unlikely, it doesn’t make sense that different stages dont share the same session.

EDIT: I tried an alternative version where the Build job is declared inside the same stage as Prepare, but this makes no difference, still a problem with Invoke-Build command.

Go to Source
Author: Plastikfan

How do i formulate file path for testsettings on the Runsettings file parameter

Whats the proper way to form relative file path/ or file path for the runsettings parameter

<SettingsFile>xxxxxAutomation.testsettings</SettingsFile> 

In my local environment this works but when i push it to build release agents it fails since
its looking for the file in a different directory.

This is the directory its looking at E:Agent_workr10axxxxxxxxx.testsettings,

here is my current way i have it set

<SettingsFile>xxxxxAutomation.testsettings</SettingsFile> 

here is the error message
Error: The test settings file E:Agent_workr10axxxxxxxxxx.testsettings, specified in the MSTestAdapter settings, is not available. Either access to the file is denied or the file does not exist. Ensure that the test settings file is available and try again.

instead of
E:Agent_workr10a_XXXXX CI BuilddropXXXXX_Automation_TestbinReleasexxxxxxxxx.testsettings

There is no documentation that states how to formulate the file path on msdn and didn’t see
anything online.

Go to Source
Author: skinnyWill

CI/CD ARM Templates – Should the cloud infrastructure and resources be checked on each deployment?

Given a company adopts infrastructure as code deployment model using ARM templates.

Is it a good practice to attempt to re-deploy the ARM templates alongside the application itself ?

If yes what deployment mode (incremental/substitution) should be used and why?

Thank you!

Go to Source
Author: Cristian E.