Hello friend! This comprehensive guide will equip you with in-depth knowledge of Azure Pipelines and help you leverage the powerful CI/CD capabilities to transform and accelerate your software delivery.
Why Continuous Integration and Continuous Delivery Matter
Let‘s first understand what motivated the evolution of modern software practices like CI/CD and DevOps.
Traditionally, software was released in long cycles – teams would work in isolation for months before integrating code to build software. This waterfall approach with lengthy testing and manual deployments led to:
- Slow feedback loops
- Integration headaches and bugs
- Long lead times between fixes
- Lack of reliability and productivity
The solution was to shift left on quality and automation!
This led to the adoption of agile practices like:
- Continuously integrating code changes
- Constant communication and visibility
- Heavy focus on test automation
- Streamlining deployments across environments
Research shows solid results from these techniques:
- 46x more frequent code deployments
- 440x faster lead time from commit to deploy
- 96x faster recovery from failures
- 5x more productive teams
It‘s clear that CI/CD and DevOps play a huge role in enabling software teams to deliver better quality applications faster.
Overview of Azure Pipelines
Azure Pipelines provides enterprise-grade CI/CD services leveraging Microsoft cloud infrastructure. With over 800,000 organizations using Azure DevOps, it has quickly emerged as a leader in the CI/CD space.
Key capabilities offered by Azure Pipelines:
- Works with major code repositories – GitHub, Azure Repos etc.
- Supports all major languages and platforms (.NET, JavaScript, Python, Java etc)
- Cloud hosted agents provide scalability
- Tight integration with other Azure services
- Open ecosystem with REST APIs and extensibility
- 5000+ free build minutes every month
- Parallel jobs for faster execution
- Detailed analytics on pipeline performance
Combining these with robust CI/CD capabilities, Azure Pipelines provides great speed, flexibility and visibility for your software delivery needs.
Core Concepts
Now that you understand the importance of CI/CD and what Azure Pipelines offers, let‘s go deeper into the key concepts and components involved in pipelines.
Triggers
Triggers indicate what events should generate a pipeline run. This allows automating pipelines based on external events like code changes rather than having to manually trigger pipelines.
| Trigger Type | Description |
|---|---|
| Continuous Integration | Launch pipeline on every code commit/pull request |
| Scheduled | Trigger pipeline daily, weekly, monthly etc |
| Manual | Trigger pipeline on demand manually |
For example, you can configure a CI trigger on your main branch to run tests and generate builds on every code commit done by developers.
Stages
Stages break down pipelines into logical steps in your DevOps process, such as Build, Test, Staging and Production. Stages provide better visibility and help you understand the end-to-end flow.
Each stage includes one or more jobs that run sequentially or in parallel depending on dependencies.
Stages enable you to optimize your development flow for productivity without compromising on software quality.
Jobs
Jobs represent units of execution that run as part of a stage. A single stage can consist of multiple jobs running across different agents and machines for greater parallelism.
For example, functional testing and performance testing can run as two simultaneous jobs under a Test stage to reduce overall execution time of pipelines.
Steps
Steps contain the actual scripts, tasks and commands that do the work within a job. They are the smallest building blocks of logic in your pipelines.
Some examples of popular steps:
- Fetch code from repositories
- Restore application dependencies
- Compile, build and publish binaries
- Run unit test cases
- Deploy artifacts and packages
By configuring job steps you orchestrate the entire workflow involved in delivering software changes.
Agents
Agents provide the execution environment where pipelines run. Azure Pipelines supports:
- Microsoft-hosted agents: Maintained by Microsoft, agents come preinstalled with a wide range of tools and software
- Self-hosted agents: You manage yourself on your own infrastructure to meet custom requirements
Based on your use case, you can leverage cloud-hosted scalability or get more control with private agents.
Artifacts
Artifacts refer to files and packages, like compiled binaries, test reports, logs etc generated when pipelines run.
Artifacts flow across pipeline stages, allowing later stages to use outputs from earlier stages. All artifacts get stored securely and immutably on Azure servers.
Environments
Environments represent deployment targets like Dev, QA, Production, to which applications can be promoted via pipelines. They provide visibility into deployments across various stages.
Environment-based deployment practices enable staging and testing code in lower environments before releasing to production.
With these core concepts covered, you should have a solid understanding now of the components involved in authoring Azure pipelines.
Now let‘s shift gears and go through the steps involved in actually creating CI/CD pipelines!
Creating Your First Azure Pipeline
We will create an automated pipeline to build, test and deploy a web app when new code gets checked in to the repository.
1. Configure source code repository
Azure pipelines support code repositories like GitHub, Azure DevOps, Bitbucket etc. You can even build from local Git repos.
For this example, we will use an Azure DevOps Git repo containing a basic NodeJS web app code:
2. Create new pipeline
Go to the Azure DevOps portal Pipelines section and click Create Pipeline:
Select your repository:
Then click on the "Starter pipeline" template which contains some boilerplate code to quickly get started.
3. Add tasks
The starter pipeline contains Placeholder steps. Let‘s now replace them with actual tasks required to build and deploy this app continuously:
steps:
- task: NodeTool@0
inputs:
versionSpec: ‘14.x‘
displayName: ‘Install Node.js‘
- task: Npm@1
inputs:
command: ‘install‘
displayName: ‘Install app dependencies‘
- script: |
npm run build
displayName: ‘Build app‘
- task: PublishPipelineArtifact@0
inputs:
artifactName: ‘dist‘
targetPath: ‘dist‘
displayName: ‘Publish artifact‘ This does the following:
- Installs NodeJS
- Restores NPM dependencies
- Builds the app
- Publishes the build outputs to use for deployments
4. Configure Azure web app deployment
Next we will deploy our built app continuously to an Azure web app using Azure App Service deploy tasks:
- task: AzureWebApp@1
inputs:
azureSubscription: ‘AzureSubscription‘
appName: ‘myWebApp‘
package: ‘$(Pipeline.Workspace)/dist‘Make sure to connect your Azure subscription. This picks up files from the published artifacts and deploys them.
The complete pipeline implements CI/CD! It builds and deploys every code change automatically.
5. Save and run
Go to the pipeline Logs to see it build, test and deploy your app seamlessly end-to-end upon commits.
In a similar way you can build deployment pipelines for mobile apps, databases etc. leveraging tasks from Azure marketplace.
This gives you an overview of how quick and powerful building Azure pipelines can be!
Now let me share some expert tips to take it to the next level.
Authoring Like an Azure Pro!
While the visual designer view provides a simple drag-and-drop interface, I recommend becoming comfortable with YAML pipelines.
Here are some benefits you get:
- Fully scriptable, no UI dependency
- Version controlled code for pipelines
- Works well with standard configurations
- Flexibility for complex scenarios
Let‘s look at some ways to optimize YAML pipeline code:
Modular Design
Structure your YAML into separate sections for triggers, parameters, stages, steps etc. each clearly defined. For example:
parameters:
- name: ‘environmentName‘
type: string
default: ‘dev‘
trigger:
- main
variables:
- group: ‘secrets‘
stages:
- stage: Build
jobs:
- job:
steps:
- script: echo "Building..."
Reuse via Templates
Define reusable templates for common jobs like building, testing and deployment. This avoids having to rewrite them in multiple places.
# common-jobs.yml
jobs:
- job: Build
steps:
- script: echo "Building solution"
- job: Deploy
steps:
- script: echo "Deploying..."You can include this template across pipelines and customize values.
runtimeParameters
For dynamic values, define runtime parameters instead of hardcoding:
parameters:
- name: vmImage
type: string
default: ubuntu-latest
jobs:
- job: Build
pool:
vmImage: $[{parameters.vmImage}]Set their values during execution.
Variables
Store configurable constants like resource names, locations centrally as variables for consistency:
variables:
azureServiceConnection: ‘contoso-service-connection‘
appName: ‘ContosoApp‘
steps:
- task: AzureFunctionApp@1
inputs:
azureSubscription: ‘$(azureServiceConnection)‘
appName: ‘$(appName)‘So in summary, some best practices are:
- Modular pipeline organization
- Separate base templates for reusability
- Parameterize configurable aspects
- Centralize common values as variables
This will ensure your pipeline code is clean, readable and maintainable.
Integrating Repo Code → Cloud Agents → Real-world Testing
While building and deploying your software is essential, comprehensive testing uncovers bugs and ensures code quality.
Azure pipelines provide excellent integration with cloud-based test automation services that run your app on real devices and browsers at scale.
For example, BrowserStack offers a real device cloud with thousands of mobile devices, browsers and operating systems available instantly with parallel testing capabilities.
You can easily integrate BrowserStack‘s test suite into pipelines to validate web and mobile apps each time developers commit code changes.
Here is a sample Azure pipeline configuration with BrowserStack:
steps:
- script: |
npm install
npm run build
displayName: ‘Build App‘
# BrowserStack credentials
env:
BROWSERSTACK_USERNAME: $(BROWSERSTACK_USERNAME)
BROWSERSTACK_ACCESS_KEY: $(BROWSERSTACK_ACCESS_KEY)
- script: |
browserstack-runner run android
displayName: ‘BrowserStack Tests‘This allows you to shift testing left and prevent bugs from progressing downstream.
There are similar integrations for tools like Selenium, Appium, JMeter and more to build robust testing regimes.
Additional Best Practices
Here are some other tips from my years of experience for smooth Azure pipelines:
- Have a branching strategy with conventions on pull requests, code reviews etc.
- Implement infrastructure as code principles via Terraform, Ansible
- Containerize builds using Docker for portability
- Perform security scanning of code and dependencies
- Enable notifications via Slack, Email etc. for visibility
- Analyze pipeline analytics to identify bottlenecks
- Follow infrastructure monitoring best practices
These processes and tools will complement your core Azure pipelines to round out your DevOps workflows.
Conclusion
In this guide, we went from covering the basics of continuous delivery to an end-to-end implementation of a pipeline plus additional integrations and practices.
Azure Pipelines provide you 70+ mind-blowing features to support all your DevOps processes. Leveraging these best practices will help you transform and rev up your software delivery lifecycles.
I hope you found this guide helpful. Feel free to reach out if you have any other questions!
