Releases: buildkite/elastic-ci-stack-for-aws
v5.4.0
v5.4.0 (2021-06-30)
Added
Changed
- Update Buildkite Agent to version 3.30.0 #868 (@esalter)
- The HttpPutResponseHopLimit from 1 to 2 #858
Fixed
- The default cost allocation tag value #859
Upgrading
Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v5.4.0/aws-stack.yml
If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):
Documentation
v5.3.2
v5.3.2 (2021-06-11)
Fixed
- s3secrets-helper for Windows #846 (DuBistKomisch)
- Docker systemd configuration to match the installed Docker version #849 (cmanou)
- Excessive instance scaling while waiting for instances to boot
Changed
Upgrading
Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v5.3.2/aws-stack.yml
If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):
Documentation
v5.3.1
v5.3.1 (2021-05-05)
Fixed
- Allow dashes and multiple forward slashes (/) in BuildkiteAgentTokenParameterStorePath #835 #837 (nitrocode)
Upgrading
Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v5.3.1/aws-stack.yml
If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):
Documentation
v5.3.0
v5.3.0 (2021-04-28)
Added
Changed
- Replace awslogs with the cloudwatch-agent #811 (yob)
- Avoid scaling down too aggressively when there are pending jobs in certain conditions #823 (yob)
- Bump docker from 19.03.x to 20.10.x #826 (yob)
- Bump docker-compose on all operating systems to 1.28.x #825 (yob)
- Bump agent from 3.27.0 to 3.29.0 #827 (yob)
- Bump lifecycled from 3.0.2 to 3.2.0 #824 (yob)
- Bump git on windows from 2.22.0 to 2.31.0 #819 (yob)
- Bump ECR plugin to v2.3.0 #816 (chloeruka)
- Documentation improvements #815 #810 (acaire)
Removed
Upgrading
Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v5.3.0/aws-stack.yml
If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):
Documentation
Elastic CI Stack for AWS v5.2.0
Added
- buildkite-agent v3.27.0 #794 (pda)
- agent names use client-side
%spawn
not server-side%n
for numbering #794 (pda) IMDSv2Tokens
parameter: optional / required #786 (holmesjr) → #788 & #789 (pda)
Changed
- Default to
gp3
volumes, previouslygp2
#784 (yob)
Fixed
c6gn.*
instances recognized as ARM #785 (yob)s3secrets-helper
installation more resilient #783 (shevaun)
Upgrading
Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v5.2.0/aws-stack.yml
If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):
Documentation
Elastic CI Stack for AWS v5.1.0
Added
- Experimental support for ARM instance types (linux only) #758 (yob)
- Support up to four instance types and mixed combinations of Spot/OnDemand instances #710 (yob)
- The
InstanceType
stack parameter can now be a CSV with up to 4 types - The new
OnDemandPercentage
stack parameter can be reduced from 100% (the default) to allow some Spot instances
- The
Changed
- Update Buildkite Agent to v3.26.0 #778 (JuanitoFatas)
- Speed up secret downloads from S3 (from ~8 seconds to under 1 second) #772 (pda)
- ECR plugin now has its own log group header to make run time visible #773 (pda)
Fixed
- Avoid IAM changes for some kinds of stack updates (like changing InstanceType) #781 (yob)
- Improved documentation
Upgrading
Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v5.1.0/aws-stack.yml
If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):
Documentation
Elastic CI Stack for AWS v5.0.1
This is a bugfix release relevant if you're using Windows and want to use AWS SSM Parameter Store for storing your agent token. Please see the v5.0.0 release for full notes on what's new in version 5 of the stack.
Fixed
- Retrieve agent token from parameter store on Windows agents #762 (chrisfowles)
Upgrading
In most cases, you should be able to upgrade in-place older versions of the stack with a CloudFormation stack update using the following template URL:
https://s3.amazonaws.com/buildkite-aws-stack/v5.0.1/aws-stack.yml
If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):
Documentation
Elastic CI Stack for AWS v5.0.0
What's new?
Previously experimental Lambda-based scaler is the new default 🚀
Our previously experimental fast autoscaler that scales up much faster is now always used. Instances will automatically scale to demand using the stack defaults and you should see good results without changing any parameters.
Experimental Windows support is now available
A new AMI built on Windows Server 2019 is optionally available to use in the stack for Windows builds. You can enable this by changing the InstanceOperatingSystem
setting from the default linux
to windows
. This release includes recent updates to Windows stability on Buildkite agent v3.25.0. Huge thanks to @jeremiahsnapp and @tduffield for their contributions here!
Note: There is a known issue with graceful handling of spot instances under windows. The agent may not disconnect gracefully, and may appear in the Buildkite UI for a few minutes after they terminate (See issue #752). We recommend using Windows on-demand instances for now.
Added
- Support for buildkite/image-builder which can enable you to customize AMIs based off the ones we ship #692 (keithduncan)
- Support for multiple security groups on instances #667 (jdub)
- AMI and Lambda Scaler support more regions: ap-east-1 (Hong Kong), me-south-1 (Bahrain), af-south-1 (Cape Town), eu-south-1 (Milan) #718 (JuanitoFatas)
- Support for loading BuildkiteAgentTokenPath from AWS Parameter Store #601 (jradtilbrook), #625 (jradtilbrook)
Changed
- Docker configuration is now isolated per-step #678 (patrobinson) #756 (yob)
- Use EC2 LaunchTemplate instead of a LaunchConfiguration #589 (lox)
- InstanceType default is now
t3.large
(wast2.nano
) #699 (pda) - Disable AZRebalancing to prevent running instances being terminated unnecessarily #751
Dependencies updated
- Bump Buildkite Agent to v3.25.0 #726 (JuanitoFatas)
- Bump Buildkite Agent Scaler to v1.0.2 #724 (JuanitoFatas) 4fafd8e (JuanitoFatas)
- Bump docker to v19.03.13 (linux) and v19.03.12 (windows) and docker-compose to v1.27.4 (linux, windows uses latest choco version) #719 (yob) #723 (JuanitoFatas)
- Bump bundled plugins to the latest versions: secrets, ecr and docker login
And much, much more, we recommend you view the full changelog for this release.
Upgrading
In most cases, you should be able to upgrade in-place older versions of the stack with a CloudFormation stack update using the following template URL:
https://s3.amazonaws.com/buildkite-aws-stack/v5.0.0/aws-stack.yml
If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):
If you were using them, the following parameters have been removed or reworked:
EnableExperimentalLambdaBasedAutoscaling
was removed (it's the default now)BuildkiteOrgSlug
was removed – the information reported by buildkite-agent-scaler make it redundant, but consider buildkite-agent-metrics if you need more detailed metric monitoring that supports multiple metric backendsBuildkiteTerminateInstanceAfterJobTimeout
,ScaleDownPeriod
andScaleCooldownPeriod
are replaced by the more conciseScaleInIdlePeriod
#586 (jeremiahsnapp) which lets agents self-terminate to scale in when they've been idle after a set periodBuildkiteTerminateInstanceAfterJobDecreaseDesiredCapacity
andScaleDownAdjustment
were removed - instances will now always try to decrement the ASG desired count when their waiting period for new jobs has elapsedScaleUpAdjustment
is replaced byScaleOutFactor
as the new lambda scaler calculates how many agents are needed at the time.ScaleOutFactor
allows you to multiply the relative quantity provisioned to the ASG, either to slow it down or speed it up
Documentation
Elastic CI Stack for AWS v5.0.0 Beta 1
This is a beta release. Most things should work, but it's possible there are undiscovered issues in the release and we recommend continuing to use our stable v4.5.0 release for critical workloads. If you do spot any problems, we'd really appreciate it if you submit a GitHub issue or reply to our forum thread for this beta release where we will offer continued support.
What's new?
Previously experimental Lambda-based scaler is the new default 🚀
Our previously experimental fast autoscaler that scales up much faster is now always used. Instances will automatically scale to demand using the stack defaults and you should see good results without changing any parameters.
Experimental Windows support is now available
A new AMI built on Windows Server 2019 is optionally available to use in the stack for Windows builds. You can enable this by changing the InstanceOperatingSystem
setting from the default linux
to windows
. We've observed issues with Windows instances when MinSize is set above 0, so please avoid that for now. Huge thanks to @jeremiahsnapp and @tduffield for their contribution here!
Added
- Support for buildkite/image-builder which can enable you to customize AMIs based off the ones we ship #692 (keithduncan)
- Support for multiple security groups on instances #667 (jdub)
- AMI and Lambda Scaler support more regions: ap-east-1 (Hong Kong), me-south-1 (Bahrain), af-south-1 (Cape Town), eu-south-1 (Milan) #718 (JuanitoFatas)
- Support for loading BuildkiteAgentTokenPath from AWS Parameter Store #601 (jradtilbrook), #625 (jradtilbrook)
Changed
- Use EC2 LaunchTemplate instead of a LaunchConfiguration #589 (lox)
- InstanceType default is now
t3.large
(wast2.nano
) #699 (pda)
Dependencies updated
- Bump Buildkite Agent to v3.24.0 #726 (JuanitoFatas)
- Bump Buildkite Agent Scaler to v1.0.1 #724 (JuanitoFatas)
- Bump docker to v19.03.13 (linux) and v19.03.12 (windows) and docker-compose to v1.27.4 (linux, windows uses latest choco version) #719 (yob) #723 (JuanitoFatas)
And much, much more, we recommend you view the full changelog for this release.
Upgrading
Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v5.0.0-beta1/aws-stack.yml
If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):
Documentation
Elastic CI Stack for AWS v4.5.0
Changed
- Added ImageIdParameter CloudFormation parameter for SSM Parameter Store image lookup #691 (@keithduncan)
Upgrading
Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v4.5.0/aws-stack.yml
If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):