Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CirrusCI change in config format #3825

Closed
wants to merge 65 commits into from
Closed
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
398147c
Initial star test
SeanTAllen Jul 23, 2021
30800b7
Fuller
SeanTAllen Jul 23, 2021
a96278b
Couldnt parse yml
SeanTAllen Jul 23, 2021
3742ece
Empty yml is bad apparently
SeanTAllen Jul 23, 2021
b5f85b4
Is the problem in the .star?
SeanTAllen Jul 23, 2021
846247c
Still broken
SeanTAllen Jul 23, 2021
54460c7
roll way back
SeanTAllen Jul 23, 2021
5f9aedb
Add in cache_info
SeanTAllen Jul 23, 2021
bfd8c5a
without invalid cache check
SeanTAllen Jul 23, 2021
dfa2cbb
Still broken
SeanTAllen Jul 23, 2021
f3d060e
Still broken 2
SeanTAllen Jul 23, 2021
d009425
without hash
SeanTAllen Jul 23, 2021
c7d0919
load hash
SeanTAllen Jul 23, 2021
544edee
call hash.md5
SeanTAllen Jul 23, 2021
01fd362
file contents as hash
SeanTAllen Jul 23, 2021
4f6c8c1
what about a str() call
SeanTAllen Jul 23, 2021
f9bd6c2
full cache key
SeanTAllen Jul 23, 2021
111a8fe
add cache handling
SeanTAllen Jul 23, 2021
273a1f0
empty .yml
SeanTAllen Jul 23, 2021
fd68fea
More resources
SeanTAllen Jul 23, 2021
2b758a2
Two tests
SeanTAllen Jul 23, 2021
c70d8cd
Two tests 2
SeanTAllen Jul 23, 2021
72d9b3d
Two tests 3
SeanTAllen Jul 23, 2021
d3fcc62
Two tests 4
SeanTAllen Jul 23, 2021
092fe01
Two tests 5
SeanTAllen Jul 23, 2021
fa6beaa
Two tests 6
SeanTAllen Jul 23, 2021
058dd38
Two tests 7
SeanTAllen Jul 23, 2021
c39bed2
Two tests 8
SeanTAllen Jul 23, 2021
5d2c457
Two tests 9
SeanTAllen Jul 23, 2021
441d853
Put yml back
SeanTAllen Jul 23, 2021
4da8c49
Break out scripts
SeanTAllen Jul 23, 2021
320f204
Add nightly task creation
SeanTAllen Jul 23, 2021
5624ba2
Slightly less nightly
SeanTAllen Jul 23, 2021
69b0e16
Don't double up task names
SeanTAllen Jul 23, 2021
87e1fce
No debug tasks
SeanTAllen Jul 23, 2021
4caeec1
Fix script function names
SeanTAllen Jul 23, 2021
2cea8c8
Add debug task creation back
SeanTAllen Jul 23, 2021
04678e8
Add release process task creation
SeanTAllen Jul 23, 2021
5df8300
Less re
SeanTAllen Jul 23, 2021
32925f6
No syntax error re
SeanTAllen Jul 23, 2021
d198865
differet release task check
SeanTAllen Jul 23, 2021
1511a4c
a little different
SeanTAllen Jul 23, 2021
6c04f63
unique task names
SeanTAllen Jul 23, 2021
3f093ec
bare findall check
SeanTAllen Jul 23, 2021
1322b22
no findall
SeanTAllen Jul 23, 2021
0d7eb92
findall variation
SeanTAllen Jul 23, 2021
e9aeb57
minimum findall
SeanTAllen Jul 23, 2021
9b6d3a0
bigger findall
SeanTAllen Jul 23, 2021
7f9b5bd
re change
SeanTAllen Jul 23, 2021
3d7fa80
another re change
SeanTAllen Jul 23, 2021
520f4bb
another re change
SeanTAllen Jul 23, 2021
27dbc11
no dot
SeanTAllen Jul 23, 2021
65853ea
with dot
SeanTAllen Jul 23, 2021
a13b121
with \
SeanTAllen Jul 23, 2021
5d9ed86
with \ in single
SeanTAllen Jul 23, 2021
51e0f5e
\d
SeanTAllen Jul 23, 2021
1b70b22
Remove re that makes it angry
SeanTAllen Jul 24, 2021
296e212
Remove regular
SeanTAllen Jul 24, 2021
3f4c1f6
Make debug config PR tasks depend on release config PR tasks
SeanTAllen Jul 24, 2021
b8bda59
Correct name for fingerprint_key
SeanTAllen Jul 24, 2021
b3d3be7
Turn regular back on
SeanTAllen Jul 26, 2021
8520117
Fix regex
SeanTAllen Jul 28, 2021
8387f47
No str for hash
SeanTAllen Jul 28, 2021
24a43c5
Revert "No str for hash"
SeanTAllen Jul 28, 2021
7087a89
Remove needed for bugfix fix
SeanTAllen Aug 15, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
140 changes: 140 additions & 0 deletions .cirrus.star
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
load("cirrus", "env", "fs", "hash", "re")

def main():
if env.get("CIRRUS_PR", "") != "":
return create_pr_tasks()

if env.get("CIRRUS_CRON", "") == "nightly":
return create_nightly_tasks()

cirrus_tag = env.get("CIRRUS_TAG", "")
re.findall(r'^\d+\.\d+\.\d+$', cirrus_tag)
return create_release_tasks()
Comment on lines +4 to +12
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decide which tasks to create. PR, Nightly, or Release.


def create_pr_tasks():
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is full and complete. Release and Nightly aren't yet.

tasks = []

pr_tasks = linux_tasks()
# release tasks
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"release" task here is release config

for t in pr_tasks:
release = linux_pr_task(t['name'],
t['image'],
t['triple_vendor'],
t['triple_os']
)

# finish release task creation
release.update(linux_pr_scripts().items())
tasks.append(release)

# debug tasks
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

create a debug config test as well

for t in pr_tasks:
debug = linux_pr_task(t['name'] + ' [debug]',
t['image'],
t['triple_vendor'],
t['triple_os']
)

# finish debug task creation
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

debug only runs if non-debug passed.

debug.update(linux_pr_scripts("debug").items())
debug['depends_on'] = [t['name']]
tasks.append(debug)

return tasks

def create_nightly_tasks():
tasks = []

pr_tasks = linux_tasks()
for t in pr_tasks:
nightly = linux_pr_task(t['name'] + ' nightly',
t['image'],
t['triple_vendor'],
t['triple_os']
)

# finish nightly task creation
nightly.update(linux_nightly_scripts().items())

nightly['env']['CLOUDSMITH_API_KEY'] = 'ENCRYPTED[!2cb1e71c189cabf043ac3a9030b3c7708f9c4c983c86d07372ae58ad246a07c54e40810d038d31c3cf3ed8888350caca!'

tasks.append(nightly)

return tasks

def create_release_tasks():
tasks = []

pr_tasks = linux_tasks()
for t in pr_tasks:
release = linux_pr_task(t['name'] + ' release',
t['image'],
t['triple_vendor'],
t['triple_os']
)

# finish release task creation
release.update(linux_release_scripts().items())

release['env']['CLOUDSMITH_API_KEY'] = 'ENCRYPTED[!2cb1e71c189cabf043ac3a9030b3c7708f9c4c983c86d07372ae58ad246a07c54e40810d038d31c3cf3ed8888350caca!'

tasks.append(release)

return tasks

def linux_tasks():
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this would be where new tasks get added. anything in the list of maps below would have configuration for PR, nightly and release built.

return [
{ 'name': 'x86-64-unknown-linux-rocky8',
'image': 'ponylang/ponyc-ci-x86-64-unknown-linux-rocky8-builder:20210707',
'triple_vendor': 'unknown',
'triple_os': 'linux-rocky8'
},
{ 'name': 'x86-64-unknown-linux-centos8',
'image': 'ponylang/ponyc-ci-x86-64-unknown-linux-centos8-builder:20210225',
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we changed the builder image, it would be updated here.

note, no cache-buster value to update separately. this means you cant do what i have done before and forget to update it. the downside is, there is no way to manually bust the cache; however, we've never really needed to do that.

'triple_vendor': 'unknown',
'triple_os': 'linux-centos8'
}
]

def linux_pr_task(name, image, triple_vendor, triple_os):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be renamed as its not just for pr.

cache_key = hash.md5(fs.read('lib/CMakeLists.txt') + image)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the cache key is now the hash of the lib/CMakeLists.txt and the docker image name. definitely simpler than the current setup in .cirrus.yml


cpu = 8
memory = 24

task = {
'name': name,
'container': {
'image': image,
'cpu': cpu,
'memory': memory
},
'env': {
'TRIPLE_VENDOR': triple_vendor,
'TRIPLE_OS': triple_os
},
Comment on lines +112 to +115
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the env is only needed for nightly and release tasks but there's no harm in having it for pr as well.

it makes everything less complicated to have it here.

'libs_cache': {
'folder': 'build/libs',
'fingerprint_key': cache_key,
'populate_script': 'make libs build_flags=-j8'
},
}

return task

def linux_pr_scripts(config="release"):
return {
'configure_script': 'make configure config=' + config,
'build_script': 'make build build_flags=-j8 config=' + config,
'test_script': 'make test-ci config=' + config
}

def linux_nightly_scripts():
return {
'nightly_script': 'bash .ci-scripts/x86-64-nightly.bash'
}

def linux_release_scripts():
return {
'release_script': 'bash .ci-scripts/x86-64-release.bash'
}
Comment on lines +125 to +140
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the only "real" difference between the release, nightly, and pr is the script to run.

Loading