Skip to content

Commit

Permalink
chore: merge branch 'master' into fix-argo-container
Browse files Browse the repository at this point in the history
  • Loading branch information
jmgilman committed Oct 30, 2024
2 parents e4c73cc + 2c8a7d3 commit d878806
Show file tree
Hide file tree
Showing 19 changed files with 864 additions and 213 deletions.
16 changes: 15 additions & 1 deletion docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,20 @@ docs_dir: src

nav:
- Overview: index.md
- News: blog/index.md
- Installation: installation.md
- Tutorials:
- Getting Started: tutorials/getting_started.md
- Concepts:
- Projects: concepts/projects.md
- CI: concepts/ci.md
- Reference:
- Blueprints: reference/blueprint.md
- Deployments: reference/deployments.md
- Releases:
- Overview: reference/releases/index.md
- Docker: reference/releases/docker.md
- GitHub: reference/releases/github.md
- Targets: reference/targets.md

theme:
Expand Down Expand Up @@ -45,4 +53,10 @@ markdown_extensions:
- admonition
- pymdownx.details
- pymdownx.snippets
- pymdownx.superfences
- pymdownx.superfences

plugins:
- blog:
archive: false
categories: false
- tags
5 changes: 5 additions & 0 deletions docs/src/blog/.authors.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
authors:
jmgilman:
name: Joshua Gilman
description: Maintainer
avatar: https://avatars.githubusercontent.com/u/2308444?v=4
2 changes: 2 additions & 0 deletions docs/src/blog/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# News

147 changes: 147 additions & 0 deletions docs/src/blog/posts/001-whats-new-in-forge.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
---
draft: false
date: 2024-10-25
authors:
- jmgilman
---

# What's New in Forge - 10-25-2024

Check out what's new in Forge this week.

<!-- more -->

## Releases

The `publish` and `release` targets are no more!
They have been replaced with an entirely new system that will enable adding more release automation going forward.
Individual releases are now defined in a project's blueprint and Forge will automatically discover and execute them in the CI
pipeline.
Each release is run in parallel to maximize speed.

The old targets will no longer automatically run in the CI.
You will need to configure new releases in your project's blueprint file to continue publishing/releasing.
The `publish` target has been replaced with the `docker` release type.
The `release` target has been replaced with the `github` release type.

For example, you can continue to use the `publish` target in your `Earthfile` by configuring a `docker` release type:

```cue
project: {
name: "myproject"
release: {
docker: {
on: {
merge: {}
tag: {}
}
config: {
tag: _ @forge(name="GIT_COMMIT_HASH")
}
target: "publish"
}
}
}
```

The above configuration will create a new docker release whenever a merge to the default branch occurs or when a new git tag is
created.
The published image will have its tag (`config.tag` above) automatically filled in with the git commit hash.
Finally, Forge will call the `publish` target in your `Earthfile` to generate the container image.

To learn more about releases, please refer to the [reference documentation](../../reference/releases/index.md).

## Deployment Templates

A new command has been introduced to the CLI: `forge deploy template`.
This command can be used to generate the raw Kubernetes manifests (in YAML) that will be applied to the target Kubernetes cluster
during automatic deployments.
This is useful when troubleshooting why a deplyoment may be failing or acting in an unexpected way.
All generated manifests will be printed to `stdout` and can be redirected to a local file for easier viewing.

The below example shows what it looks like to generate the raw manifests for the Foundry API server:

```text
$ forge deploy template foundry/api
---
# Instance: foundry-api
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/managed-by: timoni
app.kubernetes.io/name: foundry-api
app.kubernetes.io/version: 0.1.0
name: foundry-api
namespace: default
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
selector:
app.kubernetes.io/name: foundry-api
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/managed-by: timoni
app.kubernetes.io/name: foundry-api
app.kubernetes.io/version: 0.1.0
name: foundry-api
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: foundry-api
template:
metadata:
labels:
app.kubernetes.io/name: foundry-api
spec:
containers:
- image: 332405224602.dkr.ecr.eu-central-1.amazonaws.com/foundry-api:763fe7fd2bfdd39d630df9b5c5aa7e6588fc6eea
imagePullPolicy: IfNotPresent
livenessProbe:
httpGet:
path: /
port: http
initialDelaySeconds: 5
periodSeconds: 5
name: foundry-api
ports:
- containerPort: 8080
name: http
protocol: TCP
readinessProbe:
httpGet:
path: /
port: http
initialDelaySeconds: 5
periodSeconds: 5
```

For more information, please refer to the [deployments documentation](../../reference/deployments.md#templating).

## What's Next

Work is currenetly being done to improve automatic deployments for projects.
Currently, Forge assumes a GitOps repository exists and will automatically generate and commit updated deployments to the configured
repository.
This makes setup complicated and introduces a mostly unecessary step in the deployment process.

Instead, we are investigating having a GitOps operator (currently only Argo CD) point directly at a project's repository.
Since a blueprint file is self-contained, it's possible to generate Kubernetes manifests using only the information inside of it.
The first steps towards experimenting with this new solution was to create a
[custom management plugin](https://github.com/input-output-hk/catalyst-forge/tree/master/tools/argocd) capable of ingesting a
project and spitting out raw Kubernetes manifests.
With this in place, it should be possible to point Argo CD directly at a project folder and have it generate the necessary manifests
for deploying the project.
As this process matures, more documentation will be released with the updated deployment process.

That's it for this week, thanks for tuning in!
11 changes: 5 additions & 6 deletions docs/src/concepts/ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,14 @@ It then filters and orders these targets into discrete target groups using a lis
The name and dependency order of these groups is hardcoded and does not often change.

Each of these target groups can be considered _phases_ in the overall CI pipeline.
Each phase consists of the associated targets and each phase occur in dependency order.
Each phase consists of the associated targets and each phase occurs in dependency order.
The name and order of these phases is hardcoded and does not often change.

### Execution

For each phase, the CI system spawns a series of parallel jobs that executes each individual target.
Each target execution is given its own unique job in GitHub Actions to allow easy identification of failing targets as well as
providing an isolated log stream for a single target.
For each phase, the CI system spawns a series of parallel jobs that executes the current target for all discovered projects.
Each project execution is given its own unique job in GitHub Actions to allow easy identification of failing targets as well as
providing an isolated log stream.
If any target in the group fails, the entire group is considered to be failed, and CI execution stops.

!!! hint
Expand All @@ -37,13 +37,12 @@ If any target in the group fails, the entire group is considered to be failed, a
This promotes a "full ownership" philosophy where developers are responsible for ensuring their changes keep CI passing.

Projects are not required to define targets for each phase.
In some cases, a phase may only contain a subset of projects.
In some cases, a phase may only contain a subset of all projects in the repository.
If a phase ends up with zero targets, the entire job is skipped.
This allows repositories to define a small subset of targets initially and grow as project complexity increases.

Some target executions are limited to running the associated Earthly target and then immediately finishing.
Other targets have additional logic that is executed after the target finishes running.
For example, the `publish` target will automatically publish container images generated by the target to any configured registries.
For more information on supported targets, please see the [reference documentation](../reference/targets.md).

## Extending
Expand Down
Binary file modified docs/src/concepts/images/pipeline_dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/src/concepts/images/pipeline_light.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
60 changes: 60 additions & 0 deletions docs/src/concepts/projects.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Projects

The primary component of Catalyst Forge is the _project_.
Forge is designed to interact with monorepos, where each repository contains one or more projects.
A project can be classified as a discrete deliverable within a repository: it has own its dedicated process for validating,
building, testing, and potentially deploying.

## Organizing Projects

!!! tip
While there's no hierarchical order enforced by Forge, it's against best practices to have projects live at the root of a
repository.
The only exception to this case is where the repository only has a single deliverable.
Since the global blueprint at the root of a repository is always unified with project blueprints, trying to configure a project
in the global blueprint will result in overlapping values and will cause the parsing process to fail.

Catalyst Forge does not enforce projects live in any particular folder within the repository.
Developers are encouraged to organize the repository in whatever way makes sense for them.
The discovery mechanisms used by Forge will ensure that projects are found no matter where they live.

In some cases, projects may have dependencies on each other.
For example, one project may have a dependency on one or more projects that provide language libraries.
Whether or not these are treated as separate projects is up to developers.
If a library is used by more than one project, or is consumed externally, it's recommended to treat it as a separate project.

## Project Components

Forge discovers projects within a repository using a specific set of rules.
Namely, a valid project is any folder within the repository that contains a blueprint (`blueprint.cue`).
This is the _only_ requirement for forge to classify that directory as a project.
While a project may consist of one or more _other_ files or directories, the blueprint should always exist at the root of the
project folder.

Optionally, a project may also contain an `Earthfile` that contains definitions for the common targets used by the
[CI system](./ci.md).
The CI system automatically checks for the existence of this file after it discovers a project.
However, it's important to recognize the existence of an `Earthfile` _does not_ define a project according to Forge.

## Blueprints

A blueprint file contains the configuration for a project.
By convention, the blueprint file is named `blueprint.cue` and is placed at the root of the project folder.
A blueprint contains several options for configuring a project.
Please refer to the [reference documentation](../reference/blueprint.md) for more information.

In addition to project blueprint files, a _global_ blueprint file can also be provided at the root of the repository.
This blueprint configures global options that impact every project in the repository.
The final configuration always consists of a unification of the project and global blueprints.

## Tagging

When tagging a project, it's recommended to use the following format:

```
<project_name>/<version>
```

Various systems within Forge are configured to automatically detect and parse this tag structure.
For example, the `tag` event when configuring releases only triggers when a tag matching the current project name is found.
This structure also ensures that projects are versioned separately and are easy to identify when examining tags.
3 changes: 2 additions & 1 deletion docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,5 @@ It is especially inspired by the principles and philosophies of

## Getting Started

If you're new to Catalyst Forge, the quickest way to get familiar with what it offers is through the getting started tutorial.
If you're new to Catalyst Forge, the quickest way to get familiar with it is through the
[getting started tutorial](./tutorials/getting_started.md).
Loading

0 comments on commit d878806

Please sign in to comment.