Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Textual create #2404

Merged
merged 129 commits into from
May 14, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
129 commits
Select commit Hold shift + click to select a range
513de95
WIP: First stab at writing some Textual
ewels Aug 20, 2023
0e8204e
Refactor: Break into multiple files
ewels Aug 21, 2023
e37d2e5
Remove unusued imports
ewels Aug 21, 2023
ee12eb5
show error message when pressing enter
mirpedrol Aug 22, 2023
dd18bcc
show failure messages when button is pressed
mirpedrol Aug 22, 2023
486f180
add usegenomicdata screen (TODO on_button_pressed)
mirpedrol Aug 22, 2023
5b8cfd5
add custom pipeline switch table to select pipeline features
mirpedrol Aug 23, 2023
0a40dab
create new widget PipelineFeature to display feature buttons and help…
mirpedrol Aug 24, 2023
39b9e3c
add more custom features
mirpedrol Aug 24, 2023
25c9fa1
add template features to skip to TEMPLATE_CONFIG
mirpedrol Aug 25, 2023
610cbd9
animate help box
mirpedrol Aug 28, 2023
6677144
add type_nfcore screen and use PipelineCreate to create a pipeline
mirpedrol Aug 29, 2023
8bac0be
modify PipelineCreate to accept a template dictionary
mirpedrol Aug 29, 2023
43db7f6
start modifying nf-core create command to use TUI and not CLI
mirpedrol Aug 30, 2023
99f426a
small bug fixes and add a deprecation message for nf-core create command
mirpedrol Aug 31, 2023
09bd5db
create command fails if one but not all arguments are provided
mirpedrol Aug 31, 2023
9f121f7
ask if launching TUI when using deprecated nf-core create command
mirpedrol Aug 31, 2023
0c77029
add final_details screen for version and force
mirpedrol Aug 31, 2023
bf9a541
add nfcorepipeline grid with features to select
mirpedrol Aug 31, 2023
fec5e3e
first ask if the pipeline will be nf-core or custom before basicdetails
mirpedrol Aug 31, 2023
0ef16b5
use kwargs
mirpedrol Aug 31, 2023
238767f
remove inexisten variable version from send-tweet github workflow
mirpedrol Aug 21, 2023
7b4a342
fix typo in gh_badges jinja variable to github_badges
mirpedrol Aug 24, 2023
b3fa25e
bump version of nf-test snap files
mirpedrol Aug 30, 2023
6ff69b2
update changelog
mirpedrol Aug 30, 2023
17ba971
add outdir textinput and update config
mirpedrol Sep 1, 2023
0c64de8
refactor jinja template 'branded' to 'is_nfcore'
mirpedrol Sep 1, 2023
7720117
refactor params_dict to jinja_params and use that instead of the config
mirpedrol Sep 1, 2023
cb62be3
refactor function obtain_jinja_params_dict
mirpedrol Sep 1, 2023
da1c68f
run create command from the App, mv nf_core/create.py to nf_core/pipe…
mirpedrol Sep 1, 2023
3c14dd7
add new screen to create a github repo and push the new pipeline (not…
mirpedrol Oct 3, 2023
e401a6a
Some minor CSS tweaks
ewels Nov 2, 2023
8b9ad35
Add logging handler and final screen to visualise logs
mirpedrol Nov 3, 2023
4c237e2
fix githubrepo buttons handled by the same function
mirpedrol Nov 3, 2023
8318e0c
fix pytests
mirpedrol Nov 6, 2023
c7e057d
Merge branch 'dev' of https://github.com/nf-core/tools into textual-c…
mirpedrol Nov 6, 2023
c2c0561
fix test_remove_patch
mirpedrol Nov 6, 2023
ef9ea81
add types-requests to requirements-dev
mirpedrol Nov 6, 2023
798dad9
add full logging screen to show logging messages
mirpedrol Nov 10, 2023
4758c7d
autopopulate github credentials and hide password
mirpedrol Nov 13, 2023
665d378
add button to show or hide password
mirpedrol Nov 13, 2023
19f60bc
update textual and textual-dev versions
mirpedrol Nov 14, 2023
887f372
add first snapshot tests
mirpedrol Nov 14, 2023
7758698
add snapshots for all screens
mirpedrol Dec 5, 2023
0dbe982
Merge branch 'dev' of https://github.com/nf-core/tools into textual-c…
mirpedrol Dec 5, 2023
9c25603
Merge branch 'dev' of https://github.com/nf-core/tools into textual-c…
mirpedrol Jan 5, 2024
3908163
ruff code modifications
mirpedrol Jan 5, 2024
208d832
update snapshots
mirpedrol Jan 5, 2024
bebf067
ask if we have to create a github repo in a different screen
mirpedrol Jan 5, 2024
7ade926
add snapshot for github repo question and exit message
mirpedrol Jan 5, 2024
7049277
add work threads for creating a pipeline and a github repo
mirpedrol Jan 5, 2024
41fcc27
add loading screen when creating a pipeline
mirpedrol Jan 5, 2024
cfa0804
show logging only at the end
mirpedrol Jan 5, 2024
a85f1dd
add loading screen for repo creation
mirpedrol Jan 5, 2024
456a4be
update app snapshots
mirpedrol Jan 5, 2024
bb6cc97
fix typing error
mirpedrol Jan 8, 2024
abdffef
ignroe snapshot files with editorconfig
mirpedrol Jan 8, 2024
2e396d9
add exclusive=True to work threads
mirpedrol Jan 8, 2024
3329411
remove logging and add completed screen instead
mirpedrol Jan 8, 2024
f5551bc
Merge branch 'dev' of https://github.com/nf-core/tools into textual-c…
mirpedrol Jan 8, 2024
cbce046
show an error message if the pipeline already exists
mirpedrol Jan 8, 2024
62c4b3e
fix nf-core.pipelines.create import
mirpedrol Jan 8, 2024
f04ad5e
GHA use new nf-core pipelines create command and fixed default org
mirpedrol Jan 8, 2024
baa3412
fix pytests
mirpedrol Jan 8, 2024
acd7201
add loading messages
mirpedrol Jan 9, 2024
6b4ed9c
default version to 1.0.0dev
mirpedrol Jan 9, 2024
97bca8b
add logging instead of loading screen and see logging messages on rea…
mirpedrol Jan 19, 2024
0525d9c
deprecation error if 'nf-core create' is used
mirpedrol Feb 19, 2024
29afad0
more formatting and docs
mirpedrol Feb 19, 2024
2dcb0f1
add button to go back
mirpedrol Feb 19, 2024
17bb9e9
disable button to close logs while pipeline is not created
mirpedrol Feb 19, 2024
5445b74
more tweaking of logging buttons and screen
mirpedrol Feb 20, 2024
4ecb445
Merge branch 'dev' of https://github.com/nf-core/tools into textual-c…
mirpedrol Feb 21, 2024
536e3be
update create app tests according to last changes
mirpedrol Feb 21, 2024
599d2e0
Merge branch 'dev' into textual-create
ewels Mar 4, 2024
2943098
some fixes after merging dev to textual-create branch
mirpedrol Mar 6, 2024
3cfdc78
fix packaging. Thanks @mashehu for the debugging help
mirpedrol Mar 6, 2024
e7687e9
add changed from dev branch to create.py script
mirpedrol Mar 7, 2024
1debc4b
fix back button after validation of basic details
mirpedrol Mar 7, 2024
41e99a1
move button to close logging screen to the bottom
mirpedrol Mar 7, 2024
67db794
Merge branch 'dev' of https://github.com/nf-core/tools into textual-c…
mirpedrol Mar 7, 2024
a4a67bb
Fix merge conflicts
ewels Apr 2, 2024
ed70990
Remove duplicate lines in changelog from merge conflict resolution
ewels Apr 2, 2024
3a54023
Raw strings for nf-core ascii artwork
ewels Apr 2, 2024
bfaf3b1
Simplify log messages about CLI flags
ewels Apr 2, 2024
ea99d69
Move logo to utils, fix alignment
ewels Apr 2, 2024
d89218f
Tweaks for welcome and pipeline type pages
ewels Apr 2, 2024
19a6144
Display tweaks
ewels Apr 2, 2024
0bfae29
Tidy up design for GitHub repo creation page
ewels Apr 2, 2024
a1d8916
[automated] Fix code linting
nf-core-bot Apr 3, 2024
2d7c4e0
Merge branch 'dev' of github.com:nf-core/tools into textual-create
mashehu Apr 3, 2024
7a96212
update create app snapshots
mirpedrol Apr 3, 2024
eaedeb7
fix providing a prefix twitha template yaml
mirpedrol Apr 3, 2024
682d9b3
update error message in create pipeline test
mirpedrol Apr 3, 2024
3aa383c
not show pipeline logging when creating a repo
mirpedrol Apr 3, 2024
d8283bb
fix pipeline create tests
mirpedrol Apr 3, 2024
241bc20
use org instead of prefix in pipeline template
mirpedrol Apr 3, 2024
6f2a59b
add 'back' button if the pipeline exists
mirpedrol Apr 5, 2024
24704a7
add option to set repo name
mirpedrol Apr 5, 2024
5add94b
show help if repo already exists
mirpedrol Apr 5, 2024
5d9ebbd
hide buttons not used in logging screen
mirpedrol Apr 5, 2024
5a0aa46
fix threads and messages
mirpedrol Apr 5, 2024
48d6ea1
update test snapshots
mirpedrol Apr 5, 2024
01481db
allow selecting a github organisation
mirpedrol Apr 5, 2024
33a97bb
Some tweaks to text strings
ewels Apr 8, 2024
c616f91
Merge branch 'textual-create' into textual-create-changes
ewels Apr 8, 2024
1ce788f
Merge pull request #6 from mirpedrol/textual-create-changes
ewels Apr 8, 2024
1503b12
Whitespace etc
ewels Apr 8, 2024
0acb2e2
push s allcreens, not switch
mirpedrol Apr 11, 2024
5fcedcc
always push files to the remote repo
mirpedrol Apr 11, 2024
c2381ae
fix going back from nf-core to custom pipeline
mirpedrol Apr 11, 2024
30f6c5a
fix logging screen by using a class to hide buttons
mirpedrol Apr 11, 2024
bf14b4e
remove force switch and always force, show warning
mirpedrol Apr 12, 2024
b1412cf
add warning if github org not found
mirpedrol Apr 12, 2024
e3b6d44
Merge branch 'dev' of https://github.com/nf-core/tools into textual-c…
mirpedrol Apr 12, 2024
7cc3757
update snapshots
mirpedrol Apr 12, 2024
0bd1807
fix providing outdir
mirpedrol Apr 15, 2024
9976440
try fixing pytests by using tmpdir instead of mock
mirpedrol Apr 15, 2024
777f68c
fix outdir again to fix tests
mirpedrol Apr 16, 2024
e7d3886
debugging ci tests
mashehu Apr 17, 2024
6b83de8
try fixing outdir, again
mirpedrol Apr 17, 2024
e1a12f3
Merge branch 'dev' of https://github.com/nf-core/tools into textual-c…
mirpedrol May 9, 2024
dcc7757
update snapshots
mirpedrol May 9, 2024
8aceeb3
handle token not configured with gh
mirpedrol May 9, 2024
9572b23
add asyncio to pytest.ini and upload snapshot report on fail on CI
mirpedrol May 10, 2024
bc1096f
wait for workers to complete
mirpedrol May 10, 2024
af584d3
upload snapshot report always
mirpedrol May 10, 2024
0de5b69
upload snapshot report first
mirpedrol May 10, 2024
9d6838c
update snapshot from gitpod
mirpedrol May 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions nf_core/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -488,6 +488,49 @@ def lint(ctx, dir, release, fix, key, show_passed, fail_ignored, fail_warned, ma
sys.exit(1)


# nf-core pipelines subcommands
@nf_core_cli.group()
@click.pass_context
def pipelines(ctx):
"""
Commands to manage nf-core pipelines.
"""
# ensure that ctx.obj exists and is a dict (in case `cli()` is called
# by means other than the `if` block below)
ctx.ensure_object(dict)


# nf-core pipeline install
@pipelines.command("create")
@click.pass_context
@click.option(
"-n",
"--name",
type=str,
help="The name of your new pipeline",
)
@click.option("-d", "--description", type=str, help="A short description of your pipeline")
@click.option("-a", "--author", type=str, help="Name of the main author(s)")
@click.option("--version", type=str, default="1.0dev", help="The initial version number to use")
mirpedrol marked this conversation as resolved.
Show resolved Hide resolved
@click.option("-f", "--force", is_flag=True, default=False, help="Overwrite output directory if it already exists")
@click.option("-o", "--outdir", help="Output directory for new pipeline (default: pipeline name)")
@click.option("-t", "--template-yaml", help="Pass a YAML file to customize the template")
@click.option("--plain", is_flag=True, help="Use the standard nf-core template")
def create_pipeline(ctx, name, description, author, version, force, outdir, template_yaml, plain):
"""
Create a new pipeline using the nf-core template.

Uses the nf-core template to make a skeleton Nextflow pipeline with all required
files, boilerplate code and best-practices.
\n\n
Run without any command line arguments to use an interactive interface.
"""
from nf_core.pipelines.create import PipelineCreateApp

app = PipelineCreateApp()
app.run()


# nf-core modules subcommands
@nf_core_cli.group()
@click.option(
Expand Down
44 changes: 44 additions & 0 deletions nf_core/pipelines/create/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
"""A Textual app to create a pipeline."""
from textual.app import App
from textual.widgets import Button

from nf_core.pipelines.create.utils import CreateConfig
from nf_core.pipelines.create.welcome import WelcomeScreen
from nf_core.pipelines.create.basicdetails import BasicDetails
from nf_core.pipelines.create.pipelinetype import ChoosePipelineType


class PipelineCreateApp(App):
"""A Textual app to manage stopwatches."""

CSS_PATH = "create.tcss"
TITLE = "nf-core create"
SUB_TITLE = "Create a new pipeline with the nf-core pipeline template"
BINDINGS = [
("d", "toggle_dark", "Toggle dark mode"),
("q", "quit", "Quit"),
]
SCREENS = {
"welcome": WelcomeScreen(),
"basic_details": BasicDetails(),
"choose_type": ChoosePipelineType(),
}

# Initialise config as empty
TEMPLATE_CONFIG = CreateConfig()

def on_mount(self) -> None:
self.push_screen("welcome")

def on_button_pressed(self, event: Button.Pressed) -> None:
"""Handle all button pressed events."""
if event.button.id == "start":
self.switch_screen("basic_details")
elif event.button.id == "type_nfcore":
self.switch_screen("type_nfcore")
elif event.button.id == "type_custom":
self.switch_screen("type_custom")

def action_toggle_dark(self) -> None:
"""An action to toggle dark mode."""
self.dark = not self.dark
67 changes: 67 additions & 0 deletions nf_core/pipelines/create/basicdetails.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
"""A Textual app to create a pipeline."""
from textual import on
from textual.app import ComposeResult
from textual.screen import Screen
from textual.containers import Horizontal, Center
from textual.widgets import Button, Footer, Header, Markdown, Input
from textwrap import dedent

from nf_core.pipelines.create.utils import CreateConfig, TextInput


class BasicDetails(Screen):
"""Name, description, author, etc."""

def compose(self) -> ComposeResult:
yield Header()
yield Footer()
yield Markdown(
dedent(
"""
# Basic details
"""
)
)
with Horizontal():
yield TextInput(
"org",
"Organisation",
"GitHub organisation",
"nf-core",
classes="column",
)
yield TextInput(
"name",
"Pipeline Name",
"Workflow name",
classes="column",
)

yield TextInput(
"description",
"Description",
"A short description of your pipeline.",
)
yield TextInput(
"author",
"Author(s)",
"Name of the main author / authors",
)
yield Center(
Button("Next", variant="success"),
classes="cta",
)

@on(Button.Pressed)
def on_button_pressed(self, event: Button.Pressed) -> None:
"""Save fields to the config."""
config = {}
for text_input in self.query("TextInput"):
this_input = text_input.query_one(Input)
this_input.validate(this_input.value)
config[text_input.field_id] = this_input.value
try:
self.parent.TEMPLATE_CONFIG = CreateConfig(**config)
self.parent.switch_screen("choose_type")
except ValueError:
pass
32 changes: 32 additions & 0 deletions nf_core/pipelines/create/create.tcss
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#logo {
text-align:center;
}
.cta {
layout: horizontal;
margin-bottom: 1;
}
.cta Button {
margin-left: 3;
margin-right: 3;
}

.field_help {
padding: 1 1 0 1;
color: $text-muted;
text-style: italic;
}
.validation_msg {
padding: 0 1;
color: $error;
}
.-valid {
border: tall $success-darken-3;
}

Horizontal{
width: 100%;
height: auto;
}
.column {
width: 1fr;
}
49 changes: 49 additions & 0 deletions nf_core/pipelines/create/pipelinetype.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
from textual.app import ComposeResult
from textual.screen import Screen
from textual.containers import Center
from textual.widgets import Button, Footer, Header, Markdown

markdown_intro = """
# To nf-core or not to nf-core?

Next, we need to know what kind of pipeline this will be.

Choose _"nf-core"_ if:

* You want your pipeline to be part of the nf-core community
* You think that there's an outside chance that it ever _could_ be part of nf-core

Choose _"Custom"_ if:

* Your pipeline will _never_ be part of nf-core
* You want full control over *all* features that are included from the template
(including those that are mandatory for nf-core).
"""

markdown_details = """
## Not sure? What's the difference?

Choosing _"nf-core"_ effectively pre-selects the following template features:

* GitHub Actions Continuous Integration (CI) configuration for the following:
* Small-scale (GitHub) and large-scale (AWS) tests
* Code format linting with prettier
* Auto-fix functionality using @nf-core-bot
* Marking old issues as stale
* Inclusion of shared nf-core config profiles
"""


class ChoosePipelineType(Screen):
"""Choose whether this will be an nf-core pipeline or not."""

def compose(self) -> ComposeResult:
yield Header()
yield Footer()
yield Markdown(markdown_intro)
yield Center(
Button("nf-core", id="type_nfcore", variant="success"),
Button("Custom", id="type_custom", variant="primary"),
classes="cta",
)
yield Markdown(markdown_details)
91 changes: 91 additions & 0 deletions nf_core/pipelines/create/utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
from pydantic import BaseModel, field_validator
import re
from typing import Optional
from textual import on
from textual.app import ComposeResult
from textual.validation import Validator, ValidationResult
from textual.widgets import Static, Input


class CreateConfig(BaseModel):
"""Pydantic model for the nf-core create config."""

org: Optional[str] = None
name: Optional[str] = None
description: Optional[str] = None
author: Optional[str] = None
version: Optional[str] = None
force: Optional[bool] = None
outdir: Optional[str] = None
template_yaml: Optional[str] = None
is_nfcore: Optional[bool] = None

@field_validator("name")
@classmethod
def name_nospecialchars(cls, v: str) -> str:
"""Check that the pipeline name is simple."""
if not re.match(r"^[a-z]+$", v):
raise ValueError("Must be lowercase without punctuation.")
return v

@field_validator("org", "description", "author")
@classmethod
def notempty(cls, v: str) -> str:
"""Check that string values are not empty."""
if v.strip() == "":
raise ValueError("Cannot be left empty.")
return v


class TextInput(Static):
"""Widget for text inputs.

Provides standard interface for a text input with help text
and validation messages.
"""

def __init__(self, field_id, placeholder, description, default=None, **kwargs) -> None:
"""Initialise the widget with our values.

Pass on kwargs upstream for standard usage."""
super().__init__(**kwargs)
self.field_id: str = field_id
self.placeholder: str = placeholder
self.description: str = description
self.default: str = default

def compose(self) -> ComposeResult:
yield Static(self.description, classes="field_help")
yield Input(
placeholder=self.placeholder,
validators=[ValidateConfig(self.field_id)],
value=self.default,
)
yield Static(classes="validation_msg")

@on(Input.Changed)
def show_invalid_reasons(self, event: Input.Changed) -> None:
"""Validate the text input and show errors if invalid."""
if not event.validation_result.is_valid:
self.query_one(".validation_msg").update("\n".join(event.validation_result.failure_descriptions))
else:
self.query_one(".validation_msg").update("")


class ValidateConfig(Validator):
"""Validate any config value, using Pydantic."""

def __init__(self, key) -> None:
"""Initialise the validator with the model key to validate."""
super().__init__()
self.key = key

def validate(self, value: str) -> ValidationResult:
"""Try creating a Pydantic object with this key set to this value.

If it fails, return the error messages."""
try:
CreateConfig(**{f"{self.key}": value})
return self.success()
except ValueError as e:
return self.failure(", ".join([err["msg"] for err in e.errors()]))
36 changes: 36 additions & 0 deletions nf_core/pipelines/create/welcome.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
from textual.app import ComposeResult
from textual.screen import Screen
from textual.containers import Center
from textual.widgets import Button, Footer, Header, Static, Markdown

markdown = """
# nf-core create

This app will help you create a new nf-core pipeline.
It uses the nf-core pipeline template, which is kept at
mirpedrol marked this conversation as resolved.
Show resolved Hide resolved
within the [nf-core/tools repository](https://github.com/nf-core/tools).

Using this tool is mandatory when making a pipeline that may
be part of the nf-core community collection at some point.
However, this tool can also be used to create pipelines that will
never be part of nf-core. You can still benefit from the community
best practices for your own workflow.
"""


class WelcomeScreen(Screen):
"""A welcome screen for the app."""

def compose(self) -> ComposeResult:
yield Header()
yield Footer()
yield Static(
f"\n[green]{' ' * 40},--.[grey39]/[green],-."
+ "\n[blue] ___ __ __ __ ___ [green]/,-._.--~\\"
+ "\n[blue]|\ | |__ __ / ` / \ |__) |__ [yellow] } {"
+ "\n[blue] | \| | \__, \__/ | \ |___ [green]\`-._,-`-,"
+ "\n[green] `._,._,'\n",
id="logo",
)
yield Markdown(markdown)
yield Center(Button("Let's go!", id="start", variant="success"), classes="cta")
1 change: 1 addition & 0 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ pytest-datafiles
responses
Sphinx
sphinx-rtd-theme
textual-dev>=1.1.0
4 changes: 3 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@ markdown>=3.3
packaging
pre-commit
prompt_toolkit>=3.0.3
pytest>=7.0.0
pydantic>=2.2.1
pytest-workflow>=1.6.0
pytest>=7.0.0
pyyaml
questionary>=1.8.0
refgenie
Expand All @@ -17,3 +18,4 @@ requests_cache
rich-click>=1.6.1
rich>=13.3.1
tabulate
textual>=0.33.0
Loading