Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ai-env virtual package; remove python variant from jedi-base-env; bug fixes for Discover SCU16/17; doc updates for ParallelWorks; etc #1017

Merged
merged 26 commits into from
Mar 8, 2024

Conversation

climbfuji
Copy link
Collaborator

@climbfuji climbfuji commented Mar 5, 2024

Summary

  • Set env variable I_MPI_EXTRA_FILESYSTEM='ON' for Derecho and Discover SCU17
  • Add diag-disable flag for deprecated Intel classic compilers for Hercules and Derecho
  • Add protobuf and py-protobuf to list of module excludes
  • Update README.md for noaa-aws, noaa-azure, noaa-gcloud
  • Update documentation for Discover SCU16/SCU17
  • Remove variant python from jedi-base-env
  • Add ai-env virtual package and set py-torch variants in configs/common/packages.yaml
  • Skip ai-env with Intel compilers in unified-dev template

Testing

  • CI
  • @climbfuji's macOS
  • AWS ParallelCluster
  • Discover SCU16 (ai-env not tested)
  • Discover SCU17 (ai-env not tested)

Applications affected

None

Systems affected

  • Discover
  • Hercules, Derecho due to minor compiler config updates
  • All systems when building ai-env with gcc or clang

Dependencies

n/a

Issue(s) addressed

Working towards #994
Working towards #1011 (two out of three ain't bad)
Resolves #1012

Checklist

  • This PR addresses one issue/problem/enhancement, or has a very good reason for not doing so.
  • These changes have been tested on the affected systems and applications.
  • All dependency PRs/issues have been resolved and this PR can be merged.

@climbfuji climbfuji force-pushed the bugfix/discover_scu16_ldflags branch from 6514285 to e3a3184 Compare March 5, 2024 03:46
@climbfuji climbfuji changed the title Add py-torch variant to jedi-base-env, bug fixes for Discover SCU16/17, doc updates for ParallelWorks Add py-torch variant to jedi-base-env and remove variant python; bug fixes for Discover SCU16/17; doc updates for ParallelWorks Mar 5, 2024
@climbfuji climbfuji force-pushed the bugfix/discover_scu16_ldflags branch from e89ccb9 to f81d7ee Compare March 5, 2024 20:33
@@ -1,42 +1,47 @@
compilers::
Copy link
Collaborator Author

@climbfuji climbfuji Mar 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except for lines 10-12 adding the annoying deprecation warnings, and line 27 adding the recently reintroduced I_MPI env var, the rest of the changes here are due to fixing the indentation only.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These I_MPI_* variables might be needed for building or running the UFS and Apps as well. Probably this is something to keep in mind

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So these will be set automatically whenever the stack-intel compiler gets loaded on Derecho - should be sufficient for UFS

@climbfuji climbfuji marked this pull request as ready for review March 7, 2024 20:13
@climbfuji climbfuji changed the title Add py-torch variant to jedi-base-env and remove variant python; bug fixes for Discover SCU16/17; doc updates for ParallelWorks Add ai-env virtual package; remove python variant from jedi-base-env; bug fixes for Discover SCU16/17; doc updates for ParallelWorks Mar 7, 2024
@climbfuji climbfuji changed the title Add ai-env virtual package; remove python variant from jedi-base-env; bug fixes for Discover SCU16/17; doc updates for ParallelWorks Add ai-env virtual package; remove python variant from jedi-base-env; bug fixes for Discover SCU16/17; doc updates for ParallelWorks; etc Mar 7, 2024
Copy link
Collaborator

@mathomp4 mathomp4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discover stuff looks good to me. I see new things I'll like to hear about! :)

@climbfuji climbfuji force-pushed the bugfix/discover_scu16_ldflags branch from dd5e964 to 84f8089 Compare March 7, 2024 21:36
and set variant ~examples for awscli-v2
@climbfuji climbfuji force-pushed the bugfix/discover_scu16_ldflags branch from 84f8089 to 54b61ad Compare March 7, 2024 21:56
Copy link
Collaborator

@RatkoVasic-NOAA RatkoVasic-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on Hercules.
BTW, even with added comments for warnings in compilation log file is 1GB, 8M lines.
Approving.

configs/sites/noaa-azure/README.md Show resolved Hide resolved
configs/sites/noaa-azure/README.md Show resolved Hide resolved
@@ -1,42 +1,47 @@
compilers::
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These I_MPI_* variables might be needed for building or running the UFS and Apps as well. Probably this is something to keep in mind

@climbfuji climbfuji enabled auto-merge (squash) March 8, 2024 17:33
@climbfuji climbfuji merged commit 3d0b369 into JCSDA:develop Mar 8, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants