Fixes #37825 - Upgrade Rails to 7.0 #10299

ofedoren · 2024-08-29T15:23:25Z

Based on #10131, but with additional commit needed for Rails upgrade.

ekohl

In #9328 I also updated spring. Is that not needed? I recall running into it, but that could have been a Ruby thing or I was just looking at changelogs.

ekohl · 2024-08-29T15:42:50Z

app/models/concerns/hostext/token.rb

@@ -5,7 +5,7 @@ module Token
    included do
      has_one :token, :foreign_key => :host_id, :dependent => :destroy, :inverse_of => :host, :class_name => 'Token::Build'

-      scope :for_token, ->(token) { joins(:token).where(:tokens => { :value => token }).where("expires >= ?", Time.now.utc.to_s(:db)).select('hosts.*') }
+      scope :for_token, ->(token) { joins(:token).where(:tokens => { :value => token }).where("expires >= ?", Time.now.utc.to_fs(:db)).select('hosts.*') }


TIL: there's .to_fs in Rails. I wonder if Rails is smart enough to understand

Suggested change

scope :for_token, ->(token) { joins(:token).where(:tokens => { :value => token }).where("expires >= ?", Time.now.utc.to_fs(:db)).select('hosts.*') }

scope :for_token, ->(token) { joins(:token).where(:tokens => { :value => token }).where("expires >= ?", Time.now.utc).select('hosts.*') }

ekohl

Have you tried if moving the settings into initializers works with Rails 6.1?

ekohl · 2024-08-30T17:18:15Z

test/unit/has_many_common_test.rb

  class ::FakePlugin; end
-  class ::FakePlugin::FakeModel; end
+  class ::FakePlugin::FakeModel < ApplicationRecord; end


Should this even be

module FakePlugin class FakeModel < ApplicationRecord end end

ofedoren · 2024-09-19T14:51:35Z

There left only one test failure, I'm looking into it, for now I've got no clue :/

Apart from that, it's ready unless we want to be more correct and apply/incorporate these changes: https://railsdiff.org/6.1.7.8/7.0.8.4

ofedoren · 2024-09-27T15:34:56Z

Now it fails due to ldap_fluff, since it's locked on active_support < 7 (https://github.com/theforeman/ldap_fluff/blob/master/ldap_fluff.gemspec#L21). Weird that it didn't fail before...

Also, @ekohl and @adamruzicka or maybe @ShimShtein, since you have more experience with the codebase and Rails, could you please help me get why the test fails? I'll be out for the next week (30.09 - 06.10), so I hope we won't waste that week.

Regarding the test: I think it's due to code reloading happening somewhere in the test:units suit since running each of test/models, test/helpers and test/unit separately works, it works even if run rails test test/models test/helpers test/unit, which is the same what rake test:units does. Setting test_order = :sorted doesn't help. Also, I've noticed that for some reason it can pass twice in a row, but another run would fail.

app/models/nic/base.rb

ofedoren · 2024-10-09T15:50:59Z

Finally, it's green and with minimal changes needed. Ready for full review.

ekohl · 2024-10-09T16:00:54Z

app/models/concerns/foreman/sti.rb

@@ -15,13 +15,5 @@ def new(attributes = nil, &block)
        super
      end
    end
-
-    def save(*args, **kwargs)


So this was the whole problem? That it set an instance variable on the class? That looks nasty and I wonder how that worked in the past.

From reading the Rails source I get the impression should handle this itself.

Yes, it seems so. I couldn't find a reason why we did that at the first place, so I hope removing doesn't break anything.

I'm still going to try basic processes (CRUDing a virtual host, running jobs, some Katello-related stuff, oscap, etc) to verify PRs after everything open is green.

ekohl · 2024-10-22T12:17:04Z

As noted in a meeting, the failure may be due to nulldb:

foreman/.github/workflows/foreman.yml

Line 73 in 17f6c03

- 'assets:precompile RAILS_ENV=production DATABASE_URL=nulldb://nohost'

ekohl

@ofedoren I see you now disable spring for integration tests and enable bootsnap for tests. Do you consider the issue resolved with that?

Additionally, is there a way we could use spring for integration tests? Possibly by explicitly stopping the server `

ekohl · 2024-10-31T19:44:17Z

config/environments/test.rb

@@ -10,7 +10,8 @@
  # test suite. You never need to work with it otherwise. Remember that
  # your test database is "scratch space" for the test suite and is wiped
  # and recreated between test runs. Don't rely on the data there!
-  config.cache_classes = true
+  # Disable reloading for integration tests
+  config.cache_classes = ARGV.grep(/test\/integration/).any?


Shouldn't this be the other way around? Only enable it when it's none??

Also, https://github.com/rails/spring?tab=readme-ov-file#enable-reloading notes it needs to be false for spring to work.

Another note: https://github.com/rails/spring?tab=readme-ov-file#temporarily-disabling-spring says:

If you're using Spring binstubs, but temporarily don't want commands to run through Spring, set the DISABLE_SPRING environment variable.

ofedoren · 2024-11-01T14:56:15Z

@ofedoren I see you now disable spring for integration tests and enable bootsnap for tests. Do you consider the issue resolved with that?

I'd say so, yes. Mostly because of spring being disabled for integrations. I don't think bootsnap did anything though. I just enabled that since it should be quite safe and maybe help us in the future.

Additionally, is there a way we could use spring for integration tests? Possibly by explicitly stopping the server

How that will help except for running integration tests on local dev setup? I mean, the whole point of spring for tests is to reduce time between individual calls/runs after the first one. In CI it doesn't matter, since there is only one (the first) run. For local dev runs it makes life easier, thus I've tried to leave the cache_classes set to false for other test types (e.g. units). Integration test suite takes a long time anyway, it easier to use CI for that then wait for local results. Moreover, we didn't have reloading for tests ever before.

Honestly, I'm quite surprised that reloading for units/functionals seems to work at all. I'd say reloading was (and still is) mostly unsupported within Foreman and even if it worked, it kidna shouldn't (I might be wrong though, just an impression).

Also, https://github.com/rails/spring?tab=readme-ov-file#enable-reloading notes it needs to be false for spring to work.

Yes, cache_classes set to false is the same as enable_reloading set to true starting Rails 7.1. Spring with Rails 7.0 forces reloading to be set to true for test environment as well, which is per my opinion weird: dev env with reloading makes sense to make and see the changes faster, but the test environment should rather mimic prod env otherwise what's the point to write something just to see a green circle.

If you're using Spring binstubs, but temporarily don't want commands to run through Spring, set the DISABLE_SPRING environment variable.

Yeap, noticed that too and I thought to use that for CI workflow, but then it would make devs use it for their runs, which I've tried to avoid. This PR (the commit with spring conditionally disabled) tries to make as little invasive changes as possible without changing any already existing workflows: spring is not nuked, it can be still used by devs (although it won't work properly anyway), it's even supported for most test suits (except integrations), no additional env variables must be set to run commands.

I'd love having code reloading working properly, but it seems it's quite complicated to make it work in Foreman (I might lack deeper knowledge about Rails internals though, so if anyone wants to fix that, please do. Fresh eyes make miracles). I disabled spring for integration tests since reloading takes too much time (although it shouldn't) causing timeouts, messes up constant ids, which leads to errors such as organization_obj.class <= Organization #=> false.

Without this patch Ansible-related scopes for Host::Managed.includes() would look like: [ { host_ansible_roles: :ansible_roles }, { ansible_role: [{ ansible_variables: [:lookup_values] }] } ], which for some reason worked before Rails 7. After this patch, it looks like: [ { host_ansible_roles: { ansible_role: [{ ansible_variables: [:lookup_values] }] } } ].

…tional

ofedoren · 2024-11-07T14:23:44Z

config/application.rb

+    # Rails 7.0 changed this to OpenSSL::Digest::SHA256
+    # We'd probably need a rotator:
+    # https://guides.rubyonrails.org/v7.0/upgrading_ruby_on_rails.html#key-generator-digest-class-changing-to-use-sha256
+    config.active_support.hash_digest_class = OpenSSL::Digest::SHA1


This is tricky one, I've noticed that only when Katello started failing with some hashes not being matched.

Although, this is safe for now, I'd say we need to migrate from SHA1 since RHEL9 won't support it for cert signing purposes at least: https://bugs.ruby-lang.org/issues/19307
(https://github.com/Katello/katello/pull/11155/files#diff-f6f4727f55ce6e9824ffc740bc9494410dd954416f3bbf50799f38b6738e9fcfL49)
and Rails' defaults to SHA256 for caching already.

If we change this now, we'd probably need that (https://guides.rubyonrails.org/v7.0/upgrading_ruby_on_rails.html#key-generator-digest-class-changing-to-use-sha256) thingy as well as some changes in Katello since I've noticed it heavily utilizes https://github.com/Katello/katello/blob/master/app/lib/katello/util/data.rb#L9 for creating some labels/ids? https://github.com/Katello/katello/blob/master/app/models/katello/content_view_environment.rb#L117, which will break with SHA256. Although, we can fix it in Katello itself.

pr-processor bot added Not yet reviewed Waiting on contributor labels Aug 29, 2024

github-actions bot added the GraphQL label Aug 29, 2024

ekohl reviewed Aug 29, 2024

View reviewed changes

pr-processor bot removed the Not yet reviewed label Aug 29, 2024

ofedoren force-pushed the rails-7.0 branch from a2adde6 to a7905f1 Compare August 30, 2024 11:57

pr-processor bot added the Needs re-review label Aug 30, 2024

ekohl reviewed Aug 30, 2024

View reviewed changes

pr-processor bot removed the Needs re-review label Aug 30, 2024

ofedoren force-pushed the rails-7.0 branch from a7905f1 to 6e7fb07 Compare September 3, 2024 12:01

pr-processor bot added the Needs re-review label Sep 3, 2024

ofedoren force-pushed the rails-7.0 branch 2 times, most recently from b2971c8 to c93acc6 Compare September 16, 2024 14:10

pr-processor bot removed the Waiting on contributor label Sep 16, 2024

ofedoren force-pushed the rails-7.0 branch from c93acc6 to 717c5eb Compare September 16, 2024 15:01

ofedoren changed the title ~~Rails 7.0~~ Fixes #37825 - Upgrade Rails to 7.0 Sep 16, 2024

ofedoren marked this pull request as ready for review September 16, 2024 15:02

ofedoren requested a review from a team as a code owner September 16, 2024 15:02

ekohl mentioned this pull request Sep 17, 2024

Rails 7 & Ruby 3.1 #9328

Closed

ofedoren commented Oct 9, 2024

View reviewed changes

app/models/nic/base.rb Outdated Show resolved Hide resolved

pr-processor bot removed the Needs re-review label Oct 9, 2024

ofedoren force-pushed the rails-7.0 branch from b9ec6cc to b90a9be Compare October 9, 2024 15:23

pr-processor bot added the Waiting on contributor label Oct 9, 2024

ofedoren force-pushed the rails-7.0 branch from f43df65 to 628d9a3 Compare October 9, 2024 15:34

pr-processor bot added the Needs re-review label Oct 9, 2024

ekohl reviewed Oct 9, 2024

View reviewed changes

pr-processor bot removed the Needs re-review label Oct 9, 2024

ofedoren force-pushed the rails-7.0 branch from 628d9a3 to 4e8b93b Compare October 16, 2024 13:50

pr-processor bot added Needs re-review and removed Needs re-review labels Oct 16, 2024

ofedoren force-pushed the rails-7.0 branch from 4e8b93b to 6997569 Compare October 21, 2024 10:54

pr-processor bot added Needs re-review and removed Waiting on contributor labels Oct 21, 2024

ekohl mentioned this pull request Oct 23, 2024

Fixes #37849 - Support Rails 7.0 theforeman/foreman_openscap#582

Merged

ofedoren force-pushed the rails-7.0 branch from 6997569 to 216e227 Compare October 24, 2024 14:44

ofedoren force-pushed the rails-7.0 branch from 623bb51 to 472ad12 Compare October 31, 2024 15:41

ekohl reviewed Oct 31, 2024

View reviewed changes

pr-processor bot removed the Needs re-review label Oct 31, 2024

ofedoren force-pushed the rails-7.0 branch from 472ad12 to 2f040d6 Compare November 4, 2024 13:53

ofedoren added 8 commits November 7, 2024 14:11

Fixes #37825 - Upgrade Rails to 7.0

8012399

Refs #37825 - Make reloading work

21c1b11

Refs #37825 - Stop breaking Rails

c7a9951

Refs #37825 - Clear all connections after webpack:compile

b5e9f73

Refs #37825 - Enable bootsnap for test, disable reloading for integra…

cf723dd

…tional

Refs #37825 - Use #add instead of #<< for ActiveModel::Errors

4d9f64c

Refs #37825 - Return OpenSSL::Digest::SHA1 for hash_digest_class

1a151be

ofedoren force-pushed the rails-7.0 branch from 2f040d6 to 1a151be Compare November 7, 2024 14:11

ofedoren commented Nov 7, 2024

View reviewed changes

ofedoren mentioned this pull request Nov 7, 2024

Fixes #37852 - Support Rails 7.0 Katello/katello#11155

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes #37825 - Upgrade Rails to 7.0 #10299

Fixes #37825 - Upgrade Rails to 7.0 #10299

ofedoren commented Aug 29, 2024

ekohl left a comment

ekohl Aug 29, 2024

ekohl left a comment

ekohl Aug 30, 2024

ofedoren commented Sep 19, 2024

ofedoren commented Sep 27, 2024

ofedoren commented Oct 9, 2024

ekohl Oct 9, 2024

ofedoren Oct 16, 2024

ekohl commented Oct 22, 2024

ekohl left a comment

ekohl Oct 31, 2024

ofedoren commented Nov 1, 2024

ofedoren Nov 7, 2024

	scope :for_token, ->(token) { joins(:token).where(:tokens => { :value => token }).where("expires >= ?", Time.now.utc.to_fs(:db)).select('hosts.*') }
	scope :for_token, ->(token) { joins(:token).where(:tokens => { :value => token }).where("expires >= ?", Time.now.utc).select('hosts.*') }

Fixes #37825 - Upgrade Rails to 7.0 #10299

Are you sure you want to change the base?

Fixes #37825 - Upgrade Rails to 7.0 #10299

Conversation

ofedoren commented Aug 29, 2024

ekohl left a comment

Choose a reason for hiding this comment

ekohl Aug 29, 2024

Choose a reason for hiding this comment

ekohl left a comment

Choose a reason for hiding this comment

ekohl Aug 30, 2024

Choose a reason for hiding this comment

ofedoren commented Sep 19, 2024

ofedoren commented Sep 27, 2024

ofedoren commented Oct 9, 2024

ekohl Oct 9, 2024

Choose a reason for hiding this comment

ofedoren Oct 16, 2024

Choose a reason for hiding this comment

ekohl commented Oct 22, 2024

ekohl left a comment

Choose a reason for hiding this comment

ekohl Oct 31, 2024

Choose a reason for hiding this comment

ofedoren commented Nov 1, 2024

ofedoren Nov 7, 2024

Choose a reason for hiding this comment