Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix argument forwarding for "receive" with Ruby 3.2.0 #1514

Merged
merged 4 commits into from
Jan 4, 2023
Merged

Fix argument forwarding for "receive" with Ruby 3.2.0 #1514

merged 4 commits into from
Jan 4, 2023

Conversation

benoittgt
Copy link
Member

@benoittgt benoittgt commented Jan 3, 2023

It seems there is place where arguments forwarding is not ok.

I continued the work started by @pirj on #1497

fixes #1497
fixes #1502
fixes #1495
fixes #1513
fixes #1512

Related:

@benoittgt
Copy link
Member Author

benoittgt commented Jan 3, 2023

CI failures seems to be unrelated. I will look at them separately on rspec-support. Probably rspec/rspec-support#553 and rspec/rspec-support#555

@benoittgt benoittgt marked this pull request as ready for review January 3, 2023 07:54
Comment on lines +135 to +163
expect(dbl).to receive(:kw_args_method).with({a: 1, b: 2})
dbl.kw_args_method(a: 1, b: 2)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should fail, shouldn't it? We expect an options hash, and pass kwargs.

Copy link
Member Author

@benoittgt benoittgt Jan 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I am wondering if it is a side effect of changes in 3.2. Like on release note. @eregon any idea?

Screen Shot 2023-01-03 at 10 09 41

Copy link
Member

@pirj pirj Jan 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We seem to be in the opposite situation when an option hash passed as a positional argument to an expectation suddenly matches with a kwargs hash.

I suppose that the problem is here:

            if Hash === expected_args.last && Hash === actual_args.last
              if !Hash.ruby2_keywords_hash?(actual_args.last) && Hash.ruby2_keywords_hash?(expected_args.last)
                return false

In our scenario, Hash.ruby2_keywords_hash?(actual_args.last) is true, while the other one is false.
They don't match, but we don't return false like we do e.g. here:

          if Hash === expected_hash && Hash === actual_hash &&
            (Hash.ruby2_keywords_hash?(expected_hash) != Hash.ruby2_keywords_hash?(actual_hash))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest changing this:

-              if !Hash.ruby2_keywords_hash?(actual_args.last) && Hash.ruby2_keywords_hash?(expected_args.last)
+              if Hash.ruby2_keywords_hash?(actual_args.last) != Hash.ruby2_keywords_hash?(expected_args.last)

This is causing two more specs to fail, at least on Ruby 3.2:

            it "matches against a hash submitted as keyword arguments and received as positional argument (in both Ruby 2 and Ruby 3)" do
              expect(dbl).to receive(:kw_args_method).with(1, {:required_arg => 2, :optional_arg => 3})
              dbl.kw_args_method(1, :required_arg => 2, :optional_arg => 3)
            end
        it "matches against a hash submitted as keyword arguments a and received as a positional argument (in both Ruby 2 and Ruby 3)" do
          opts = {:a => "a", :b => "b"}
          expect(a_double).to receive(:random_call).with(opts)
          a_double.random_call(:a => "a", :b => "b")
        end

But if we'd make this comparison strict, I agree that it should fail.

Do you guys agree to make such a change, and see if it affects other Rubies?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The failure message is suboptimal:

received unexpected message :kw_args_method with ({:a=>1, :b=>2})

while the method was indeed called, and arguments were alike, it's just the Hash.ruby2_keywords_hash? that didn't match.
We do it better here:

                "  expected: (#{expected_input.inspect}, {:one=>1}) (keyword arguments)\n" \                                                  │
                "       got: (#{actual_input.inspect}, {:one=>1}) (options hash)\n" \

but this differ doesn't seem to come into play.

Might be related: rspec/rspec-support#537

Copy link
Contributor

@eregon eregon Jan 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is the comment above:

# if both arguments end with Hashes, and if one is a keyword hash and the other is not, they don't match

is super confusing. That should be updated to something like:

# if both arguments end with Hashes, and if #with was called with keywords but the method was called without keywords, they don't match
# if #with is called without keywords, e.g., with({a: 1}), then it's fine to call it with either foo(a: 1) or foo({a: 1}) because it's as if the method was: def foo(options).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this is only true for methods that accept positional parameters, right? E.g. for:

class A
  def foo(bar:)
  end
end

a = A.new
expect(a).to receive(:foo).with({bar: 1})
a.foo({bar: 1})

If we follow

if #with is called without keywords, e.g., with({a: 1}), then it's fine to call it with either foo(a: 1) or foo({a: 1}) because it's as if the method was: def foo(options).

The spec will pass, but the real-life code:

a = A.new
a.foo({bar: 1})

would blow up with:

ArgumentError: wrong number of arguments (given 1, expected 0; required keyword: bar)

that means we would be concealing the error.

Is my understanding correct, @eregon ?

Copy link
Contributor

@eregon eregon Jan 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a mock, so the method might not exist, right?
From with({a: 1}) that doesn't indicate the method takes keyword arguments, hence it can be called with whatever (passing a: 1 or {a: 1} shouldn't make any difference).

If the method does exist like in that case, I think maybe RSpec could warn that you call with without kwargs when the method accepts kwargs.
Or, you could actually check whether the existing method accepts kwargs or not for the kwargs-vs-positional check, but then what happens if the method does not exist?

Wouldn't this specific example fail anyway because it actually calls the original method?
And if it doesn't call the original method, isn't it wrong to look at the original method?

EDIT: it indeed does not call the original for expect(a).to receive(:foo).with({bar: 1}).
That's super confusing to me, it just returns nil instead.
MSpec would always call the original, unless one uses .and_return.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also this example wouldn't be caught by your change, right? (no ruby2_keyword flag on both sides)
So it's a bug of writing .with({bar: 1}) which means "expect no kwargs" when the real method only accepts kwargs. Hence maybe that's worth a warning.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this example wouldn't be caught by your change

Fair enough. The rest makes total sense, too.

@paddor
Copy link
Contributor

paddor commented Jan 3, 2023

This branch fixes my test suite on Ruby 3.2. 👍

@pirj
Copy link
Member

pirj commented Jan 3, 2023

I dared pushing a commit that makes the kwargs vs positional options hash check more strict.

TODO:

  • spec/rspec/mocks/matchers/receive_spec.rb fails for some weird reason similar to the one described above in spec/rspec/mocks/argument_matchers_spec.rb, but I couldn't quickly find a workaround the same way I did for spec/rspec/mocks/argument_matchers_spec.rb
  • failure message is suboptimal. We need some refactoring so that for a MethodDouble the Proxy called raise_unexpected_message_args_error instead of raise_unexpected_message_error the same way it does in other cases when there's a message expectation, but arguments don't match

@pirj
Copy link
Member

pirj commented Jan 3, 2023

@paddor May I kindly ask you to repeat your test after the last commit?

@pirj
Copy link
Member

pirj commented Jan 3, 2023

rspec/rspec-core#2993 is needed to make the CI green(er) for rspec-core sub-build.

@paddor
Copy link
Contributor

paddor commented Jan 3, 2023

@pirj It mostly passes. Some #receive_message_chain-lines with kwargs fail:

#<Double (anonymous)> received unexpected message :store_event with ({:message=>"core.system_events.force_variables", :object=>{:note=>nil, :release_at=>nil, :variable=>{:code=>nil, :desc_i18n=>nil, :label=>nil, :label_i18n=>nil}}, :username=>nil})

Note that the call-site correctly specified these arguments as keywords, not as a Hash.

If I refactor them to use #receive with intermediate spy-objects, the whole testsuite passes.

@eregon
Copy link
Contributor

eregon commented Jan 3, 2023

@pirj I think it would be safer to revert remove that strict check (i.e., stricter than Ruby itself is), see #1514 (comment). Otherwise it sounds like a huge backward-incompatible change for something which doesn't really gain anything.

@pirj
Copy link
Member

pirj commented Jan 3, 2023

@eregon The check is certainly stricter now for def foo(options = {}) or def foo(options).

But it's not stricter than Ruby's for def foo(**options) and def foo(bar:).

To make a distinction between those two cases, we would have to make ArgumentListMatcher aware of the method signature.

All those cases are different instance.method(:foo).parameters:

Class.new { def foo(options); end }.new.method(:foo).parameters # => [[:req, :options]]
Class.new { def foo(options = {}); end }.new.method(:foo).parameters # => [[:opt, :options]]
Class.new { def foo(**options); end }.new.method(:foo).parameters # => [[:keyrest, :options]]
Class.new { def foo(bar:); end }.new.method(:foo).parameters # => [[:keyreq, :bar]]

What is making matter worse is that Ruby was doing weird things with extracting some arguments from the options hash, see e.g. rspec/rspec-support#537

@rickychilcott
Copy link

Running this branch fixes all of my mocking issues. Thank you!

This is more of an rspec-rails issue, but I'm noticing a have_enqueued_job matcher is no longer finding the job.

expect {
    get :create, params: {onboarding_signup_id: onboarding_signup_id}
  }.to have_enqueued_job(Sendgrid::ListSignupJob).with(
    email: user.email, list: OnboardingSignupsController::CURRENT_ONBOARDING_LIST_NAME
  )

  expected to enqueue exactly 1 jobs, with [{:email=>"clair_mcdermott@example.net", :list=>"PROD Onboarding v1"}], but enqueued 0
  Queued jobs:
    Sendgrid::ListSignupJob job with [{:email=>"clair_mcdermott@example.net", :list=>"PROD Onboarding v1"}], on queue default

^^^ passes in Ruby 3.1

@Mange
Copy link

Mange commented Jan 4, 2023

I've tested this branch but expectations on any keyword argument seem to not work.

require "spec_helper"

class Ex
  def self.foo(a:)
    puts "Called foo(a: #{a.inspect})"
    true
  end
end

RSpec.describe "example" do
  it "finds no call on single call" do
    allow(Ex).to receive(:foo).and_call_original

    Ex.foo(a: 1)

    #  Failure/Error: expect(Ex).to have_received(:foo).with(a: 1)
    #    (Ex (class)).foo({:a=>1})
    #        expected: 1 time with arguments: ({:a=>1})
    #        received: 0 times
    expect(Ex).to have_received(:foo).with(a: 1)
  end

  it "finds other call on two calls, but still fails" do
    allow(Ex).to receive(:foo).and_call_original

    Ex.foo(a: 1)
    Ex.foo(a: 2)

    # #<Ex (class)> received :foo with unexpected arguments
    #   expected: ({:a=>1}) (options hash)
    #        got: ({:a=>2}) (keyword arguments)
    #   Diff:
    #   @@ -1 +1 @@
    #   -[{:a=>1}]
    #   +[{:a=>2}]
    expect(Ex).to have_received(:foo).with(a: 1)
    expect(Ex).to have_received(:foo).with(a: 2)

    # Note that just expecting the second call will still fail, but with an
    # inverted error message; it will show the first call.
  end
end
Output
Randomized with seed 60000

example
Called foo(a: 1)
  finds no call on single call (FAILED - 1)
Called foo(a: 1)
Called foo(a: 2)
  finds other call on two calls, but still fails (FAILED - 2)

Failures:

  1) example finds no call on single call
     Failure/Error: expect(Ex).to have_received(:foo).with(a: 1)

       (Ex (class)).foo({:a=>1})
           expected: 1 time with arguments: ({:a=>1})
           received: 0 times
     # ./spec/broken_spec.rb:20:in `block (2 levels) in <top (required)>'

  2) example finds other call on two calls, but still fails
     Failure/Error: expect(Ex).to have_received(:foo).with(a: 1)

       #<Ex (class)> received :foo with unexpected arguments
         expected: ({:a=>1}) (options hash)
              got: ({:a=>2}) (keyword arguments)
       Diff:
       @@ -1 +1 @@
       -[{:a=>1}]
       +[{:a=>2}]

     # ./spec/broken_spec.rb:36:in `block (2 levels) in <top (required)>'

Finished in 0.01183 seconds (files took 0.26323 seconds to load)
2 examples, 2 failures

Failed examples:

rspec ./spec/broken_spec.rb:11 # example finds no call on single call
rspec ./spec/broken_spec.rb:23 # example finds other call on two calls, but still fails
Gem versions (commits)
GIT
  remote: https://github.com/rspec/rspec-core.git
  revision: 522b7727d02d9648c090b56fa68bbdc18a21c04d
  branch: main
  specs:
    rspec-core (3.13.0.pre)
      rspec-support (= 3.13.0.pre)

GIT
  remote: https://github.com/rspec/rspec-expectations.git
  revision: b1fa2e620b03ed05c737db10b2727a0706eca7d3
  branch: main
  specs:
    rspec-expectations (3.13.0.pre)
      diff-lcs (>= 1.2.0, < 2.0)
      rspec-support (= 3.13.0.pre)

GIT
  remote: https://github.com/rspec/rspec-mocks.git
  revision: d8a213c26c31bc940f5060145b85131cbcf43715
  branch: ruby-3.2
  specs:
    rspec-mocks (3.13.0.pre)
      diff-lcs (>= 1.2.0, < 2.0)
      rspec-support (= 3.13.0.pre)

GIT
  remote: https://github.com/rspec/rspec-rails.git
  revision: c60ff7907559653cd9d1ec1a6113bf86c9359fab
  branch: main
  specs:
    rspec-rails (6.1.0.pre)
      actionpack (>= 6.1)
      activesupport (>= 6.1)
      railties (>= 6.1)
      rspec-core (= 3.13.0.pre)
      rspec-expectations (= 3.13.0.pre)
      rspec-mocks (= 3.13.0.pre)
      rspec-support (= 3.13.0.pre)

@paddor
Copy link
Contributor

paddor commented Jan 4, 2023

Now it works with and without the refactorings in my testsuite. 👍

.github/workflows/ci.yml Outdated Show resolved Hide resolved
Copy link
Contributor

@eregon eregon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@pirj
Copy link
Member

pirj commented Jan 4, 2023

@paddor Just to make sure, do your receive_message_chain work, too?

pirj and others added 2 commits January 4, 2023 18:08
Changelog.md Outdated Show resolved Hide resolved
Copy link
Member

@pirj pirj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@pirj
Copy link
Member

pirj commented Jan 4, 2023

@JonRowe I intend to:

  • merge this
  • fix the ReentrantMutex issue to fix Ruby 3.2 sub-builds
  • handle a coordinated update of buildfiles from rspec-dev to add Ruby 3.2 to ci

@JonRowe
Copy link
Member

JonRowe commented Jan 7, 2023

Thanks for tackling this, would have been nice to wait for my review but as this ended up as a simple one line lib change it's ok, good work on the updated comment too!

JonRowe pushed a commit that referenced this pull request Jan 7, 2023
@JonRowe
Copy link
Member

JonRowe commented Jan 7, 2023

Released in 3.12.2

@JonRowe
Copy link
Member

JonRowe commented Jan 7, 2023

I added the 3.2 builds myself coordinating with 3-12-maintenance, with a temporary suppression of the broken spec, I still think it can be fixed rather than skipped like this.

Also I've now disabled rebase merges on our repos as they've been used accidentally again and it makes cherry picking to maintenance branches a nightmare, we want single commits for PRs (merge or squash only).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
7 participants