Fixes #34839 - Add support for VMware NVME Controllers #10168

girijaasoni · 2024-05-15T00:51:32Z

blocked by PR
hammer doc PR

I put together a document for better understanding. Please feel free to go through it and provide suggestions/improvisations.

Ready for testing :D

Test cases need the fog-vsphere PR merge. They pass on my local and we can trigger them again after the merge of fog-vpshere PR!

ShimShtein · 2024-05-22T10:41:32Z

app/controllers/concerns/foreman/controller/normalize_controller_attributes.rb

+    attrs["nvme_controllers"] = ctrls_and_vol[:controllers]&.select { |controller| controller[:type].include?("VirtualNVMEController") }
+    attrs["scsi_controllers"] = ctrls_and_vol[:controllers]&.select { |controller| !controller[:type].include?("VirtualNVMEController") }


You can use #split method to divide the controllers part

I have used #partition instead, wasn't able to understand the use case #split

ShimShtein · 2024-05-22T10:48:24Z

app/models/compute_resources/foreman/model/vmware.rb

@@ -484,7 +485,7 @@ def parse_args(args)

      # see #26402 - consume scsi_controller_type from hammer as a default scsi type
      scsi_type = args.delete(:scsi_controller_type)


We will need to create a generic option for Hammer too here: instead of scsi_controller_type it would become controller_type. Plus we will need to deprecate the older scsi_controller_type

scsi_controller_type has been removed from compute attributes in this commit Its best we remove this part all together as we use scsi_controller attribute to pass the type and the key

app/models/compute_resources/foreman/model/vmware.rb

ShimShtein · 2024-05-22T10:56:27Z

webpack/assets/javascripts/react_app/redux/reducers/hosts/storage/vmware.js

@@ -26,7 +26,18 @@ const initialState = Immutable({
  volumes: [],
 });

-const availableControllerKeys = [1000, 1001, 1002, 1003, 1004];
+const availableControllerKeys = [
+  1000,


Can you please give me a bit more details about this array and the meaning of the items in it?

This is to assign the keys to scsi or nvme controllers. We can have max of 4 controllers each so the total number of available keys are 8.

When I was in university my professor would reduce your grade if you used magic values. Anything other than 0, 1 or 2 needed to be a constant that described what the magic value actually was.

In Foreman that rule isn't strictly applied, but another way to write this would be:

Array.from({length: 8}, (value, index) => 1000 + index);

I'm not sure if it's that much clearer, but as I read the code it's a list of 8 numbers where the base number is 1000.

Having said that, how do you prevent a user from assigning 8 nvme controllers or 8 scsi controllers?

That is handled by Rbvmomi. We dont place any restrictions from the UI or the api or hammer

ekohl · 2024-05-22T16:29:27Z

app/controllers/concerns/foreman/controller/normalize_controller_attributes.rb

Since controller is ambiguos, I'd suggest to be a bit more explicit and name it something like normalize_vmware_storage_controller_attributes.rb

ekohl · 2024-05-22T16:30:35Z

app/models/compute_resources/foreman/model/vmware.rb

@@ -192,12 +192,13 @@ def nictypes
      }
    end

-    def scsi_controller_types
+    def controller_types


storage_controller_types?

ekohl

Test failures look related.

ekohl · 2024-06-03T12:02:20Z

app/controllers/concerns/foreman/controller/normalize_vmware_storage_controller_attributes.rb

+    volumes = {}
+    ctrls_and_vol[:volumes].each_with_index do |vol, index|
+      volumes[index.to_s] = vol
+    end


Suggested change

volumes = {}

ctrls_and_vol[:volumes].each_with_index do |vol, index|

volumes[index.to_s] = vol

end

volumes = ctrls_and_vol[:volumes].each_with_index.to_h { |vol, index| [index.to_s, vol] }

ekohl · 2024-06-03T12:04:47Z

app/models/compute_resources/foreman/model/vmware.rb

+      normalized['nvme_controllers'] = nvme_controllers.map.with_index do |ctrl, idx|
+        [idx.to_s, ctrl]
+      end.to_h


Suggested change

normalized['nvme_controllers'] = nvme_controllers.map.with_index do |ctrl, idx|

[idx.to_s, ctrl]

end.to_h

normalized['nvme_controllers'] = nvme_controllers.with_index.to_h do |ctrl, idx|

[idx.to_s, ctrl]

end

girijaasoni · 2024-06-03T12:11:00Z

Test failures look related.

Test cases are related to changes in the fog-vpshere. They all pass on my local and we can re-trigger them once the fog-vsphere changes are in.

ekohl · 2024-06-03T12:17:31Z

Test cases are related to changes in the fog-vpshere. They all pass on my local and we can re-trigger them once the fog-vsphere changes are in.

You can change this line here:

foreman/bundler.d/vmware.rb

Line 2 in 89a3f5f

gem 'fog-vsphere', '>= 3.6.4', '< 4.0'

If you point that to your git branch (using https://bundler.io/guides/git.html) then you can run CI here. Once a release of fog-vsphere it out, you can change the line to require that specific release which includes the feature. That way our packaging will either fail or pick it up properly. We have some automation in that area.

ShimShtein · 2024-07-17T14:06:38Z

@ekohl any objections to merging this?

ekohl

I still think https://github.com/theforeman/foreman/pull/10168/files#r1624514222 is valid, but don't consider it blocking.

I'd prefer to rebase this because the fog-libvirt issues should be resolved. That should tell us if the host_js_test.rb failure is relevant or not.

ShimShtein · 2024-07-18T13:05:04Z

@girijaasoni can you please rebase?

@ekohl as for the delete - I prefer less side effects to the input params. so I am fine with the input remaining there.

ekohl · 2024-07-22T15:57:11Z

@girijaasoni have you investigated the test failure?

girijaasoni · 2024-07-22T16:14:38Z

@girijaasoni have you investigated the test failure?

I don't think the test failure is related

ekohl

We don't see the error in develop and it is explicitly about VMs so that makes me think the error is related. Could you provide a bit more reasoning why you think it's not related?

ekohl · 2024-07-22T17:20:46Z

app/models/compute_resources/foreman/model/vmware.rb

      args = parse_args args
+      args = args.deep_symbolize_keys


Nit

Suggested change

args = parse_args args

args = args.deep_symbolize_keys

args = parse_args(args).deep_symbolize_keys

test/controllers/hosts_controller_test.rb

ekohl

Tests are still running, but 👍 once they're green.

ShimShtein · 2024-07-24T13:27:26Z

Merged, thanks @girijaasoni !

girijaasoni requested a review from a team as a code owner May 15, 2024 00:51

pr-processor bot added the Not yet reviewed label May 15, 2024

github-actions bot added the UI label May 15, 2024

girijaasoni force-pushed the nvme_controllers branch from 2b0d314 to 15201f1 Compare May 15, 2024 00:55

girijaasoni marked this pull request as draft May 15, 2024 01:53

girijaasoni force-pushed the nvme_controllers branch from 15201f1 to 7b1d6ae Compare May 16, 2024 10:10

girijaasoni marked this pull request as ready for review May 16, 2024 10:14

girijaasoni force-pushed the nvme_controllers branch from 7b1d6ae to 267a5f4 Compare May 16, 2024 10:53

ekohl changed the title ~~Fixes #34839 - Add support for NVME Controllers~~ Fixes #34839 - Add support for VMware NVME Controllers May 17, 2024

girijaasoni mentioned this pull request May 17, 2024

Add support for NVME Controllers fog/fog-vsphere#300

Merged

stejskalleos assigned stejskalleos and ShimShtein May 22, 2024

ShimShtein requested changes May 22, 2024

View reviewed changes

pr-processor bot added Waiting on contributor and removed Not yet reviewed labels May 22, 2024

ekohl reviewed May 22, 2024

View reviewed changes

girijaasoni force-pushed the nvme_controllers branch from 267a5f4 to 28c3277 Compare May 30, 2024 13:47

pr-processor bot added Needs re-review and removed Waiting on contributor labels May 30, 2024

girijaasoni force-pushed the nvme_controllers branch 2 times, most recently from d4dfa53 to d4c33a1 Compare May 31, 2024 06:58

girijaasoni mentioned this pull request Jun 3, 2024

Fixes #37689 - Add NVME controllers to hammer help theforeman/hammer-cli-foreman#628

Merged

ekohl reviewed Jun 3, 2024

View reviewed changes

girijaasoni force-pushed the nvme_controllers branch 5 times, most recently from 4baa998 to 2eb9c33 Compare June 3, 2024 13:46

girijaasoni force-pushed the nvme_controllers branch 3 times, most recently from f872498 to bef42fa Compare July 17, 2024 13:12

ShimShtein previously approved these changes Jul 17, 2024

View reviewed changes

ekohl requested changes Jul 18, 2024

View reviewed changes

pr-processor bot added the Waiting on contributor label Jul 18, 2024

girijaasoni force-pushed the nvme_controllers branch from bef42fa to 49fc393 Compare July 22, 2024 08:01

pr-processor bot added Needs re-review and removed Waiting on contributor labels Jul 22, 2024

ekohl reviewed Jul 22, 2024

View reviewed changes

girijaasoni force-pushed the nvme_controllers branch 7 times, most recently from aa356e3 to 321c4ce Compare July 23, 2024 10:00

girijaasoni dismissed ShimShtein’s stale review via fe01eaa July 23, 2024 10:32

girijaasoni force-pushed the nvme_controllers branch from 321c4ce to fe01eaa Compare July 23, 2024 10:32

ShimShtein reviewed Jul 24, 2024

View reviewed changes

test/controllers/hosts_controller_test.rb Outdated Show resolved Hide resolved

Fixes #34839 - Add support for VMware NVME Controllers

aa5a1ac

girijaasoni force-pushed the nvme_controllers branch from fe01eaa to aa5a1ac Compare July 24, 2024 11:21

ekohl approved these changes Jul 24, 2024

View reviewed changes

pr-processor bot removed the Needs re-review label Jul 24, 2024

ShimShtein merged commit 175b5f2 into theforeman:develop Jul 24, 2024
64 of 67 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes #34839 - Add support for VMware NVME Controllers #10168

Fixes #34839 - Add support for VMware NVME Controllers #10168

girijaasoni commented May 15, 2024 •

edited

Loading

ShimShtein May 22, 2024

girijaasoni Jun 3, 2024

ShimShtein May 22, 2024

girijaasoni May 30, 2024 •

edited

Loading

ShimShtein May 22, 2024

girijaasoni May 30, 2024

ekohl May 30, 2024

girijaasoni May 30, 2024

ekohl May 22, 2024

ekohl May 22, 2024

ekohl left a comment

ekohl Jun 3, 2024

ekohl Jun 3, 2024

girijaasoni commented Jun 3, 2024

ekohl commented Jun 3, 2024

ShimShtein commented Jul 17, 2024

ekohl left a comment

ShimShtein commented Jul 18, 2024

ekohl commented Jul 22, 2024

girijaasoni commented Jul 22, 2024

ekohl left a comment

ekohl Jul 22, 2024

ekohl left a comment

ShimShtein commented Jul 24, 2024

		attrs["nvme_controllers"] = ctrls_and_vol[:controllers]&.select { \|controller\| controller[:type].include?("VirtualNVMEController") }
		attrs["scsi_controllers"] = ctrls_and_vol[:controllers]&.select { \|controller\| !controller[:type].include?("VirtualNVMEController") }

		@@ -484,7 +485,7 @@ def parse_args(args)

		# see #26402 - consume scsi_controller_type from hammer as a default scsi type
		scsi_type = args.delete(:scsi_controller_type)

	args = parse_args args
	args = args.deep_symbolize_keys
	args = parse_args(args).deep_symbolize_keys

Fixes #34839 - Add support for VMware NVME Controllers #10168

Fixes #34839 - Add support for VMware NVME Controllers #10168

Conversation

girijaasoni commented May 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

girijaasoni May 30, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ekohl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

girijaasoni commented Jun 3, 2024

ekohl commented Jun 3, 2024

ShimShtein commented Jul 17, 2024

ekohl left a comment

Choose a reason for hiding this comment

ShimShtein commented Jul 18, 2024

ekohl commented Jul 22, 2024

girijaasoni commented Jul 22, 2024

ekohl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ekohl left a comment

Choose a reason for hiding this comment

ShimShtein commented Jul 24, 2024

girijaasoni commented May 15, 2024 •

edited

Loading

girijaasoni May 30, 2024 •

edited

Loading