Any way to run this on Intel Arc?

[HyunJae5463]

Guess it doesn't support IPEX? Tried it and it gives TypeError: 'NoneType' object is not iterable Error

[congdm]

It can run on IPEX. I haven't tested extensions but generating images is fine.

[HyunJae5463]

It can run on IPEX. I haven't tested extensions but generating images is fine.

As i said, when i try to generate anything it says "TypeError: 'NoneType' object is not iterable"

[congdm]

How about trying to reinstall it?

1. Delete the venv folder
2. Check if COMMANDLINE_ARGS=--use_ipex in webui-user.bat
3. Run webui-user.bat to reinstall
   If even reinstalling doesn't fix it then maybe it's another problem

[HyunJae5463]

How about trying to reinstall it?

1. Delete the venv folder
2. Check if COMMANDLINE_ARGS=--use_ipex in webui-user.bat
3. Run webui-user.bat to reinstall
   If even reinstalling doesn't fix it then maybe it's another problem

Tried reinstalling twice. Second time on a fresh Windows User. Doesnt work.

[redrum-llik]

Have the same issue on a clean install - the underlying exception is as following:

  File "C:\Forge\modules\rng.py", line 86, in create_generator
    generator = torch.Generator(device).manual_seed(int(seed))
RuntimeError: Device type XPU is not supported for torch.Generator() api.
Device type XPU is not supported for torch.Generator() api.

I have been able to work around the issue by amending the modules/rng.py script - since this Generator class seems to be used for the RNG only, I have amended the device checks to use CPU instead of GPU if Intel Arc (XPU) device is detected. Please find the unified diff below:

 modules/rng.py | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/modules/rng.py b/modules/rng.py
index 4a3bbb20..1426a428 100644
--- a/modules/rng.py
+++ b/modules/rng.py
@@ -3,6 +3,8 @@ import torch
 from modules import devices, rng_philox, shared
 
 
+TORCH_UNSUPPORTED_DEVICES = ['mps', 'xpu']
+
 def randn(seed, shape, generator=None):
     """Generate a tensor with random numbers from a normal distribution using seed.
 
@@ -19,7 +21,7 @@ def randn(seed, shape, generator=None):
     if shared.opts.randn_source == "NV":
         return torch.asarray((generator or nv_rng).randn(shape), device=devices.device)
 
-    if shared.opts.randn_source == "CPU" or devices.device.type == 'mps':
+    if shared.opts.randn_source == "CPU" or devices.device.type in TORCH_UNSUPPORTED_DEVICES:
         return torch.randn(shape, device=devices.cpu, generator=generator).to(devices.device)
 
     return torch.randn(shape, device=devices.device, generator=generator)
@@ -34,7 +36,7 @@ def randn_local(seed, shape):
         rng = rng_philox.Generator(seed)
         return torch.asarray(rng.randn(shape), device=devices.device)
 
-    local_device = devices.cpu if shared.opts.randn_source == "CPU" or devices.device.type == 'mps' else devices.device
+    local_device = devices.cpu if shared.opts.randn_source == "CPU" or devices.device.type in TORCH_UNSUPPORTED_DEVICES else devices.device
     local_generator = torch.Generator(local_device).manual_seed(int(seed))
     return torch.randn(shape, device=local_device, generator=local_generator).to(devices.device)
 
@@ -47,7 +49,7 @@ def randn_like(x):
     if shared.opts.randn_source == "NV":
         return torch.asarray(nv_rng.randn(x.shape), device=x.device, dtype=x.dtype)
 
-    if shared.opts.randn_source == "CPU" or x.device.type == 'mps':
+    if shared.opts.randn_source == "CPU" or x.device.type in TORCH_UNSUPPORTED_DEVICES:
         return torch.randn_like(x, device=devices.cpu).to(x.device)
 
     return torch.randn_like(x)
@@ -61,7 +63,7 @@ def randn_without_seed(shape, generator=None):
     if shared.opts.randn_source == "NV":
         return torch.asarray((generator or nv_rng).randn(shape), device=devices.device)
 
-    if shared.opts.randn_source == "CPU" or devices.device.type == 'mps':
+    if shared.opts.randn_source == "CPU" or devices.device.type in TORCH_UNSUPPORTED_DEVICES:
         return torch.randn(shape, device=devices.cpu, generator=generator).to(devices.device)
 
     return torch.randn(shape, device=devices.device, generator=generator)
@@ -82,7 +84,8 @@ def create_generator(seed):
     if shared.opts.randn_source == "NV":
         return rng_philox.Generator(seed)
 
-    device = devices.cpu if shared.opts.randn_source == "CPU" or devices.device.type == 'mps' else devices.device
+    device = devices.cpu if shared.opts.randn_source == "CPU" or devices.device.type in TORCH_UNSUPPORTED_DEVICES else devices.device
     generator = torch.Generator(device).manual_seed(int(seed))
     return generator

There is also a warning stating that Forge expects a higher version of torch:

You are running torch 2.0.0a0+gite9ebda2.
The program is tested to work with torch 2.1.2.

This is the same version I end up with when I run Forge with --reinstall-torch flag. @congdm, if it works for you without any manual changes, do you mind sharing your local version of torch? Perhaps the newer torch releases do support xpu devices; if they do not, I will create a PR for the above change.

[redrum-llik]

Apparently, yeah, xpu is supported by Torch 2.1+ but on Windows, Forge will run with 2.0.X custom build at the moment. We might want to wait until there is a newer build available and the installation script is updated, or test it out manually for a quicker turnaround.

[congdm]

@redrum-llik Hi, actually I also got that Generator error too, however Forge allowed us to change RNG type to CPU in Settings/Stable Diffusion tab (Random number generator source setting), no need to edit any source file.

@HyunJae5463 I just discovered that some sampler could cause TypeError: 'NoneType' exception. Have you tested it on various samplers or settings, maybe it could help.

[redrum-llik]

no need to edit any source file.

Ah cool, thank you for the notice!
I have run into one more issue that might be specific to Intel Arc GPUs, which is described here. IPEX is known to have a limit of 4 GBs for memory allocations (for performance reasons), and there is a piece of code that is meant to handle that, but it does not seem to be invoked in my case. Have you seen this one, and, if yes, is there any workaround you could recommend?

[redrum-llik]

A quick update on that - made a crude hack that seems to fix the allocation issue on the newer versions, you may find the diff here.

[Tolga077]

mine works fine install through stability matrix. https://github.com/LykosAI/StabilityMatrix

[them3rcury]

Launching Web UI with arguments: --use_ipex --skip-torch-cuda-test
Traceback (most recent call last):
  File "E:\stable-diffusion-webui-forge\launch.py", line 51, in <module>
    main()
  File "E:\stable-diffusion-webui-forge\launch.py", line 47, in main
    start()
  File "E:\stable-diffusion-webui-forge\modules\launch_utils.py", line 541, in start
    import webui
  File "E:\stable-diffusion-webui-forge\webui.py", line 17, in <module>
    initialize_forge()
  File "E:\stable-diffusion-webui-forge\modules_forge\initialization.py", line 50, in initialize_forge
    import ldm_patched.modules.model_management as model_management
  File "E:\stable-diffusion-webui-forge\ldm_patched\modules\model_management.py", line 122, in <module>
    total_vram = get_total_memory(get_torch_device()) / (1024 * 1024)
  File "E:\stable-diffusion-webui-forge\ldm_patched\modules\model_management.py", line 91, in get_torch_device
    return torch.device(torch.cuda.current_device())
  File "E:\stable-diffusion-webui-forge\venv\lib\site-packages\torch\cuda\__init__.py", line 769, in current_device
    _lazy_init()
  File "E:\stable-diffusion-webui-forge\venv\lib\site-packages\torch\cuda\__init__.py", line 298, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx

After trying to use it on Windows with Intel Arc A750. I wonder what I'm doing wrong.

[redrum-llik]

Hello! You have --use_ipex in your args, but the correct one is --use-ipex (with hyphen). Please rerun the script with --use-ipex --reinstall-torch for the first run, and subsequent runs should be good with --use-ipex only. If that does not help for any reason, as @congdm suggested below, remove venv folder and run the script with just --use-ipex.

[congdm]

Launching Web UI with arguments: --use_ipex --skip-torch-cuda-test
Traceback (most recent call last):
  File "E:\stable-diffusion-webui-forge\launch.py", line 51, in <module>
    main()
  File "E:\stable-diffusion-webui-forge\launch.py", line 47, in main
    start()
  File "E:\stable-diffusion-webui-forge\modules\launch_utils.py", line 541, in start
    import webui
  File "E:\stable-diffusion-webui-forge\webui.py", line 17, in <module>
    initialize_forge()
  File "E:\stable-diffusion-webui-forge\modules_forge\initialization.py", line 50, in initialize_forge
    import ldm_patched.modules.model_management as model_management
  File "E:\stable-diffusion-webui-forge\ldm_patched\modules\model_management.py", line 122, in <module>
    total_vram = get_total_memory(get_torch_device()) / (1024 * 1024)
  File "E:\stable-diffusion-webui-forge\ldm_patched\modules\model_management.py", line 91, in get_torch_device
    return torch.device(torch.cuda.current_device())
  File "E:\stable-diffusion-webui-forge\venv\lib\site-packages\torch\cuda\__init__.py", line 769, in current_device
    _lazy_init()
  File "E:\stable-diffusion-webui-forge\venv\lib\site-packages\torch\cuda\__init__.py", line 298, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx

After trying to use it on Windows with Intel Arc A750. I wonder what I'm doing wrong.

Delete venv folder in webui and run again, it should work, I think

[them3rcury]

So after changing it to --use-ipex and reinstalling the webui, it gave out the error TypeError: 'NoneType' object is not iterable

Changing the sampler didnt change the outcome. Then I went on and changed the RNG source to CPU and voilà it works like a charm.

512x512 with Euler A getting 8.80it/s and 768x1024 at 4.40it/s with Euler A. Thanks all, appreciate the help.

[redrum-llik]

One more hack to collection, this time related to the issue with GFPGAN/Codeformer not running on IPEX which affects ReActor. If you use it (or if you run face restoration with your generations), consider changing the device that is used to run those models to CPU using this approach. We probably could benefit from being able to select this device via settings or CMD flags.

[them3rcury]

I know I said it fixed the issue, but I still get the error now and then randomly while generating images, only fix is to restart the webui and hope it doesnt do it again. It's more common when using SDXL or SDXL sometimes doesnt work at all. Pretty random. But when it works its making wonders.

[redrum-llik]

Can you share the full stacktrace? Perhaps you are seeing the allocation issue which I have mentioned above.

[them3rcury]

Traceback (most recent call last):
  File "E:\stable-diffusion-webui-forge\modules_forge\main_thread.py", line 37, in loop
    task.work()
  File "E:\stable-diffusion-webui-forge\modules_forge\main_thread.py", line 26, in work
    self.result = self.func(*self.args, **self.kwargs)
  File "E:\stable-diffusion-webui-forge\modules\txt2img.py", line 111, in txt2img_function
    processed = processing.process_images(p)
  File "E:\stable-diffusion-webui-forge\modules\processing.py", line 752, in process_images
    res = process_images_inner(p)
  File "E:\stable-diffusion-webui-forge\modules\processing.py", line 922, in process_images_inner
    samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
  File "E:\stable-diffusion-webui-forge\modules\processing.py", line 1275, in sample
    samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
  File "E:\stable-diffusion-webui-forge\modules\sd_samplers_kdiffusion.py", line 251, in sample
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "E:\stable-diffusion-webui-forge\modules\sd_samplers_common.py", line 263, in launch_sampling
    return func()
  File "E:\stable-diffusion-webui-forge\modules\sd_samplers_kdiffusion.py", line 251, in <lambda>
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "E:\stable-diffusion-webui-forge\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "E:\stable-diffusion-webui-forge\repositories\k-diffusion\k_diffusion\sampling.py", line 145, in sample_euler_ancestral
    denoised = model(x, sigmas[i] * s_in, **extra_args)
  File "E:\stable-diffusion-webui-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable-diffusion-webui-forge\modules\sd_samplers_cfg_denoiser.py", line 182, in forward
    denoised = forge_sampler.forge_sample(self, denoiser_params=denoiser_params,
  File "E:\stable-diffusion-webui-forge\modules_forge\forge_sampler.py", line 88, in forge_sample
    denoised = sampling_function(model, x, timestep, uncond, cond, cond_scale, model_options, seed)
  File "E:\stable-diffusion-webui-forge\ldm_patched\modules\samplers.py", line 289, in sampling_function
    cond_pred, uncond_pred = calc_cond_uncond_batch(model, cond, uncond_, x, timestep, model_options)
  File "E:\stable-diffusion-webui-forge\ldm_patched\modules\samplers.py", line 258, in calc_cond_uncond_batch
    output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
  File "E:\stable-diffusion-webui-forge\ldm_patched\modules\model_base.py", line 90, in apply_model
    model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
  File "E:\stable-diffusion-webui-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable-diffusion-webui-forge\ldm_patched\ldm\modules\diffusionmodules\openaimodel.py", line 867, in forward
    h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
  File "E:\stable-diffusion-webui-forge\ldm_patched\ldm\modules\diffusionmodules\openaimodel.py", line 55, in forward_timestep_embed
    x = layer(x, context, transformer_options)
  File "E:\stable-diffusion-webui-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable-diffusion-webui-forge\ldm_patched\ldm\modules\attention.py", line 620, in forward
    x = block(x, context=context[i], transformer_options=transformer_options)
  File "E:\stable-diffusion-webui-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable-diffusion-webui-forge\ldm_patched\ldm\modules\attention.py", line 447, in forward
    return checkpoint(self._forward, (x, context, transformer_options), self.parameters(), self.checkpoint)
  File "E:\stable-diffusion-webui-forge\ldm_patched\ldm\modules\diffusionmodules\util.py", line 194, in checkpoint
    return func(*inputs)
  File "E:\stable-diffusion-webui-forge\ldm_patched\ldm\modules\attention.py", line 507, in _forward
    n = self.attn1(n, context=context_attn1, value=value_attn1, transformer_options=extra_options)
  File "E:\stable-diffusion-webui-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable-diffusion-webui-forge\ldm_patched\ldm\modules\attention.py", line 399, in forward
    out = optimized_attention(q, k, v, self.heads)
  File "E:\stable-diffusion-webui-forge\ldm_patched\ldm\modules\attention.py", line 332, in attention_pytorch
    out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)
RuntimeError: Native API failed. Native API returns: -997 (Command failed to enqueue/execute) -997 (Command failed to enqueue/execute)
Native API failed. Native API returns: -997 (Command failed to enqueue/execute) -997 (Command failed to enqueue/execute)
*** Error completing request
    Traceback (most recent call last):
      File "E:\stable-diffusion-webui-forge\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
    TypeError: 'NoneType' object is not iterable

---

This is with SDXL, ADetailer/HiresFix disabled, hypertile enabled

Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:20<00:00,  1.01s/it]
NeverOOM Enabled for UNet (always maximize offload)████████████████████████████████████| 20/20 [00:20<00:00,  1.03it/s]
To load target model SDXL
Begin to load 1 model
ERROR diffusion_model.output_blocks.0.1.transformer_blocks.0.ff.net.0.proj.weight [enforce fail at C:\Users\vfirs\projects\ipex-standalone\pytorch\c10\core\impl\alloc_cpu.cpp:72] data. DefaultCPUAllocator: not enough memory: you tried to allocate 52428800 bytes.
ERROR diffusion_model.output_blocks.0.1.transformer_blocks.1.ff.net.0.proj.weight [enforce fail at C:\Users\vfirs\projects\ipex-standalone\pytorch\c10\core\impl\alloc_cpu.cpp:72] data. DefaultCPUAllocator: not enough memory: you tried to allocate 52428800 bytes.
Traceback (most recent call last):
  File "E:\stable-diffusion-webui-forge\modules_forge\main_thread.py", line 37, in loop
    task.work()
  File "E:\stable-diffusion-webui-forge\modules_forge\main_thread.py", line 26, in work
    self.result = self.func(*self.args, **self.kwargs)
  File "E:\stable-diffusion-webui-forge\modules\txt2img.py", line 111, in txt2img_function
    processed = processing.process_images(p)
  File "E:\stable-diffusion-webui-forge\modules\processing.py", line 752, in process_images
    res = process_images_inner(p)
  File "E:\stable-diffusion-webui-forge\modules\processing.py", line 922, in process_images_inner
    samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
  File "E:\stable-diffusion-webui-forge\modules\processing.py", line 1275, in sample
    samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
  File "E:\stable-diffusion-webui-forge\modules\sd_samplers_kdiffusion.py", line 207, in sample
    sampling_prepare(self.model_wrap.inner_model.forge_objects.unet, x=x)
  File "E:\stable-diffusion-webui-forge\modules_forge\forge_sampler.py", line 105, in sampling_prepare
    model_management.load_models_gpu(
  File "E:\stable-diffusion-webui-forge\ldm_patched\modules\model_management.py", line 494, in load_models_gpu
    loaded_model.model_load(async_kept_memory)
  File "E:\stable-diffusion-webui-forge\ldm_patched\modules\model_management.py", line 332, in model_load
    raise e
  File "E:\stable-diffusion-webui-forge\ldm_patched\modules\model_management.py", line 328, in model_load
    self.real_model = self.model.patch_model(device_to=patch_model_to) #TODO: do something with loras and offloading to CPU
  File "E:\stable-diffusion-webui-forge\ldm_patched\modules\model_patcher.py", line 216, in patch_model
    out_weight = self.calculate_weight(self.patches[key], temp_weight, key).to(weight.dtype)
RuntimeError: [enforce fail at C:\Users\vfirs\projects\ipex-standalone\pytorch\c10\core\impl\alloc_cpu.cpp:72] data. DefaultCPUAllocator: not enough memory: you tried to allocate 26214400 bytes.
[enforce fail at C:\Users\vfirs\projects\ipex-standalone\pytorch\c10\core\impl\alloc_cpu.cpp:72] data. DefaultCPUAllocator: not enough memory: you tried to allocate 26214400 bytes.
*** Error completing request
    Traceback (most recent call last):
      File "E:\stable-diffusion-webui-forge\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
    TypeError: 'NoneType' object is not iterable

---

And I would believe this is the allocation error that you have mentioned. It doesnt happen when I generate but happens after a couple generations. Also if I dont have Never OOM UNet enabled, SDXL doesnt work as "stable" as with UNet and it crashes still with UNet.
This is with SDXL, ADetailer/HiresFix disabled, hypertile disabled and UNet enabled.
Edit: I have deleted my prompts from the logs for making it easier to read.

[redrum-llik]

RuntimeError: Native API failed. Native API returns: -997 - I am seeing this once I had an OOM at least once, until the process is restarted. Was this the first exception you saw, or a subsequent one?

The second sounds like the model was being offloaded from VRAM (to regular RAM, I suppose), but allocation has failed. Do you have any swap enabled on your system? I had some issues until I have set my swap to be twice as large as my RAM - even though it is situated on HDD, it seems to help.

[HyunJae5463]

I know I said it fixed the issue, but I still get the error now and then randomly while generating images, only fix is to restart the webui and hope it doesnt do it again. It's more common when using SDXL or SDXL sometimes doesnt work at all. Pretty random. But when it works its making wonders.

Sounds like the same issue I have. I just get "Error" but no further console log. Just have to restart SD. This only happens when I use any Face Restoration. It doesn't happen on SD.Next tho. Only on Forge and A1111.