You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This issue has been automatically marked as inactive due to lack of recent activity. Should you believe it remains unresolved and warrants attention, kindly leave a comment on this thread.
此问题由于长期未有新进展而被系统自动标记为不活跃。如果您认为它仍有待解决,请在此帖下方留言以补充信息。
是否已有关于该错误的issue或讨论? | Is there an existing issue / discussion for this?
该问题是否在FAQ中有解答? | Is there an existing answer for this in FAQ?
当前行为 | Current Behavior
微调Qwen_1.8b_chat_int4模型,分别使用lora和qlora方法,合并模型时报错
ValueError: Cannot merge LORA layers when the model is gptq quantized
期望行为 | Expected Behavior
解决该问题
复现方法 | Steps To Reproduce
python qwen_lora_merge.py
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
path_to_adapter="/home/ren/Finetuning/Qwen-1.8-chat/"
new_model_directory="/home/ren/Finetuning/llm_model/Qwen-1_8B-Chat-Int4_law"
model = AutoPeftModelForCausalLM.from_pretrained(
path_to_adapter, # path to the output directory
device_map="auto",
trust_remote_code=True
).eval()merged_model = model.merge_and_unload()
max_shard_size and safe serialization are not necessary.
They respectively work for sharding checkpoint and save the model to safetensors
merged_model.save_pretrained(new_model_directory, max_shard_size="2048MB", safe_serialization=True)
运行环境 | Environment
备注 | Anything else?
no
The text was updated successfully, but these errors were encountered: