Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finetune stuck at vae encode #413

Open
2 tasks
foreverpiano opened this issue Oct 12, 2024 · 2 comments
Open
2 tasks

Finetune stuck at vae encode #413

foreverpiano opened this issue Oct 12, 2024 · 2 comments
Assignees

Comments

@foreverpiano
Copy link

System Info / 系統信息

11.8 pytorch 2.5.0

Information / 问题信息

  • The official example scripts / 官方的示例脚本
  • My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

def encode_video(video):
    video = video.to(accelerator.device, dtype=vae.dtype).unsqueeze(0)
    video = video.permute(0, 2, 1, 3, 4)  # [B, C, F, H, W]
    latent_dist = vae.encode(video).latent_dist
    return latent_dist
train_dataset.instance_videos = [encode_video(video) for video in train_dataset.instance_videos]

this fuction stuck
https://github.com/THUDM/CogVideo/blob/main/finetune/train_cogvideox_lora.py

Expected behavior / 期待表现

workr normal

@foreverpiano
Copy link
Author

@zRzRzRzRzRzRzR

@foreverpiano
Copy link
Author

不知道是不是数据格式的问题,请问可以提供下video.shape吗,我数据是从disney按照官方的要求下载的

@zRzRzRzRzRzRzR zRzRzRzRzRzRzR self-assigned this Oct 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants