Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doubts about results #49

Open
zhang2002forwin opened this issue Jul 3, 2024 · 5 comments
Open

Doubts about results #49

zhang2002forwin opened this issue Jul 3, 2024 · 5 comments

Comments

@zhang2002forwin
Copy link

zhang2002forwin commented Jul 3, 2024

I used Van Gogh's starry sky works as a style reference image, but the result I obtained is not the result in the paper. How did I obtain the result in the paper? Figure 1 is my runtime code, and Figure 2 is the result of my inference. Thank you a lot !

1
img4_Without-contentprompt

@haofanwang
Copy link
Member

Which code you are using? Could you try it on our HF demo?

@zhang2002forwin
Copy link
Author

zhang2002forwin commented Jul 4, 2024

Which code you are using? Could you try it on our HF demo?

infer_style.py actually. I just change the model paths and target_blocks=["up_blocks.0.attentions.1","down_blocks.2.attentions.1"]

@haofanwang
Copy link
Member

Here is my result, when I set target_blocks = ["up_blocks.0.attentions.1", "down_blocks.2.attentions.1"] and image = "./assets/4.jpg" and directly run infer_style.py.

image

But I can reproduce similar result as you with other random seed. The thing here is the scale is too strong, set it to a smaller value should be enough. Enjoy! @zhang2002forwin

@zhang2002forwin
Copy link
Author

Here is my result, when I set target_blocks = ["up_blocks.0.attentions.1", "down_blocks.2.attentions.1"] and image = "./assets/4.jpg" and directly run infer_style.py.

image

But I can reproduce similar result as you with other random seed. The thing here is the scale is too strong, set it to a smaller value should be enough. Enjoy! @zhang2002forwin

Oh, thanks. But your result looks like IP-Adpater method in paper.
IP-adapter

@haofanwang
Copy link
Member

That is why I said we should set target_blocks = ["up_blocks.0.attentions.1"]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants