Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: A param that send a float when no audio #40

Open
liudger opened this issue May 11, 2023 · 3 comments
Open

Feature: A param that send a float when no audio #40

liudger opened this issue May 11, 2023 · 3 comments

Comments

@liudger
Copy link
Contributor

liudger commented May 11, 2023

Oculus has a sil param that gives a 1 if there is no audio. You can make a param in uLipSync that has silence but this value also goes to 0 if there is no audio. So it would be nice to have a fall back for this.

Is this also something we want for the other system (blendshape, textures etc) or only the animator? Should it be processed in the core? Or can I build something in the animator.

@liudger
Copy link
Contributor Author

liudger commented May 11, 2023

I am a bit confused. When I print out the params weight the sil is at 1. But the animator (controller) parameter is not 1 but 0. Is there another calculation that changes this value?

    void UpdateVowels()
    {
        float sum = 0f;
		float total = 0f;
        var ratios = _info.phonemeRatios;

        foreach (var param in parameters)
        {
            float targetWeight = 0f;
            if (ratios != null && !string.IsNullOrEmpty(param.phoneme))
            {
                ratios.TryGetValue(param.phoneme, out targetWeight);
            }
            float weightVel = param.weightVelocity;
            param.weight = SmoothDamp(param.weight, targetWeight, ref weightVel, minimalValueThreshold);
            param.weightVelocity = weightVel;
            sum += param.weight;
        }

        foreach (var param in parameters)
        {
            param.weight = sum > 0f ? param.weight / sum : 0f;
	    Debug.Log("param: " + param.name + " weight: " + param.weight);
        }
        

image

@liudger
Copy link
Contributor Author

liudger commented May 11, 2023

Ah I found the issue.

    void OnApplyAnimator()
    {
        if (!animator) return;

        foreach (var param in parameters)
        {
            if (param.index < 0) continue;
            animator.SetFloat(param.nameHash, 0f);
        }

        foreach (var param in parameters)
        {
            if (param.index < 0) continue;
            float weight = animator.GetFloat(param.nameHash);
            weight += param.weight * param.maxWeight * volume;
            animator.SetFloat(param.nameHash, weight);
        }
    }

for param sil it shouldn't multiply the volume. What do you think a good workaround for this is?
Maybe a default parameter that is always there for silence? this we could hardcode to skip volume? Or extra button on the parameter that it should skip volume

@liudger
Copy link
Contributor Author

liudger commented Apr 24, 2024

I guess this solved the issue. Need to test it
#57

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant