Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add ability to switch output languages for multilingual models #69 #72

Closed
wants to merge 27 commits into from
Closed

Add ability to switch output languages for multilingual models #69 #72

Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
89fbc24
Testing Circleci on main
Jul 19, 2023
93f9643
Merge branch 'main' of https://github.com/SamDewriter/SimulEval
Jul 19, 2023
fd41dc2
Testing Circleci on main
Jul 19, 2023
4becf02
Testing Circleci on main
Jul 19, 2023
cc893ef
Testing Circleci on main
Jul 19, 2023
a6f00f8
Testing Circleci on main
Jul 19, 2023
0f41351
correct Circle config
Jul 20, 2023
78d13d0
correct Circle config
Jul 20, 2023
98ae6ec
correct Circle config
Jul 20, 2023
cdffc8d
correct Circle config
Jul 20, 2023
cacfbc9
Revert "[demo] s2t + s2s agent pipelines (#58)"
Jul 20, 2023
f97cdfa
resolve branch changes
Aug 14, 2023
4ddf84a
add target language
Aug 15, 2023
69e5816
add target language as a parameter
Aug 15, 2023
812bcbc
Test dynamic language
Aug 18, 2023
233aa35
Switch language dynamically
Aug 18, 2023
f15634c
Add ability to switch output language (#69)
Aug 18, 2023
c06fb0c
Add tgt language argument
Aug 18, 2023
5e0165a
Add Namespace to args argument (#69)
Aug 18, 2023
c43e9da
Modify code to read target language from a file
Aug 18, 2023
bbe6a88
Add ability to switch input language
Aug 18, 2023
03c06b1
Add a tgt-lang file to test
Aug 18, 2023
9d97d18
Add tgt_lang to instance
Aug 20, 2023
3e5fbe6
Add tgt_lang to AgentStates
Aug 25, 2023
6d06a8e
States
Aug 25, 2023
50efb42
Add tgt_lang from state to test (#69)
Aug 25, 2023
1c07b35
Target language to test
Aug 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: TestPyPI Publish

on:
push:
branches:
- main

jobs:
test_pypi_publish:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2

- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.x

- name: Install dependencies
run: |
sudo apt-get update
sudo apt-get install -y python3-pip
pip install pipenv
pipenv install twine

- name: Build and Publish
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.TEST_PYPI_TOKEN }}
REPOSITORY_URL: https://test.pypi.org/legacy/
run: |
python setup.py sdist bdist_wheel
pipenv run twine upload --repository-url $REPOSITORY_URL dist/*
237 changes: 0 additions & 237 deletions examples/demo/silero_vad.py
View file
Open in desktop

This file was deleted.

75 changes: 0 additions & 75 deletions examples/speech_to_speech/english_counter_agent.py
View file
Open in desktop

This file was deleted.

10 changes: 0 additions & 10 deletions examples/speech_to_speech_demo/english_counter_pipeline.py
View file
Open in desktop

This file was deleted.

11 changes: 0 additions & 11 deletions examples/speech_to_speech_demo/readme.md
View file
Open in desktop

This file was deleted.

9 changes: 6 additions & 3 deletions examples/speech_to_text/counter_in_tgt_lang_agent.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,16 @@ class CounterInTargetLanguage(SpeechToTextAgent):
def __init__(self, args):
super().__init__(args)
self.wait_seconds = args.wait_seconds
self.tgt_lang = args.tgt_lang
if args is not None:
with open(args.tgt_lang, "r") as file:
tgt_lang = file.read()
self.tgt_lang = tgt_lang
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this part here as a fallback? It is fine to leave it in as a fallback, but if this is the sole logic for getting tgt_lang in this agent then it won't meet our purpose.

We primarily want to get the tgt lang param dynamically passed to the policy method instead of solely getting set at initialization. You understand that solely getting set at initialization means it will remain static, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I added this part is just to preprocess the tgt_lang because we are reading it from a file. I came to this conclusion after several trials and errors. I noticed that before adding the logic, the evaluation works well but the tgt_lang is not being passed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing I'd also need clarification on is that in the counter_in_tgt_lang_agent, the tgt_lang is being passed to the class after inheriting from the parent class Agent, which means that the tgt_lang is not implemented in the parent class. I suppose the right thing will be to implement it in the parent class so that all the children classes will automatically have it

Copy link
Contributor

@ibanesh ibanesh Aug 23, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to set tgt_lang in the init method by the end of this effort, so don't worry about moving this attribute to the parent class for now.

If the objective is still not clear to you, imagine that

SimulEval/examples/speech_to_text/counter_in_tgt_lang_agent.py

Lines 22 to 24 in c7a1749

parser.add_argument(
"--tgt-lang", default="en", type=str, choices=["en", "es", "de"]
)
and

SimulEval/examples/speech_to_text/counter_in_tgt_lang_agent.py

Line 17 in c7a1749

self.tgt_lang = args.tgt_lang
will be removed. We want to be able to get the tgt_lang passed down to the policy method as part of the states, something like:

def policy(self, states: Optional[AgentStates] = None):
    ....
    ....
    tgt_lang = states.tgt_lang
    if tgt_lang == "en": 
        prediction += "seconds"
    elif tgt_lang == "es":
        prediction += "segundos"
    elif tgt_lang == "de":
        prediction += "sekunden"
    else:
        prediction += "<unknown>"
    ....
    ....

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh! Now I understand the goal. Thanks for the clarification!


@staticmethod
def add_args(parser):
parser.add_argument("--wait-seconds", default=1, type=int)
parser.add_argument(
"--tgt-lang", default="en", type=str, choices=["en", "es", "de"]
"--tgt-lang"
)

def policy(self, states: Optional[AgentStates] = None):
Expand All @@ -47,4 +50,4 @@ def policy(self, states: Optional[AgentStates] = None):
return WriteAction(
content=prediction,
finished=states.source_finished,
)
)
Loading
Loading