Merge pull request #554 from vespa-engine/jobergum/fallback-to-title #266
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
name: pyvespa documentation search feed | |
on: | |
push: | |
branches: [ master ] | |
env: | |
DATA_PLANE_PUBLIC_KEY : ${{ secrets.VESPA_TEAM_DATA_PLANE_PUBLIC_CERT }} | |
DATA_PLANE_PRIVATE_KEY: ${{ secrets.VESPA_TEAM_DATA_PLANE_PRIVATE_KEY }} | |
jobs: | |
build: | |
runs-on: ubuntu-latest | |
steps: | |
- uses: actions/checkout@v3 | |
- uses: ruby/setup-ruby@v1 | |
with: | |
ruby-version: 3.1 | |
bundler-cache: true | |
- uses: actions/setup-python@v4 | |
with: | |
python-version: '3.x' | |
- name: Install Sphinx | |
run: | | |
sudo apt-get install -y pandoc | |
python3 -m pip install --upgrade pip | |
python3 -m pip install -r docs/sphinx/source/requirements-feed.txt | |
- name: Build site | |
run: | | |
sphinx-build -E -D nbsphinx_allow_errors=True -b html docs/sphinx/source docs/sphinx/build | |
find docs/sphinx/build/ -name \*.html | while read f; do (echo -e "---\n---\n"; cat ${f})>${f}.new; mv ${f}.new ${f}; done | |
# Strip jekyll liquid macros | |
sed -i.orig 's/{%/{ %/g; s/%}/% }/g; s/{{/{ {/g; s/}}/} }/g' docs/sphinx/build/*.html ; rm docs/sphinx/build/*.orig | |
bundle exec jekyll build -s docs/sphinx/build/ -p _plugins-vespafeed --config _config.yml | |
- name: Install dependencies | |
run: | | |
pip3 install PyYAML spacy mmh3 requests html5lib beautifulsoup4 markdownify tiktoken | |
- name: Feed site | |
run: | | |
./feed_to_vespa.py _config.yml | |
- name: Feed paragraphs site | |
run: | | |
./feed-split.py pyvespa_index.json https://pyvespa.readthedocs.io questions.jsonl | |
./feed_to_vespa.py _paragraphs_config.yml | |
- name: Feed suggestions | |
run: | | |
./feed_to_vespa.py _suggestions_config.yml |