Skip to content

Question answering system for factoid based questions

Notifications You must be signed in to change notification settings

akashmondal1810/factoid-QA

Repository files navigation

factoid-QA

Setup:

  • Install dependencies pip install -r dependencies.txt
  • Download the BERT Pretrained model trained on SQuAD from here and extract it inside BERTap folder
  • Download the glove.6B.300d.txt and extract it inside word2vec_repo/glove6b folder. Also each corpus need to start with a line containing the vocab size and the vector size in that order. So in our case add this line "400000 300" as the first line of glove.6B.300d.txt.
  • Download nltk stopwords:
    import nltk
    nltk.download('stopwords')
  • Run processed_data/txt_to_csv.py to generate the processed dataframe
  • Run pass_ret/txt_to_tfidf_dict.py to generate the tf-idf values over the corpus
  • Run word2vec_repo/gen_w2v_modal.py to generate the word2vec model

Default directory structure

QAsys
├── data (or $QA_Knowledge_DATA)
|   └── <unique/non overlapping doc name>.txt
├── word2vec_repo   
│   ├── glove6b
│   |   └── glove.6B.300d.txt
│   └── domain2vec.bin (or $TRAINED_W2V_MODEL)
├── pass_ret
│   └── data_tfidf.json
└── BERTap
        └── model
            ├──bert_config.json
            └── pytorch_model.bin
                (or $TRAINED_SQuAD_MODEL)

Execute:

A basic User Interface using Python's Tkinter library is provided .

  • python UI_app.py for answer

About

Question answering system for factoid based questions

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages