[PRE REVIEW]: Multiple Choice Reasoning via. Process of Elimination using Multi-Modal Models

[editorialbot]

Submitting author: @hellokayas (Sayak Chakrabarty)
Repository: https://github.com/souradipp76/MM-PoE
Branch with paper.md (empty if default branch): main
Version: v0.1.2
Editor: @faroit
Reviewers: Pending
Managing EiC: Chris Vernon

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/918fbea56c25d3949f988a8c8d1cdf42"><img src="https://joss.theoj.org/papers/918fbea56c25d3949f988a8c8d1cdf42/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/918fbea56c25d3949f988a8c8d1cdf42/status.svg)](https://joss.theoj.org/papers/918fbea56c25d3949f988a8c8d1cdf42)

Author instructions

Thanks for submitting your paper to JOSS @hellokayas. Currently, there isn't a JOSS editor assigned to your paper.

@hellokayas if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). You can search the list of people that have already agreed to review and may be suitable for this submission.

Editor instructions

The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:

@editorialbot commands

[editorialbot]

Hello human, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.90  T=0.06 s (935.1 files/s, 168692.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          18            712            578           6738
Bourne Shell                    14             59             68            646
CSV                             10              0              0            495
Markdown                         9            228              0            479
TeX                              1             39              0            271
YAML                             6             21             13            173
make                             1             16              4            104
-------------------------------------------------------------------------------
SUM:                            59           1075            663           8906
-------------------------------------------------------------------------------

Commit count by author:

   102	Souradip Pal
    13	hellokayas
     7	Sayak Chakrabarty
     5	dependabot[bot]
     1	souradipp76

[editorialbot]

Paper file info:

📄 Wordcount for paper.md is 2261

✅ The paper includes a Statement of need section

[editorialbot]

License info:

✅ License found: Apache License 2.0 (Valid open source OSI approved license)

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- None

🟡 SKIP DOIs

- No DOI given, and none found for title: Language models are few-shot learners
- No DOI given, and none found for title: On the consistency of maximum likelihood estimatio...
- No DOI given, and none found for title: Learning by the process of elimination
- No DOI given, and none found for title: Large language models are zero-shot reasoners
- No DOI given, and none found for title: : A Dynamically Adaptive Defense to a Novel Attack...
- No DOI given, and none found for title: Training language models to follow instructions wi...
- No DOI given, and none found for title: Leveraging large language models for multiple choi...
- No DOI given, and none found for title: Beyond the imitation game: Quantifying and extrapo...
- No DOI given, and none found for title: CommonsenseQA: A question answering challenge targ...
- No DOI given, and none found for title: Llama: Open and efficient foundation language mode...
- No DOI given, and none found for title: PINTO: Faithful language reasoning using prompt-ge...
- No DOI given, and none found for title: Self-consistency improves chain of thought reasoni...
- No DOI given, and none found for title: Chain of thought prompting elicits reasoning in la...
- No DOI given, and none found for title: Tree of thoughts: Deliberate problem solving with ...
- No DOI given, and none found for title: Satisfiability-aided language models using declara...
- No DOI given, and none found for title: Calibrate before use: Improving few-shot performan...
- No DOI given, and none found for title: BLIP2: Opt Model
- No DOI given, and none found for title: BLIP2: FLAN-T5-XL Model
- No DOI given, and none found for title: InstructBLIP: Vicuna Model
- No DOI given, and none found for title: GIT: Base VQA Model
- No DOI given, and none found for title: GIT: Base TextVQA Model
- No DOI given, and none found for title: PaliGemma: Fine-Tuned Science QA Model
- No DOI given, and none found for title: PaliGemma: Fine-Tuned VQAV2 Model
- No DOI given, and none found for title: PaliGemma: Fine-Tuned AI2D Model
- No DOI given, and none found for title: Idefics2: 8b Model
- No DOI given, and none found for title: VQA: Visual Question Answering
- No DOI given, and none found for title: Learn to Explain: Multimodal Reasoning via Thought...

❌ MISSING DOIs

- 10.18653/v1/2020.emnlp-main.49 may be a valid DOI for title: Event extraction by answering (almost) natural que...
- 10.18653/v1/2023.acl-long.783 may be a valid DOI for title: Mitigating label biases for in-context learning
- 10.18653/v1/2023.emnlp-main.273 may be a valid DOI for title: POE: Process of Elimination for Multiple Choice Re...
- 10.18653/v1/2021.emnlp-main.564 may be a valid DOI for title: Surface form competition: Why the highest probabil...
- 10.18653/v1/2023.acl-long.129 may be a valid DOI for title: Z-ICL: Zero-shot in-context learning with pseudo-d...
- 10.18653/v1/2022.acl-long.565 may be a valid DOI for title: Coherence boosting: When your pretrained language ...
- 10.18653/v1/2022.acl-long.365 may be a valid DOI for title: Noisy channel language model prompting for few-sho...
- 10.18653/v1/2020.acl-main.441 may be a valid DOI for title: Adversarial NLI: A new benchmark for natural langu...
- 10.18653/v1/d19-1454 may be a valid DOI for title: Social IQa: Commonsense reasoning about social int...
- 10.18653/v1/2022.findings-acl.65 may be a valid DOI for title: Debiasing event understanding for visual commonsen...
- 10.18653/v1/2023.findings-emnlp.811 may be a valid DOI for title: Automatic prompt augmentation and selection with c...
- 10.18653/v1/2023.findings-acl.824 may be a valid DOI for title: Challenging BIG-bench tasks and whether chain-of-t...
- 10.1007/978-3-319-46493-0_15 may be a valid DOI for title: A Diagram is Worth a Dozen Images

❌ INVALID DOIs

- None

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[editorialbot]

Five most similar historical JOSS papers:

Learning from Crowds with Crowd-Kit
Submitting author: @dustalov
Handling editor: @arfon (Active)
Reviewers: @jorgedch, @Mind-the-Cap, @mitchellg
Similarity score: 0.5428

Efficiently Learning Relative Similarity Embeddings with Crowdsourcing
Submitting author: @stsievert
Handling editor: @ajstewartlang (Active)
Reviewers: @hoechenberger, @stain, @jorgedch
Similarity score: 0.5379

CVtreeMLE: Efficient Estimation of Mixed Exposures using Data Adaptive Decision Trees and Cross-Validated Targeted Maximum Likelihood Estimation in R
Submitting author: @blind-contours
Handling editor: @osorensen (Active)
Reviewers: @GaryBAYLOR, @cpalmer718, @wleoncio
Similarity score: 0.5366

modeLLtest: An R Package for Unbiased Model Comparison using Cross Validation
Submitting author: @ShanaScogin
Handling editor: @usethedata (Retired)
Reviewers: @andrewheiss, @usethedata
Similarity score: 0.5364

mikropml: User-Friendly R Package for Supervised Machine Learning Pipelines
Submitting author: @pschloss
Handling editor: @arfon (Active)
Reviewers: @JonnyTran, @FedericoComoglio
Similarity score: 0.5362

⚠️ Note to editors: If these papers look like they might be a good match, click through to the review issue for that paper and invite one or more of the authors before considering asking the reviewers of these papers to review again for JOSS.

[crvernon]

@editorialbot invite @faroit as editor

👋 @faroit are you able to take this one on as editor?

[editorialbot]

Invitation to edit this submission sent!

[faroit]

@crvernon yes, i can edit this one. I won't be able to start before 11/4 though.

[faroit]

@editorialbot assign @faroit as editor

[editorialbot]

Assigned! @faroit is now the editor

[faroit]

@editorialbot remind @faroit in one week

[editorialbot]

Reminder set for @faroit in one week