Submitting to the leaderboard

To submit your method to the leaderboard, contact kdmarino [at] cs [dot] cmu [dot] edu and include (1) the OK-VQA test results output file, (2) a name for the method, (3) a github repo or paper link, (4) your institution.


Evaluation

We follow the same format as VQA for evaluation. See the instructions at README and use this code for evaluation.

OK-VQA Leaderboard

Rank Model Overall Accuracy

1

ConceptBERT

Ecole Poyltechnique

33.66

2

MUTAN + AN

AI2 and CMU

27.84

3

MUTAN

26.41

4

BAN + AN

AI2 and CMU

25.61

5

BAN

25.17

6

MLP

20.67

7

Q only

14.93