Submitting to the leaderboard

To submit your method to the leaderboard, contact kdmarino [at] cs [dot] cmu [dot] edu and include (1) the OK-VQA test results output file, (2) a name for the method, (3) a github repo or paper link, (4) your institution.


Evaluation

We follow the same format as VQA for evaluation. See the instructions at README and use this code for evaluation.

OK-VQA Leaderboard

Rank Model Overall Accuracy

1

Jun 25, 2019
MUTAN + AN

AI2 and CMU

27.84

2

Jun 25, 2019
MUTAN

26.41

3

Jun 25, 2019
BAN + AN

AI2 and CMU

25.61

4

Jun 25, 2019
BAN

25.17

5

Jun 25, 2019
MLP

20.67

6

Jun 25, 2019
Q only

14.93