Submitting to the leaderboard

To submit your method to the leaderboard, contact okvqa.comm [at[ gmail [dot] com and include (1) the OK-VQA test results output file, (2) a name for the method, (3) a github repo or paper link, (4) your institution.


Evaluation

We follow the same format as VQA for evaluation. See the instructions at README and use this code for evaluation.

OK-VQA Leaderboard

Rank Model Overall Accuracy

1

KAT

Microsoft, CMU & Yale

54.41

2

PICa

Microsoft

48.0

3

CBM

Hitz Center, UPV

47.9

4

MCAN

Hangzhou Dianzi University

44.65

5

UnifER

NUS

42.13

6

MAVEx

UT Austin & AI2

41.37

7

KRISP

FAIR & CMU

38.90

8

ConceptBERT

Ecole Poyltechnique

33.66

9

MUTAN + AN

AI2 and CMU

27.84

10

MUTAN

26.41

11

BAN + AN

AI2 and CMU

25.61

12

BAN

25.17

13

MLP

20.67

14

Q only

14.93