Code

feat: added evaluation script#14

Code

Open

Markus28 wants to merge 41 commits intomainjina-ai/mteb-de:mainfrom feat-eval-scriptjina-ai/mteb-de:feat-eval-scriptCopy head branch name to clipboard

+379-294

Commits

Commits on Oct 31, 2023

• add gerdalir dataset

  guenthermi

  committed
  77c1586

  Copy full SHA for 77c1586

• fix: gerdalir dataset

  guenthermi

  committed
  bd5e945

  Copy full SHA for bd5e945

• fix: lang from en to de

  guenthermi

  committed
  3eb5c71

  Copy full SHA for 3eb5c71

Commits on Nov 1, 2023

• feat: add wikiclir en-de task

  violenil

  committed
  2d00b58

  Copy full SHA for 2d00b58

• chore: add ir datasets to requirements

  violenil

  committed
  830cd60

  Copy full SHA for 830cd60

• refactor: limit queries to 10k

  violenil

  committed
  6730483

  Copy full SHA for 6730483

Commits on Nov 2, 2023

• refactor: update description of task with limit

  violenil

  committed
  11866e6

  Copy full SHA for 11866e6

• Merge pull request #2 from jina-ai/feat-wikiclir-de

  Show description for cd955ec
  violenil

  authored
  cd955ec

  Copy full SHA for cd955ec

Commits on Nov 3, 2023

• solve merge conflict

  guenthermi

  committed
  151668c

  Copy full SHA for 151668c

• revert style changes

  guenthermi

  committed
  2b9c20f

  Copy full SHA for 2b9c20f

• Merge pull request #1 from jina-ai/add-gerdalir-dataset

  Show description for 262ab27
  guenthermi

  authored
  262ab27

  Copy full SHA for 262ab27

• add XMarket dataset

  guenthermi

  committed
  1a9278f

  Copy full SHA for 1a9278f

• add xmarket to init file

  guenthermi

  committed
  1f0c797

  Copy full SHA for 1f0c797

Commits on Nov 6, 2023

• feat: add german stsbenchmarksts task

  violenil

  committed
  5818581

  Copy full SHA for 5818581

• feat: add revision id

  guenthermi

  committed
  dacf64a

  Copy full SHA for dacf64a

• feat: update revision id

  violenil

  committed
  5049d05

  Copy full SHA for 5049d05

• refactor: update revision id after changes in scores

  violenil

  committed
  14448de

  Copy full SHA for 14448de

• Merge pull request #4 from jina-ai/feat-add-sts-de

  Show description for 7e05fc0
  violenil

  authored
  7e05fc0

  Copy full SHA for 7e05fc0

• add paws x dataset

  guenthermi

  committed
  628fbb1

  Copy full SHA for 628fbb1

Commits on Nov 7, 2023

• Merge pull request #3 from jina-ai/add-xmarket-de

  Show description for b360db4
  guenthermi

  authored
  b360db4

  Copy full SHA for b360db4

• add GermanDPR dataset

  guenthermi

  committed
  cbd8d95

  Copy full SHA for cbd8d95

• fix loading

  guenthermi

  committed
  7316147

  Copy full SHA for 7316147

• solve merge conflict

  guenthermi

  committed
  b8e5afc

  Copy full SHA for b8e5afc

• Update mteb/tasks/Retrieval/GermanDPRRetrieval.py

  Show description for 30a9611
  

  guenthermi

  and

  jupyterjazz

  authored
  30a9611

  Copy full SHA for 30a9611

Commits on Nov 9, 2023

• Merge pull request #5 from jina-ai/add-paws-x-dataset

  Show description for 3de1bb1
  guenthermi

  authored
  3de1bb1

  Copy full SHA for 3de1bb1

• feat: add miracl reranking task for german

  violenil

  committed
  9e2ac3e

  Copy full SHA for 9e2ac3e

• refactor: cleanup task

  violenil

  committed
  60a61ac

  Copy full SHA for 60a61ac

• Add ir_datasets as dependency

  Markus28

  committed
  55155d6

  Copy full SHA for 55155d6

• Merge pull request #8 from jina-ai/add_ir_dependency

  Show description for 969e202
  Markus28

  authored
  969e202

  Copy full SHA for 969e202

• Fix: Adding MTEB_SINGLE_GPU environment variable

  Markus28

  committed
  c6733bb

  Copy full SHA for c6733bb

• Merge pull request #9 from jina-ai/fix_multi_gpu

  Show description for a6c9cb7
  Markus28

  authored
  a6c9cb7

  Copy full SHA for a6c9cb7

Commits on Nov 10, 2023

• fix: Use MTEB_SINGLE_GPU environment variable also in BeIRTask.py (#10)

  Markus28

  authored
  cf2c380

  Copy full SHA for cf2c380

Commits on Nov 13, 2023

• Merge pull request #6 from jina-ai/add-german-dpr

  Show description for 9aea329
  guenthermi

  authored
  9aea329

  Copy full SHA for 9aea329

• Merge pull request #7 from jina-ai/feat-add-miracl-reranking

  Show description for 19f77dd
  violenil

  authored
  19f77dd

  Copy full SHA for 19f77dd

Commits on Nov 14, 2023

• prevent duplicate pos docs

  guenthermi

  committed
  2822839

  Copy full SHA for 2822839

Commits on Jan 11, 2024

• fix: use test split in MIRACL (#13)

  Show description for 76b7b45
  Markus28

  authored
  76b7b45

  Copy full SHA for 76b7b45

Commits on Jan 15, 2024

• feat: added evaluation script

  Markus28

  committed
  0a4a7e5

  Copy full SHA for 0a4a7e5

• feat: renamed script, updated README

  Markus28

  committed
  545746d

  Copy full SHA for 545746d

• feat: fixed phrasing in README

  Markus28

  committed
  5b9f053

  Copy full SHA for 5b9f053

• feat: fixed phrasing in README

  Markus28

  committed
  3b9ff9d

  Copy full SHA for 3b9ff9d

• feat: added back WikiCLIR

  Markus28

  committed
  64f03b5

  Copy full SHA for 64f03b5