Code

Litellm perf jan4 2#7657

Code

Open

ishaan-jaff wants to merge 17 commits intolitellm_perf_jan4BerriAI/litellm:litellm_perf_jan4from litellm_perf_jan4_2BerriAI/litellm:litellm_perf_jan4_2Copy head branch name to clipboard

+94-72

Commits

Commits on Jan 9, 2025

• use 2 workers

  ishaan-jaff

  committed
  d68ff91

  Copy full SHA for d68ff91

• use uvloop

  ishaan-jaff

  committed
  9201172

  Copy full SHA for 9201172

• fix uvloop

  ishaan-jaff

  committed
  906f5c2

  Copy full SHA for 906f5c2

• fix og issue aith transform

  ishaan-jaff

  committed
  42193d3

  Copy full SHA for 42193d3

• use uvloop

  ishaan-jaff

  committed
  6b257e7

  Copy full SHA for 6b257e7

• fix cache handler

  ishaan-jaff

  committed
  9620c78

  Copy full SHA for 9620c78

• use asyncio.create_task for track dep metrics

  ishaan-jaff

  committed
  abe1078

  Copy full SHA for abe1078

• remove executor.submit

  ishaan-jaff

  committed
  1381ff0

  Copy full SHA for 1381ff0

• fix _response_cost_calculator

  ishaan-jaff

  committed
  b89907d

  Copy full SHA for b89907d

• remove set_response_headers

  ishaan-jaff

  committed
  0e209d1

  Copy full SHA for 0e209d1

• remove pre call

  ishaan-jaff

  committed
  32b6df8

  Copy full SHA for 32b6df8

• remove copy.copy proxy_server_request

  ishaan-jaff

  committed
  e83e12a

  Copy full SHA for e83e12a

• use cleaned up user api key auth

  ishaan-jaff

  committed
  5a083f0

  Copy full SHA for 5a083f0

Commits on Jan 10, 2025

• use await _read_request_body

  ishaan-jaff

  committed
  2732613

  Copy full SHA for 2732613

• latency fix ready from user_api_key_dict

  ishaan-jaff

  committed
  ab2624a

  Copy full SHA for ab2624a

• use 3 workers

  ishaan-jaff

  committed
  57def2c

  Copy full SHA for 57def2c

• tune workers to 5

  ishaan-jaff

  committed
  1f95a91

  Copy full SHA for 1f95a91