Deprecation warning for sv.MeanAveragePrecision and other outdated metrics

[patel-zeel]

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Description

What's the problem?

I found two ways of computing mAP metric: sv.MeanAveragePrecision and sv.metrics.MeanAveragePrecision. However, I see here that sv.MeanAveragePrecision and a few other metrics will be deprecated.

Why is it important?

Both sv.MeanAveragePrecision and sv.metrics.MeanAveragePrecision give different results as shown in the following example.

import os
import numpy as np
import supervision as sv
from ultralytics import YOLO

# Download dataset
if not os.path.exists("/tmp/rf_animals"):
    !wget https://universe.roboflow.com/ds/1LLwpXz2td?key=8JnJML5YF6 -O /tmp/rf_animals.zip
    !unzip /tmp/dataset.zip -d /tmp/rf_animals

# Load dataset
dataset = sv.DetectionDataset.from_yolo("/tmp/rf_animals/train/images", "/tmp/rf_animals/train/labels", "/tmp/rf_animals/data.yaml")

# Inference
model = YOLO("yolov8s")
targets, detections = [], []
for image_path, image, target in dataset:
    targets.append(target)
    
    prediction = model(image, verbose=False)[0]
    detection = sv.Detections.from_ultralytics(prediction)
    detection = detection[np.isin(detection['class_name'], dataset.classes)]
    detection.class_id = np.array([dataset.classes.index(class_name) for class_name in detection['class_name']])
    detections.append(detection)
    
# Method #1
mAP = sv.metrics.MeanAveragePrecision().update(detections, targets).compute()
print(f"mAP50: {mAP.map50:.4f}")

# Method #2
mAP = sv.MeanAveragePrecision.from_detections(detections, targets)
print(f"mAP50: {mAP.map50:.4f}")

Output

mAP50: 0.1553
mAP50: 0.2100

As per the docstrings, Method 1 computes the average precision, given the recall and precision curves following https://github.com/rafaelpadilla/Object-Detection-Metrics and Method 2 uses 101-point interpolation (COCO) method. People may end up using a mixture of these methods and may derive wrong conclusions.

Use case

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!