Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Image feature extraction on Hub as a task #402

Merged
merged 5 commits into from
Feb 19, 2024

Conversation

merveenoyan
Copy link
Contributor

This PR adds image feature extraction to Hub.
Icon looks like below 👇

Screenshot 2023-12-07 at 15 49 53

Copy link
Collaborator

@gary149 gary149 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

Copy link
Contributor

@osanseviero osanseviero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would add a filter/tag that leads to 0 models on the Hub (see https://huggingface.co/models?other=image-feature-extraction&sort=trending). We should first:

  • add the tag to the corresponding transformers and timm repos. Which models would have this tag?
  • explore adding the pipeline to transformers

From a quick look, most models in https://huggingface.co/models?pipeline_tag=feature-extraction&other=vision are candidates to be image-feature-extraction. These models are usually automatically tagged, so step 3 in https://www.notion.so/huggingface2/How-do-pipelines-get-determined-on-the-Hub-2fdc2f3ae1a44690b43e28adbe326d11 needs to be done. This also needs to align with the transformers team first to see if there's a need for this pipeline/task cc @LysandreJik @amyeroberts

There is a very related discussion on huggingface/transformers#20830 (comment) about this, we should check if feature-extraction works for these

@merveenoyan
Copy link
Contributor Author

merveenoyan commented Dec 9, 2023

@osanseviero I made a script to run over timm models so I will be autotagging them after this PR is merged, can do the same for transformers feature-extraction models that are for vision. There's a lot of backbones on Hub, they're not explored well among other modalities and I think this is detrimental to UX of people using Hub for vision, this is discussed in the issue linked to this PR. (this issue was mainly raised by Ross & Niels BTW so not sure Amy and Lysandre are in favor or aware in general)

@osanseviero
Copy link
Contributor

can do the same for transformers feature-extraction models that are for vision

Please carefully review the resources/Notion linked above. We should not open PRs for the vision-tagged transformers models. The pipeline_tag in transformers models is automatically detected, so actually we need to add ImageFeatureExtractionPipeline there and update the mapping, which will automatically update the tags for transformers repos

Copy link
Contributor

@osanseviero osanseviero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@merveenoyan merveenoyan merged commit 705588e into huggingface:main Feb 19, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants