Benchmark Datasets
Datasets Overview
| Dataset | Retrieval Setup | Gallery Type | #Images (Query) | #Images (Gallery) | #Products (Query) | #Products (Gallery) | Access | |
|---|---|---|---|---|---|---|---|---|
|
Au
Automotive v1
Industry Partner
|
Inter | Closed-Set | 2,062 | 19,610 | 379 | 10,379 | Private | |
|
DI
DIY v1
Industry Partner
|
Inter | Closed-Set | 5,008 | 221,376 | 5,008 | 221,376 | Private | |
|
Cl
Clips-and-Connectors v1
Industry Partner
|
Inter | Closed-Set | 3,624 | 200,496 | 453 | 12,531 | Private | |
|
St
Stanford Online Products
Academic
|
Intra | Closed-Set | 60,502 | 60,502 | 11,316 | 11,316 | Public | |
|
Pr
Products-10K
Academic
|
Intra | Closed-Set | 11,200 | 11,200 | 3,450 | 3,450 | Public | |
|
IL
ILIAS
Academic
|
Inter | Open-Set | 1,232 | 504,715 | 2,500 | 1,000 | Public |
Retrieval Setup
Separate query and gallery sets. Query images are typically user-captured or real-world photos, while the gallery holds catalog or reference images. Mimics real-world visual search where users search against a fixed product catalog.
Query and gallery come from the same pool. Each query image should retrieve other images of the same product. Common in academic benchmarks for instance-level retrieval.
Gallery Type
The gallery contains only images of known products; every query has at least one correct match in the gallery. Evaluation measures how well the model ranks the true match among the catalog.
The gallery may include distractors or a large set of candidates; not every gallery product has a corresponding query. Evaluates retrieval at scale with many negatives.