Web Data Commons Training and Test Sets for Large-Scale Product Matching - Version 2.0

Principal Investigator(s):  View help for Principal Investigator(s) Ralph Peeters, University of Mannheim (Germany); Anna Primpeli, University of Mannheim (Germany); Christian Bizer, University of Mannheim (Germany)

Version:  View help for Version V1

Published:  View help for Published Date November 26, 2020

Citation

Project Citation: 

Peeters, Ralph, Primpeli, Anna, and Bizer, Christian. Web Data Commons Training and Test Sets for Large-Scale Product Matching - Version 2.0. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2020-11-26. https://doi.org/10.3886/E127481V1

Persistent URL:  http://doi.org/10.3886/E127481V1

Project Description

Project Title:  View help for Project Title Web Data Commons Training and Test Sets for Large-Scale Product Matching - Version 2.0
Summary:  View help for Summary Many e-shops have started to mark-up product data within their HTML pages using the schema.org vocabulary. The Web Data Commons project regularly extracts such data from the Common Crawl, a large public web crawl. The Web Data Commons Training and Test Sets for Large-Scale Product Matching contain product offers from different e-shops in the form of binary product pairs (with corresponding label ?match? or ?no match?) for four product categories, computers, cameras, watches and shoes. In order to support the evaluation of machine learning-based matching methods, the data is split into training, validation and test sets. For each product category, we provide training sets in four different sizes (2.000-70.000 pairs). Furthermore there are sets of ids for each training set for a possible validation split (stratified random draw) available. The test set for each product category consists of 1.100 product pairs. The labels of the test sets were manually checked while those of the training sets were derived using shared product identifiers from the Web weak supervision. The data stems from the WDC Product Data Corpus for Large-Scale Product Matching - Version 2.0 which consists of 26 million product offers originating from 79 thousand websites. For more information and download links for the corpus itself, please follow the links below.
Original Distribution URL:  View help for Original Distribution URL http://webdatacommons.org/largescaleproductcorpus/v2/index.html

Scope of Project

Subject Terms:  View help for Subject Terms schema.org; product matching; entity matching; identity resolution; record linkage; e-commerce


Name Size File Type Download/
Preview
file wdc_lspc_v2_sets.zip 394 MB application/zip Download
Download this project

Published Versions

V1 [2020-11-26]

Export Metadata

OAI-PMH

DDI 2.5

Report a Problem

Found a serious problem with the data, such as disclosure risk or copyrighted content? Let us know.

This material is distributed exactly as it arrived from the data depositor. ICPSR has not checked or processed this material. Users should consult the investigator(s) if further information is desired.