Tough Tables (2T) is a dataset designed to evaluate table annotation approaches in solving the CEA and CTA tasks. The dataset is compliant with the data format used in SemTab2019, and it can be used as an additional dataset without any modification. The target knowledge graph is DBpedia 2016-10. Check out the 2T GitHub repository for more details about the dataset generation. New in v1.0: We release 2T_WD, the 2T version for Wikidata! The target knowledge graph is Wikidata (online instance) and the dataset complies with the SemTab2020 data format. This work is based on the following paper: Cutrona, V., Bianchi, F., Jimenez-Ruiz, E. and Palmonari, M. (2020). Tough Tables: Carefully Evaluating Entity Linking for Tabular Data. ISWC 2020, LNCS 12507, pp. 1–16. Note on License: This dataset includes data from the following sources. Refer to each source for license details: - Wikipedia https://www.wikipedia.org/ - DBpedia https://dbpedia.org/ - Wikidata https://www.wikidata.org/ - SemTab2019 https://doi.org/10.5281/zenodo.3518539 - GeoDatos https://www.geodatos.net - The Pudding https://pudding.cool/ - Offices.net https://offices.net/ - DATA.GOV https://www.data.gov/ THIS DATA IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. Changelog: v1.0 New Wikidata version (2T_WD) Fix header for tables CTRL_DBP_MUS_rock_bands_labels.csv and CTRL_DBP_MUS_rock_bands_labels_NOISE2.csv (column 2 was reported with id 1 in target - NOTE: the affected column has been removed from the SemTab2020 evaluation) Remove duplicated entries in tables Remove rows with wrong values (e.g., the Kazakhstan entity has an empty name "''") Many rows and noised columns are shuffled/changed due to the random noise generator algorithm Remove row "Florida","Floorida","New York, NY" from TOUGH_WEB_MISSP_1000_us_cities.csv (and all its NOISE1 variants) Fix header of tables: CTRL_WIKI_POL_List_of_current_monarchs_of_sovereign_states.csv CTRL_WIKI_POL_List_of_current_monarchs_of_sovereign_states_NOISE2.csv TOUGH_T2D_BUS_29414811_2_4773219892816395776___videogames_developers.csv TOUGH_T2D_BUS_29414811_2_4773219892816395776___videogames_developers_NOISE2.csv v0.1-pre First submission. It contains only tables, without GT and Targets.

GT and Target files have different formats in 2T and 2T_WD (CEA). 2T complies with the SemTab2019 CEA format (tab_id, col_id, row_id, entity), while 2T_WD follows the SemTab2020 CEA format (tab_id, row_id, col_id, entity). Visit the SemTab challenge website (https://www.cs.ox.ac.uk/isg/challenges/sem-tab/) for more details.