Call For Challenges

A great way to advance the state of the art in a given domain is to create competition. We invite you to propose an ISWC 2023 Challenge, in which you define an open competition on a problem of your choice within the Semantic Web domain.

Important Dates – All deadlines are 23:59 AoE (anywhere on earth)

Topics

For ISWC 2023, Challenge proposals are invited for any challenge involving Semantic Web tasks, including but not limited to

• Ontology and knowledge graph alignment
• Ontology and knowledge graph quality assurance (QA)
• Knowledge graph construction and refinement
• Query and reasoning scalability
• Open information extraction
• Semantic Web and data mining
• Semantic Web, machine learning and neuro-symbolic AI
• Link prediction
• Question answering
• Stream processing and reasoning
• Semantic table understanding

ISWC 2023 will be an in-person conference that will take place on 6-10 November in Athens. Recent Challenge examples can be found at:
https://iswc2022.semanticweb.org/index.php/challenges/
https://iswc2021.semanticweb.org/challenges

Proposal Stage

ISWC 2023 will run multiple challenges with the aim of evaluating and comparing software solutions for the Semantic Web and Knowledge Graphs in a systematic way. In order to propose a challenge, we ask you to:

• specify a task to be addressed
• provide an evaluation dataset
• define evaluation measures to compare the performance of participating systems

Showcase Stage

At ISWC 2023, each challenge will receive one slot during the main conference, during which organizers present the respective challenge, participating systems with accepted solutions as well as the winner of the challenge. This means that at least one of the organizers should register and be present at the conference. Participants are not required to register or attend the conference. Participants will also have the option to present their systems during the poster and demo session. The winners from each challenge will receive a certificate.

Proposal Submission Guidelines

The contents of a proposal should include the following (in 2–4 pages)

• names and affiliations of the organizers
• description of the challenge, including its relevance to the Semantic Web community
• process for selecting, acquiring, and preparing the data for training and evaluation, including conditions of availability. It is strongly recommended, but not required, to describe the data in relation to the FAIR principles.
• procedure for evaluating the performance of participating systems, including evaluation metrics and the availability of evaluation software
• a timeline based on the template for all Challenges provided below
• number of expected participants with supporting evidence (expression of interests, discussions in forum/mailing lists, attendance in previous editions, etc.)
• plan for dissemination of the challenge (targeted mailing lists, social media, etc.)
• any other relevant information, including previous evaluations for the proposed challenge

Selection criteria

• Relevance to the Semantic Web community
• Potential number of interested participants
• Rigor and transparency of task description and evaluation procedure

Please submit through EasyChair and select the track “Challenge-Proposal”.
https://easychair.org/conferences/?conf=iswc2023

Tentative timeline template for Challenge Organizers

The tentative timeline template for organizing a Challenge is as follows: All deadlines are 23:59 AoE (anywhere on earth).

Note the deadlines and mostly the dataset release deadline and system submission deadline can be flexibly adjusted according to the challenge, especially if the challenge includes several evaluation rounds (e.g., http://www.cs.ox.ac.uk/isg/challenges/sem-tab/2020/index.html)

Semantic Web Challenge Chairs

Valentina Ivanova
RISE Research Institutes of Sweden, Sweden
valentina.ivanova@ri.se

Wen Zhang
Zhejiang University, China
zhang.wen@zju.edu.cn

Acknowledgments: The text of this call for Semantic Web Challenges is reusing most of the text of the call for Semantic Web Challenges from ISWC 2022 by Catia Pesquita and Daniele Dell’Aglio.