Patent Information Users Group, Inc.

ChEMU evaluation  lab of information extraction tasks over chemical reactions from patents @ 11th Conference and Labs of the Evaluation Forum (CLEF-2020). (Sep. 23-24, 2020, online, Online registration have closed Sep. 18, 2020)

ChEMU (Cheminformatics Elsevier Melbourne University) evaluation  lab,  part of the 11th Conference and Labs of the Evaluation Forum (CLEF-2020) involves extraction tasks over chemical reactions from patents. Task 1—Named entity recognition—involves identifying chemical compounds as well as their role in chemical reaction. Task 2—Event extraction over chemical reactions.([1], see examples at  Fig. 1 and Table 2)

From CLEF 2020 program:

https://clef2020.clef-initiative.eu/index.php?page=Pages/programme.html

Online registration have closed Sep. 18, 2020. Organizer's contacts at eventbrite.co.uk.

Brought to you by the University of Melbourne natural language processing group in the School of Computing and Information System, the Elsevier Content Transformations, Life Science team, and RMIT University, the ChEMU lab series provides an opportunity for development of information extraction models over chemical patents.

ChEMU2020, part of Conference and Labs of the Evaluation Forum 2020 (CLEF 2020), is the first running of our ChEMU lab series. In ChEMU2020, we invited teams from academic and industrial communities to work on the task of information extraction over chemical reactions from chemical patents. The challenge consisted of two sub-tasks. The first sub-task, Named Entity Recognition (NER), focuses on identifying chemical compounds as well as their types in context, i.e., to assign the label of a chemical compound according to the role which the compound plays within a chemical reaction. The second task, Event extraction (EE), involves identification of chemical reactions, represented in terms of relations between compounds and reaction steps. Teams were invited to build models for the NER and EE sub-tasks, and also to build end-to-end systems that address both sub-tasks simultaneously, reflecting a complete system that supports extraction of reaction events from chemical patents.

The shared task was held from 10 April 2020 to 4 June 2020. It attracted 36 registrants from 13 countries including Portugal, Switzerland, Germany, India, Japan, United States, China, and United Kingdom. We received 26 submissions from 11 teams for the NER sub-task, 10 submissions from 5 teams for the EE sub-task, and 10 submissions from 4 teams for end-to-end systems, respectively. Submissions from a team from the company Melax Technologies (from Houston, TX, USA) [see below] ranked first in all 3 sub-tasks.

http://chemu.eng.unimelb.edu.au/

[1] Nguyen, D.Q., Zhai, Z., Yoshikawa, H., Fang, B., Druckenbrodt, C., Thorne, C., Hoessel, R., Akhondi, S.A., Cohn, T., Baldwin, T., Verspoor, K., 2020. ChEMU: Named Entity Recognition and Event Extraction of Chemical Reactions from Patents, in: Jose, J.M., Yilmaz, E., Magalhães, J., Castells, P., Ferro, N., Silva, M.J., Martins, F. (Eds.), Advances in Information Retrieval, Lecture Notes in Computer Science. Springer International Publishing, Cham, pp. 572–579. https://doi.org/10.1007/978-3-030-45442-5_74 (Submitted copy)

A winner of 2020 CHEMU Lab contest:

Melax Technologies Inc. is an emerging AI company specializing in machine learning and natural language processing (NLP) in the biomedical domain. The Melax team has over 20 years of experience in developing clinical NLP methods, tools, and applications, top-ranked in a dozen international NLP challenges in the medical domain.

CLEF2020 presentation (Sep. 23, 2020) "Melaxtech: A report for CLEF 2020 CheMU Task of Chemical Reaction Extraction from Patents" (Jungqi Wang, Yuankai Ren, Zhi Zhang and Yaoyun Zhang)

About CLEF 2020 Conference

CLEF 2020 Conference and Labs of the Evaluation Forum Information Access Evaluation meets Multilinguality, Multimodality, and Visualization

CLEF 2020 is the 11th CLEF conference continuing the popular CLEF campaigns which have run since 2000 contributing to the systematic evaluation of information access systems, primarily through experimentation on shared tasks…. CLEF 2020 consists … a set of labs …designed to test different aspects of mono and cross-language Information retrieval systems.

Update 9/23/2020 11:28 A title of Melax Technologies at CLEF 2020 has been added.