Patent Information Users Group, Inc.

Below is presented background and bibliographic information related to selected presentations at PatentSemTech 2021 workshop at SIGIR2021, July, 15, 2021

Update: PatentSemTech 2021 Workshop Proceedings published:

Proceedings of the 2nd Workshop on Patent Text Mining and Semantic Technologies (PatentSemTech) 2021 co-located with the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021), Edited by Ralf Krestel, Hidir Aras, Linda Andersson, Florina Piroi, Allan Hanbury, and Dean Alderucci. Online, July 15th, 2021. http://ceur-ws.org/Vol-2909/

From the program:

10:10 - 11:30 AM ET | 4:10 - 5:30 PM CET
Chemical Reaction Reference Resolution in Patents  [Note 1]
Hiyori Yoshikawa^1,4, Saber A. Akhondi²,  Camilo Thorne⁵, Christian Druckenbrodt⁵, Ralph Hoessel⁵, Zenan Zhai¹, Jiayuan He^1,3, Timothy Baldwin¹ and Karin Verspoor^1,3
1.The University of Melbourne, Melbourne, Australia; 2. Elsevier B.V., Amsterdam, The Netherlands; 3. RMIT University, Melbourne, Australia; 4. Fujitsu Laboratories Ltd., Kawasaki, Japan; 5. Elsevier Information Systems GmbHFrankfurt am Main, Germany

Prior Art Search and Reranking for Generated Patent Text [Note 2]
Jieh-Sheng Lee and Jieh Hsiang, Department of Computer Science and Information Engineering, National Taiwan University Taipei, Taiwan

Modular Development in Patent AI Space: A Case Study [Note 3]
Mahesh Maan (GreyB (India), Team Lead, Prior Art/Data Science), Sam Zellner (PQAI, Product Lead) and Anirudh Sanutra (GreyB, Team Lead, IP Solutions)


11:40 AM  - 12:25 PM ET |  5:40 - 6:25 PM CET
PatentExplorer: Refining Patent Search with Domain-specific Topic Models
Mark Buckley¹, Sophia Althammer² and Arber Qoku^1,3
1. Siemens AG, Munich, Germany; 2. TU Vienna, Vienna, Austria 3. DKFZ German Cancer Research Center

WIPO Pearl – Insights into the Concept Map Search and Linguistic Search  [Note 4]
Geoffrey Westgate (WIPO, PCT Translation Division, Head of Support Section,), Cristina Valentini (WIPO, PCT Translation Division, Support Section, Head of the Terminology Unit)

The next generation AI-based Prior Art Search tools can be sustainable and transparent.   [Note 5]
Linda Andersson^1,2, Peter Pollak², Tobias Fink^1,2, Florina Piroi^1,2
1. Institute of Software Technology and Interactive Systems, TU Vienna, Vienna, AT; 2. Artificial Researcher IT GmbH, Vienna, AT

2:15 - 3:25 PM ET |  8:15 - 9:25 PM CET
Linguistically Informed Masking for Representation Learning in the Patent Domain [Note 6]
Sophia Althammer¹, Mark Buckley², Sebastian Hofstätter¹ and Allan Hanbury¹
1. TU Vienna, Vienna, Austria 2. Siemens AG, Munich, Germany.

PatentMatch: A Dataset for Matching Patent Claims & Prior Art  [Note 7]
Julian Risch¹, Nicolas Alder¹, Christoph Hewel² and Ralf Krestel¹
1. Hasso Plattner Institute, University of Potsdam, Germany; 2. BETTEN & RESCH Patent- und Rechtsanwälte PartGmbB,Munich, Germany

A Multimodal Approach for Semantic Patent Image Retrieval [Note 8]
Kader Pustu-Iren, M.Sc. Gerrit Bruns and Ralph Ewerth ,Prof, Dr.(TIB- Leibniz Information Centre for Science & Technology, Hannover, Germany)

3:35 - 4:35 PM ET |  9:35 - 10:35 PM CET
Panel discussion: "Artificial Intelligence and Patent Analysis: Friends or Foes?"

AI in and for Patent Analytics: A hype or an efficient support tool for patent analysts?
Irene Kitsara (World Intellectual Property Organization )[Note 9]

IPLodB: Using linked open data in the innovation field - opportunities unveiled and problems encountered  [Note 10]
Dolores Modic (IPLodB project, Nord University)

Challenges for patent practitioners to apply AI in their workflows  [Note 11]
Christoph Hewel (Patent Lawyer at BETTEN & RESCH)

Artificial Intelligence Opportunities in the Patent Grant Process: An IP office perspective
Alexander Klenner-Bajaja (Head of Data Science, European Patent Office)  [Note 12]


Notes

Note 1.
Chemical Reaction Reference Resolution in Patents (Hiyori Yoshikawa at el., see above)
Related recent publications:
Yoshikawa, H., Akhondi, S., Thorne, C., Druckenbrodt, C., Hoessel, R., Zhai, Z., He, J., Baldwin, T., & Verspoor, K. (2021). Chemical Reaction Reference Resolution in Patents. Proceedings of the 2nd Workshop on Patent Text Mining and Semantic Technologies (PatentSemTech) 2021 Co-Located with the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021), July 15th, 2021
He, J., Fang, B., Yoshikawa, H., Li, Y., Akhondi, S.A., Druckenbrodt, C., Thorne, C., Afzal, Z., Zhai, Z., Cavedon, L., Cohn, T., Baldwin, T., Verspoor, K., 2021. ChEMU 2021: Reaction Reference Resolution and Anaphora Resolution in Chemical Patents, in: Hiemstra, D., Moens, M.-F., Mothe, J., Perego, R., Potthast, M., Sebastiani, F. (Eds.), Advances in  Information Retrieval, Lecture Notes in Computer Science. Springer International Publishing, Cham, pp. 608–615. https://doi.org/10.1007/978-3-030-72240-1_71
Fang, B., Druckenbrodt, C., Akhondi, S.A., He, J., Baldwin, T., Verspoor, K., 2021. ChEMU-Ref: A Corpus for Modeling Anaphora Resolution in the Chemical Domain, in: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. Presented at the EACL 2021, Association for Computational Linguistics, Online, pp. 1362–1375. https://aclanthology.org/2021.eacl-main.116 (open access) [... we introduce a novel annotation scheme, based on which we create the ChEMU-Ref dataset from reaction description snippets in English-language chemical patents. We propose a neural approach to anaphora resolution]
ChEMU - Cheminformatics Elsevier Melbourne University lab. [ChEMU lab series provides a unique opportunity for development of information extraction tools over chemical patents.]
Fang, B., Druckenbrodt, C., Yeow Hui Shiuan, C., Novakovic, S., Hössel, R., Akhondi, S.A., He, J., Mistica, M., Baldwin, T., Verspoor, K., 2021. ChEMU-Ref dataset for Modeling Anaphora Resolution in the Chemical Domain. Publisher: Mendeley. https://doi.org/10.17632/r28xxr6p92.1. Includes:
- - - 2021. Annotation Guideline for ChEMU-Ref:  A Corpus for Modeling Anaphora Resolution in the Chemical Domain. Project ChEMU. 28 p. [The purpose of this annotation task is to model referring relationship types within various expressions in chemical patents and “link” all the specific expressions that contain referring relationship.]
He, J., Druckenbrodt, C.. Guidelines for Annotating Chemical Reaction References in Chemical Patents [CheMU2021] v.1.05 Feb. 2021, 21 p. [This document is the guideline for the Full-Text Patent Annotation (FTPA) project and should advise how to annotate manually chemical reactions and their general conditions in patents correctly]
Note: ChEMU 2021 Lab evaluation would take place at CLEF 2021 Conference and Labs of the Evaluation Forum (online event broadcast from the University "Politehnica" of Bucharest, Romania, 21-24 September 2021)
[Back to Program]

Note 2.
Prior Art Search and Reranking for Generated Patent Text
Jieh-Sheng Lee and Jieh Hsiang, Department of Computer Science and Information Engineering, National Taiwan University Taipei, Taiwan
Jieh-Sheng Lee is an in-house patent counsel at Novatek Microelectronics Corp and a Ph.D. candidate focusing on Deep Learning for patents (as of April 2020), See “Augmented Inventing & Deep Learning for Patents” Project at ResearchGate.com

Sel. Refs.
Lee, J.-S., Hsiang, J., 2020. Prior Art Search and Reranking for Generated Patent Text. arXiv:2009.09132 [cs]. Submitted 19 Sep 2020, 7 p. arXiv:2009.09132v1
The objective of our prior art search is to identify retrospectively the most similar patent text spans in the training data of the GPT-2 model. Although our experiments show the effectiveness of reranking in the patent domain, they also show that semantic search for longer text remains very challenging…. The topics for future researchers include: How to make reranking more effective? How to measure the “novelty” and “non-obviousness” between the generated patent text and prior patent text?
[Back to Program]

Note 3.
Modular Development in Patent AI Space: A Case Study  (Mahesh Maan, at al. see above)

PQAI.  Patent Quality Artificial Intelligence. A collaborative initiative to build a common AI-based prior-art search tool. (Sam Zellner, AT&T (Retired), PQAI) Chair). A description of the project at the Georgia Intellectual Property Alliance (GIPA) website. PQAI is an initiative inspired by AT&T to improve patent quality, drive innovation within AT&T and more broadly, among all inventors… Address the fundamental problem that no universal platform is available for the public to easily test and understand the novelty of their inventions. Solution: Create an Open Source patent search engine capable of performing US 102 (obviousness) and 103 (anticipation) novelty searches to better predict the likelihood of obtaining a patent. Scope.This initiative is focused on improving the patent process by enabling better prior art search and analysis.
Project PQAI (https://projectpq.ai). Timeline: Conceived - 2018…. Coverage expanded to all technologies covered by US patents (except Chemical and Biotech)  - Feb. 2020;  Released a new version of 102 (anticipatory)
 prior art engine - June 2020;  Combinational Search (103 engine) Beta version - Nov. 2020
PQAI Search Interface (https://search.projectpq.ai/)
The data coverage of this beta version is limited to ~ 11 mln US patents and applications (USPTO Bulk Datasets) and nearly 11.5 million research papers in the fields of engineering and computer science (emantic Scholar's Open Research Corpus)
PQAI API Usage Guide Queries: 1. Retrieve prior-art documents with text query (/search/102/); 2. Retrieve prior-art combinations with text query (/search/103/); 3. Retrieve prior-art for a patent (documents published before the filing date (/prior-art/patent/), etc.
PQAI profile at ML4Patents (include slides gallery)
Claessen, Rolf, 2021 Open Source AI-based Patent Search Tool PQAI - Interview with Sam Zellner May, 6, 2021 (Youtube, 23:31)
Claessen, Rolf and Suzan, Ken. AI Based Prior Art Search – Open Source Project PQAI – Interview with Sam Zellner. Podcast IP Fridays. Episode 118 (May 1, 2021, 36 m)
Exclusive Interview with Sam Zellner | Inventor Spotlight
[Back to Program]

Note 4.
WIPO Pearl – Insights into the Concept Map Search and Linguistic Search
Geoffrey Westgate and  Cristina Valentini (WIPO)
Presentations:
Westgate, G., Valentini, C., 2020. How terminology work is organized at WIPO, and how it interfaces with translation. Workshop at the Terminology Summer School  TSS2020, July 1st - 4^th, 2020. Introduction  (24 p.); Workshop  (24 p.).
Valentini, C., 2018. Terminology across borders: the example of WIPO Pearl. Presentation at the 2018 EAFT Terminology Summit, San Sebastián, Spain, 22-23 November 2018. (video, 33:46,  presentation, 27 p.) (EAFT -European Association for Terminology)
WIPO. PCT Terminology and WIPO Pearl, February 2015.14 p. [p.13 Who will use WIPO Pearl?... Patent...searchers]
WIPO Pearl in Amman, S., Overview of the PATENTSCOPE search system, Sep. 2019, p.25-28
WIPO Pearl @WIPO website:
WIPO Pearl [WIPO’s Multilingual Terminology Portal, integrated with Patenscope] (video (3:15), Linguistic Search,  Concept Map Search, User Guide (Glossary))
WIPO Pearl Database enhancement (March 29, 2021) [The WIPO Pearl database has been updated and now contains over 205,000 terms, all validated by WIPO-PCT language experts. In addition, over 21,000 concepts (or 82%) are linked to other concepts in the database, and these relationships can be browsed in our Concept Map Search. Further relationships can be shown in unvalidated “concept clouds” produced via a machine learning algorithm.]
WIPO’s Terminology Database Boosts Performance Across Ten Languages, In Focus (WIPO), August 20, 2019 […In addition to providing human-built and validated Concept Maps, WIPO Pearl leverages AI to generate so-called Concept Clouds.]
WIPO Pearl News Archive (Dec. 16, 2015 (Search for images, etc.), Dec. 8, 2016 (“Concept Path Search”), July 12, 2017 (“Concept clouds”)) Academic publications:
Valentini, C., Westgate, G., Rouquet, P., 2016. The PCT Termbase of the World Intellectual Property Organization: Designing a database for multilingual patent terminology. Terminology. International Journal of Theoretical and Applied Issues in Specialized Communication 22 (2), 171–200. https://doi.org/10.1075/term.22.2.02val (free copy n/a)
Caffrey, C., Valentini, C., 2019. Applications of technology in the Patent Cooperation Treaty (PCT) Translation Division of the World Intellectual Property Organization (WIPO), in: The Routledge Handbook of Translation and Technology. Routledge. p. 127-147 https://doi.org/10.4324/9781315311258-8 (GoogleBooks)
Támás, D.M., Presentation of the WIPO Pearl patent terminology database. Fordítástudomány (Translation Studies) 23(1), 49–62.(May 2021) https://doi.org/10.35924/fordtud.23.1.3 (in Hungarian, could be translated in English by uploading pdf file to GoogleTranslate)
[Back to Program]

Note 5
The next generation AI-based Prior Art Search tools can be sustainable and transparent. Linda Andersson, et al
Linda Anderson is PhD student in TU Wien, PhD Thesis The Essence of Patent Text Mining to be completed in 2022. Linda also CTO and co-CEO  of Artificial Researcher IT GmbH
From the PhD thesis abstract: “In order to demonstrate the importance to recognize different information needs and linguistic diversity within the patent domain, we develop several real-world text mining applications, from information extraction in terms of domain specific terminology, identification and ontology population, to specific question answering systems. During a series of IR experiments, we developed a complete information retrieval system for patent passage retrieval, which incorporate domain knowledge and linguistic diversity within the patent text genre, and meets the specific requirements of patentability and invalidity search.”
[Back to Program]

Note 6.
Linguistically Informed Masking for Representation Learning in the Patent Domain. Sophia Althammer et. al.
Refs.
Althammer, S., Buckley, M., Hofstätter, S., Hanbury, A., 2021. Linguistically Informed Masking for Representation Learning in the Patent Domain. arXiv:2106.05768 [cs]. 11p. Submitted 10 Jun 2021 Presentation at SIGIR 2021 PatentSemTech workshop
…In this paper we propose the empirically motivated Linguistically Informed Masking (LIM) method to focus domain-adaptative pre-training on the linguistic patterns of patent…. We quantify the relevant differences between patent, scientific and general-purpose language and demonstrate for two different language models (BERT and SciBERT) that domain adaptation with LIM leads to systematically improved representations by evaluating the performance of the domain-adapted representations of patent language on two independent downstream tasks, the IPC classification and similarity matching. We demonstrate the impact of balancing the learning from different information sources during domain adaptation for the patent domain. We make the source code as well as the domain-adaptive pre-trained patent language models publicly available at this https URL [@GitHub, Copyright (c) Siemens AG, 2020]

[Back to Program]

Note 7
PatentMatch: A Dataset for Matching Patent Claims & Prior Art (Julian Risch at al, see above)
Refs.
Risch, J., Alder, N., Hewel, C., Krestel, R.: PatentMatch: A Dataset for Matching Patent Claims & Prior Art. Proceedings of the 2nd Workshop on Patent Text Mining and Semantic Technologies (PatentSemTech@SIGIR) (2021). [Download]
… we address the computer-assisted search for prior art by creating a training dataset for supervised machine learning called PatentMatch. It contains pairs of claims from patent applications and semantically corresponding text passages of different degrees from cited patent documents. Each pair has been labeled by technically-skilled patent examiners from the European Patent Office. Accordingly, the label indicates the degree of semantic correspondence (matching), i.e., whether the text passage is prejudicial to the novelty of the claimed invention or not. Preliminary experiments using a baseline system show that PatentMatch can indeed be used for training a binary text pair classifier on this challenging information retrieval task. The dataset is available online: https://hpi.de/naumann/s/patentmatch.
Risch, J., Alder, N., Hewel, C., Krestel, R.: PatentMatch: A Dataset for Matching Patent Claims with Prior Art. ArXiv e-prints 2012.13919. (2020). [Abstract] [ Download]
Project: PAAR: Patent Analysis and Retrieval (Patent Classification and Embeddings, PatentMatch Dataset)
https://hpi.de/naumann/projects/web-science/paar-patent-analysis-and-retrieval.html
Krestel, R., Chikkamath, R., Hewel, C., Risch, J., 2021. A survey on deep learning for patent analysis. World Patent Information 65, 102035. https://doi.org/10.1016/j.wpi.2021.102035
[Back to Program]

Note 8
A Multimodal Approach for Semantic Patent Image Retrieval (Kader Pustu-Iren et al.)
Project: ExpResViP - Exploitation of research results through visual patent retrieval (July 2020 – August 2023) (Team: Gerrit Bruns, Kader Pustu-Iren, Visual Analytics Group)
The aim of the project is to develop a novel visual search for patent retrieval based on the automatic recognition of image similarities and text-image references in patent specifications. The innovative, image-based search is based on machine learning methods and aims to increase the findability and visibility of patents and to overcome language and terminology barriers. … to meet the demand for innovations in the field of patent exploitation in consultation with the Leibniz Gemeinschaft [The Leibniz Association]. This requires the classification of different image types, recognition of semantic concepts, development of measures for similarity estimation, user-oriented development and design of the tool's user interface.
TIB responsible for: …Basic and advanced algorithms and methods of image analysis
Cooperation: Fraunhofer IAIS, Leibniz Headquarters, University Hildesheim (IWIST)
https://www.tib.eu/en/research-development/project-overview/project-summary/expresvip
[Back to Program]

Note 9.
AI in and for Patent Analytics: A hype or an efficient support tool for patent analysts?
Irene Kitsara (WIPO)
From the PatentSemTech 2021 Workshop presentation abstract:
The future of patent analytics is expected to include AI, even if the exact form and extent are not yet clear. In this talk we will share some thoughts and observations about the status of AI tools for patent analytics, related benefits and challenges. We will use as basis for these thoughts a. WIPO`s exploratory work (2016 and ongoing work) on the use of open source tools and machine learning for patent analytics tasks in the framework of preparation of related methodological resources; and b. USPTO`s report (2020) comparing the performance of a patent professional team using traditional search and analysis approaches for the WIPO Technology Trends report on AI (2019) with the results of an AI model to retrieve and group AI-related patent documents, using WIPO`s patent dataset as benchmark.
Cited Refs:
Oldham, P., Kitsara, I., 2016. The WIPO Manual on Open Source Patent Analytics.
Oldham, P., 2019. The WIPO Patent Analytics Handbook. Work-in-progress.
USPTO, 2020. FY 2020 Performance and Accountability Report: Expanding American Innovation. [The USPTO successfully launched a new patent search system prototype that uses sophisticated AI capabilities to assist patent examiners with performing a complete patent search. … These capabilities allow patent examiners to automatically retrieve prior art documents, review those documents more efficiently than with traditional search tools, leverage suggested search areas to ensure complete search, and use “explainable AI” to help patent examiners understand results. (“Develop Artificial Intelligence Programs for Patents at the USPTO, p.64-65”)]
WIPO, WIPO Technology Trends 2019 – Artificial Intelligence. 2019. 158 p.
Gojon, S., Migeon, A., Petit, P., Lopez, P., Kitsara, I., Data collection method and clustering scheme. Background paper .(WIPO Technology Trends 2019: Artificial Intelligence.). 2018, 25 p.
[Back to Program]

Note 10
IPLodB: Using linked open data in the innovation field - opportunities unveiled and problems encountered. Dolores Modic (IPLodB project, Nord University)
The short talk addresses the linked open data (LOD) approach for enabling access to (linked) patent information. …The two datasets [EPO and the Springer Nature] represent the core on which we started building a new patent-centric LOD sub-cloud [in the IPLodB project]. Hence, we will look at AI and patent analysis from a linked open data perspective and try to discuss its technological impact for future developments.
Ref.
Modic, Dolores Using EPO linked open data: opportunities unveiled by the IPLOD project, Presentation at 20210 EPO Patent Information Conference. Nov. 4, 2020 (Presentation, 16 p.)
See also an extensive additional information on LOD and  IPLodB project to the above referenced presentation in the Oct. 25, 2020 PIUG-PF post

[Back to Program]

Note 11.
Challenges for patent practitioners to apply AI in their workflows. Christoph Hewel (Patent Lawyer at BETTEN & RESCH)
From Christoph profile: Christoph is specialising in the field of machine learning (i.e. artificial intelligence), .... In this context he has further initiated a research project with the Hasso-Plattner-Intitute in the field of NLP (natural language processing) [see above presentation on PatentMatch]. In particular, he is investigating together with researchers of the institute how deep learning (i.e. neural networks) can be leveraged to process patent texts, e.g. for prior art search.
Krestel, R., Chikkamath, R., Hewel, C., Risch, J., 2021. A survey on deep learning for patent analysis. World Patent Information 65, 102035. https://doi.org/10.1016/j.wpi.2021.102035
Chikkamath, R., Endres, M., Bayyapu, L., Hewel, C., 2020. An Empirical Study on Patent Novelty Detection: A Novel Approach Using Machine Learning and Natural Language Processing, in: 2020 Seventh International Conference on Social Networks Analysis, Management and Security (SNAMS). Presented at the 2020 Seventh International Conference on Social Networks Analysis, Management and Security (SNAMS), December 14-16, 2020.pp. 1–7. https://doi.org/10.1109/SNAMS52053.2020.9336557 (See Fig. 1. Manual claim segmentation (credits: Mr Christoph Hewel))
[Back to Program]

Note 12.
Artificial Intelligence Opportunities in the Patent Grant Process: An IP office perspective
Alexander Klenner-Bajaja (Head of Data Science, European Patent Office)
Presentations:
Holcome, Jamie (USPTO), Klenner-Bajaja, Alexander (EPO), Sollie, Jens Petter (Norweigan IPO), Scott, Peter (WIPR). How IP Offices are utilising AI: Image Searches, Machine Translation, and Administration. WIPR Webinar, Jan 12 2021, (video, 59 min).
Klenner-Bajaja, A., 2020. AI already speaks the language of patents. Presentation on EPO Conference “The role of patents in an AI driven world”, 17-18 December 2020 (Day 1) (video, 23 min, 2:02:30-2:25:34)
Klenner-Bajaja, A., EPO, 2020. Transformer models speaking the language of patents: Distribution, Classification and Search of patent applications with artificial intelligence at the EPO using state of the art language model architectures. Presented at AI & Patent Data Workshop @ JURIX 2020, Dec. 9, 2020 (abstract, video, 15 m)
The future directions of search and searchers in an artificial intelligence era (Panellists: Francesco Zaccà, Principal Director Operations, Information and Communication Technology, EPO; Alexander Klenner-Bajaja, Data Scientist, Knowledge and Search Services, EPO;     Geert Boedt, Business Analyst, Patent Information, EPO;    Samuel Davis, CEO/Founder, amplified.ai). Panel discussion at EPO conference Search Matters 2019, May 7, 2019 (video, 59 min)
Klenner-Bajaja, A., 2017. Towards Semantic Search at the European Patent Office. Presentation at II-SDV 2017 Conference, Nice, 24 - 25 April 2017, , 40 p.
Klenner-Bajaja, A., Exploring Automated Patent Search with KNIME Possibilities, Limits, Future. Presentation at the KNIME Spring Summit 2016,  Berlin February 22 - 26, 2016 (slides, 29 p.)
Klenner-Bajaja, A., 2017. Future Challenges in (automated) Patent Search. Presentations at II-SDV Conference, 20 - 21 April, 2015, Nice, France, 31 p.
[Back to Program]