AI & Scientific Discovery Workshop

Just as coding assistants have dramatically increased productivity for coding tasks over the last two years, researchers in the NLP community have begun to explore methods and opportunities ahead for creating scientific assistants that can help with the process of scientific discovery and increase the pace at which novel discoveries are made.

Historically, major results in AI and scientific discovery have been restricted to problem-specific methods—such as DeepMind AlphaFold and RosettaFold, which are systems specifically designed for protein folding, or multistage discovery pipelines designed for use in identification of novel materials. Over the last year, language models are being used to create problem-general scientific discovery assistants that are not restricted to narrow problem domains or formulations. Such applications hold opportunity for assisting researchers in broad domains, or scientific reasoning more generally. Beyond providing assistance, a growing body of work has begun to focus on the prospect of creating largely autonomous scientific discovery agents that can make novel discoveries with minimal human intervention.

We have also observed rising interest in evaluating systems that help with or perform scientific discovery. Difficulties with evaluation persist; as novel scientific discoveries (by definition) haven't been made yet, it is hard to formulate and evaluate a benchmark that provides insights about valuable machine-generated contributions that boost the discovery process. The community has begun to answer this with benchmarks focused on facets of the discovery process, such as data-driven discovery, experiment replication, reviewing, idea generation, or proxy tasks such as end-to-end discovery in virtual environments, all of which begin to address the challenges of evaluating progress towards generating novel discoveries.

These recent developments highlight the possibility of rapidly accelerating the pace of scientific discovery in the near term. Given the influx of researchers into this expanding field, this workshop proposes to serve as a vehicle for bringing together a diverse set of perspectives from this quickly expanding subfield, helping to disseminate the latest results, standardize evaluation, foster collaboration between groups, and allow discussing aspirational goals for 2025 and beyond.

Speakers

Call for Papers

We welcome submissions on all topics related to AI and Scientific Discovery including but not limited to:

• Literature-based Discovery
• Agent-centered Approaches
• Automated Experiment Execution
• Automated Replication
• Data-driven Discovery
• Discovery in Virtual Environments
• Discovery with Humans in the Loop
• Assistants for Scientific Writing

Organizers

Submission Guidelines

We welcome three types of papers: archival workshop papers, non-archival papers, and non-archival cross-submissions. Only regular workshop papers will be included in the workshop proceedings. Regular workshop submissions (both archival and non-archival) should be in PDF format and made through the OpenReview website set up for this workshop link. In line with the ACL main conference policy, camera-ready versions of regular workshop papers will be given one additional page of content. Non-archival cross-submissions should be made through the form [link].

• Archival regular workshop papers: Authors should submit a paper up to 8 pages (both short and long papers are welcome), with unlimited pages for references, following the ACL author guidelines. The reported research should be substantially original. All submissions will be reviewed in a single track, regardless of length. Accepted papers will be presented as posters by default, and best papers may be given the opportunity for a brief talk to introduce their work. Reviewing will be double-blind, and thus no author information should be included in the papers; self-reference that identifies the authors should be avoided or anonymised. Accepted papers will appear in the workshop proceedings. Preference for oral presentation slots in the workshop will be given to archival papers.
• Non-archival regular workshop papers: This is the same as the option above, but these papers will not appear in the proceedings and will typically only receive poster presentation slots. Non-archival submissions in this category will still undergo the review process. This is appropriate for nearly finished work that is intended for submission to another venue at a later date.
• Non-archival cross-submissions: We also solicit cross-submissions, i.e., papers on relevant topics that have already appeared in other venues (e.g., workshop or conference papers at NLP, ML, or cognitive science venues, among others). Accepted papers will be presented at the workshop, with an indication of original venue, but will not be included in the workshop proceedings. Cross-submissions are ideal for related work which would benefit from exposure to the audience working on Scientific Discovery. Papers in this category do not need to follow the ACL format, and the submission length is determined by the original venue. The paper selection will be solely determined by the organizing committee in a non-blind fashion. These papers will typically receive poster presentation slots.

In addition, we welcome papers on relevant topics that are under review or to be submitted to other venues (including the ACL 2024 main conference). These papers must follow the regular workshop paper format and will not be included in the workshop proceedings. Papers in this category will be reviewed by workshop reviewers.

Note to authors: For archival and non-archival regular workshop submissions, while you submit your paper through OpenReview (link), please select the "Track" properly based on the guidelines. For cross-submissions, please fill out this form ([link]) and do NOT submit through OpenReview.

For questions about the submission guidelines, please contact workshop organizers via aisd-organizers@googlegroups.com.

• Variable Extraction for Model Recovery in Scientific Literature
  Chunwei Liu, Enrique Noriega-Atala, Adarsh Pyarelal, Clayton T Morrison, Mike Cafarella
  [Archival]
  
• How Well Do Large Language Models Extract Keywords? A Systematic Evaluation on Scientific Corpora
  Nacef Ben Mansour, Hamed Rahimi, Motasem Alrahabi
  [Archival]
  
• A Human-LLM Note-Taking System with Case-Based Reasoning as Framework for Scientific Discovery
  Douglas B Craig
  [Archival]
  
• Towards AI-assisted Academic Writing
  Daniel J. Liebling, Malcolm Kane, Madeleine Grunde-McLaughlin, Ian Lang, Subhashini Venugopalan, Michael Brenner
  [Archival]
  
• Evaluating and Enhancing Large Language Models for Novelty Assessment in Scholarly Publications
  Ethan Lin, Zhiyuan Peng, Yi Fang
  [Archival]
  
• LLM-Assisted Translation of Legacy FORTRAN Codes to C++: A Cross-Platform Study
  Nishath Rajiv Ranasinghe, Shawn M. Jones, Michal Kucer, Ayan Biswas, Daniel O'Malley, Alexander Most, Selma Liliane Wanna, Ajay Sreekumar
  [Archival]
  
• FlavorDiffusion: Predicting Food Pairings and Chemical Interactions Using Diffusion Models
  Junpyo Seo
  [Archival]
  

• ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery
  Ziru Chen, Shijie Chen, Yuting Ning, Qianheng Zhang, Boshi Wang, Botao Yu, Yifei Li, Zeyi Liao, Chen Wei, Zitong Lu, Vishal Dey, Mingyi Xue, Frazier N. Baker, Benjamin Burns, Daniel Adu-Ampratwum, Xuhui Huang, Xia Ning, Song Gao, Yu Su, Huan Sun
  [Non-Archival Published]
  
• Large Language Models Reflect Human Citation Patterns with a Heightened Citation Bias (NAACL 2025)
  Andres Algaba, Carmen Mazijn, Vincent Holst, Floriano Tori, Sylvia Wenmackers, Vincent Ginis
  [Non-Archival Published]
  
• MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses (ICLR 2025)
  Zonglin Yang, Wanhao Liu, Ben Gao, Tong Xie, Yuqiang Li, Wanli Ouyang, Soujanya Poria, Erik Cambria, Dongzhan Zhou
  [Non-Archival Published]
  
• Efficient Evolutionary Search Over Chemical Space with Large Language Models (ICLR 2025)
  Haorui Wang*, Marta Skreta*, Cher Tian Ser, Wenhao Gao, Lingkai Kong, Felix Strieth-Kalthoff, Chenru Duan, Yuchen Zhuang, Yue Yu, Yanqiao Zhu, Yuanqi Du, Alan Aspuru-Guzik, Kirill Neklyudov, Chao Zhang
  [Non-Archival Prepublished]
  
• DiscoveryBench: Towards Data-Driven Discovery with Large Language Models (ICLR 2025)
  Bodhisattwa Prasad Majumder, Harshit Surana, Dhruv Agarwal, Bhavana Dalvi Mishra, Abhijeetsingh Meena, Aryan Prakhar, Tirth Vora, Tushar Khot, Ashish Sabharwal, Peter Clark
  [Non-Archival Published]
  
• DiscoveryWorld: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents (NeurIPS 2024 Spotlight)
  Peter Jansen, Marc-Alexandre Côté, Tushar Khot, Erin Bransom, Bhavana Dalvi Mishra, Bodhisattwa Prasad Majumder, Oyvind Tafjord, Peter Clark
  [Non-Archival Published]
  
• Hypothesis Generation with Large Language Models (EMNLP 2024)
  Yangqiaoyu Zhou, Haokun Liu, Tejes Srivastava, Hongyuan Mei, Chenhao Tan
  [Non-Archival Published]
  

• VISION: A Modular AI Assistant for Natural Human-Instrument Interaction at Scientific User Facilities
  [Non-Archival]
  
• Automatic Evaluation Metrics for Artificially Generated Scientific Research
  [Non-Archival]
  
• WithdrarXiv: A Large-Scale Dataset for Retraction Study
  [Non-Archival]
  
• FARM: Functional Group-Aware Representations for Small Molecules
  [Non-Archival]
  
• Scideator: Iterative Human-LLM Scientific Idea Generation and Novelty Evaluation Grounded in Research-Paper Facet Recombination
  [Non-Archival]
  
• What Can Large Language Models Do for Sustainable Food?
  [Non-Archival]
  
• Learning to Generate Research Idea with Dynamic Control
  [Non-Archival]
  
• Map2Text: New Content Generation from Low-Dimensional Visualizations
  [Non-Archival]
  
• Data Driven Design as a Challenge Task for Few- and Zero-Shot Information Extraction
  [Non-Archival]
  
• Language Modeling by Language Models with Genesys
  [Non-Archival]