| First Authors |
Charvi Jain
|
| Authors |
Charvi Jain, Sahar Vahdati, Nandu Gopan, Ivo F. Sbalzarini, Jens Lehmann
|
| Corresponding Authors |
Jens Lehmann
|
| Last Authors |
Jens Lehmann
|
| Conference Proceedings Volume Title |
Proc. EKAW 2024 Workshops, Tutorials, Posters and Demos, 24th International Conference on Knowledge Engineering and Knowledge Management (EKAW 2024) |
| Series Title |
|
| Conference Name |
24th International Conference on Knowledge Engineering and Knowledge Management (EKAW 2024) |
| Conference Location |
Amsterdam, The Netherlands |
| Conference Start Date |
2024-11-26 |
| Conference End Date |
2024-11-28 |
| Chapter Number |
|
| Publisher |
Springer |
| Conference Proceedings Editors |
|
| ISBN |
|
| First Page |
1 |
| Last Page |
6 |
| Open Access |
false |
| Print Publication Date |
2024-01-01 |
| Online Publication Date |
|
| Abstract |
We evaluate the effectiveness of current large language model (LLM) literature review systems in interdisciplinary domains. While LLMs can support and accelerate reviewing the scientific literature, it is unclear how they cope with interdisciplinary science, where sources from multiple fields must be integrated according to relevance defined by context. We study this from the perspective of systems biology, a field that combines biology, mathematics, physics, and computer science. Using a set of expert-defined research questions, we assess the ability of LLMs to meaningfully integrate cross-domain knowledge and correctly reflect relevance. Specifically, we evaluate the quality of generated reports and the relevance of retrieved references from five different review models. We find that LLMs are a valuable augmentative tool for literature reviews, but trade off report quality for completeness in interdisciplinary domains. We address these limitations by proposing a novel method, termed AURORA, which is particularly designed for interdisciplinary applications. On the interdisciplinary systems biology benchmark, AURORA offers good coverage with high-quality reports. |
| PDF |
Jain_2024_8866.pdf
(2.1 MB)
|
| Cover Image |
|
| Affiliated With |
Sbalzarini
|
| Selected By |
|
| Acknowledged Services |
|
| Publication Status |
Published |
| Edoc Link |
|
| Sfx Link |
|
| DOI |
|
| PubMed ID |
|
| WebOfScience Link |
|
| Alternative Full Text URL |
|
| Display Publisher Download Only |
false |
| Visible On MPI-CBG Website |
true |
| PDF Downloadable |
true |
| Created By |
sbalzari |
| Added Date |
2024-12-08 |
| Last Edited By |
sbalzari |
| Last Edited Date |
2024-12-08 20:01:56.421 |
| Library ID |
8866 |
| Document ID |
|
| Entry Complete |
false |
| eDoc Compliant |
true |
| Include in Edoc Report |
false |
| In Pure |
false |
| Ready for eDoc Export |
false |
| Author Affiliations Complete |
false |
| Project Name |
SECAI |
| Project URL |
|
| Grant ID |
|
| Funding Programme |
Zuse Schools of Excellence in AI |
| Funding Organisation |
DAAD |