Research Data Leeds Repository

Citation

Li, Fangjun and Hogg, David C. and Cohn, Anthony G. (2024) SpatialLM-StepGame. University of Leeds. [Dataset] https://doi.org/10.5518/1468

Dataset description

This dataset, associated with the AAAI-24 paper "Advancing Spatial Reasoning in Large Language Models: An In-depth Evaluation and Enhancement Using the StepGame Benchmark", aims to enhance spatial reasoning evaluations in language models. It rectifies template errors in the StepGame benchmark, providing a refined version in the bAbI format, promoting more accurate evaluation of language models' capabilities in spatial reasoning tasks.

Additional information:	This dataset has been developed from StepGame (https://github.com/ZhengxiangShi/StepGame), providing a refined version
Keywords:	spatial reasoning, language model, multi-hop, StepGame
Subjects:	I000 - Computer sciences > I400 - Artificial intelligence
Divisions:	Faculty of Engineering and Physical Sciences > School of Computing
Related resources:	Location Type https://doi.org/10.1609/aaai.v38i17.29811 Publication https://eprints.whiterose.ac.uk/211546/ Publication
License:	MIT License
Date deposited:	04 Sep 2024 10:14
URI:	https://archive.researchdata.leeds.ac.uk/id/eprint/1321
Additional details Additional details Creators or authors: Creators ORCID Other ID Email Primary affiliation Primary affiliation ID Primary affiliation ID type Secondary affiliation Secondary affiliation ID Secondary affiliation ID type Li, Fangjun https://orcid.org/0000-0002-1109-6285 University of Leeds https://ror.org/024mrxd33 ror Hogg, David C. https://orcid.org/0000-0002-6125-9564 University of Leeds https://ror.org/024mrxd33 ror Cohn, Anthony G. https://orcid.org/0000-0002-7652-8907 University of Leeds https://ror.org/024mrxd33 ror Type of data: Dataset Publication date: 2024 Resource language: English Metadata language: English Publisher: University of Leeds Contact email: a.g.cohn@leeds.ac.uk Last Modified: 04 Sep 2024 11:40

Files

Documentation

README_Fangjun-etal_2024.txt [3kB] README_Fangjun-etal_2024.txt [3kB] README_Fangjun-etal_2024.txt [3kB] Security: Anyone File name: README_Fangjun-etal_2024.txt Content type: Documentation Mime-Type: text/plain License: MIT License File size: 3kB

Data

data.zip [25MB] data.zip [25MB] data.zip [25MB] Security: Anyone File name: data.zip Content type: Data Description: Data for the AAAI-24 paper, refining the StepGame dataset for accurate model evaluations Mime-Type: application/zip License: MIT License File size: 25MB

Program

correct.py [18kB] correct.py [18kB] correct.py [18kB] Security: Anyone File name: correct.py Content type: Program License: MIT License File size: 18kB