Research Data Leeds Repository

SpatialLM-StepGame

Citation

Li, Fangjun and Hogg, David C. and Cohn, Anthony G. (2024) SpatialLM-StepGame. University of Leeds. [Dataset] https://doi.org/10.5518/1468¹

Dataset description

This dataset, associated with the AAAI-24 paper "Advancing Spatial Reasoning in Large Language Models: An In-depth Evaluation and Enhancement Using the StepGame Benchmark", aims to enhance spatial reasoning evaluations in language models. It rectifies template errors in the StepGame benchmark, providing a refined version in the bAbI format, promoting more accurate evaluation of language models' capabilities in spatial reasoning tasks.

Additional information:	This dataset has been developed from StepGame (https://github.com/ZhengxiangShi/StepGame), providing a refined version
Keywords:	spatial reasoning, language model, multi-hop, StepGame
Subjects:	I000 - Computer sciences > I400 - Artificial intelligence2
Divisions:	Faculty of Engineering and Physical Sciences > School of Computing3
Related resources:	Location Type https://doi.org/10.1609/aaai.v38i17.298114 Publication https://eprints.whiterose.ac.uk/211546/5 Publication
License:	MIT License
Date deposited:	04 Sep 2024 10:14
URI:	https://archive.researchdata.leeds.ac.uk/id/eprint/13216
Additional details Additional details Creators or authors: Creators ORCID Other ID Email Primary affiliation Primary affiliation ID Primary affiliation ID type Secondary affiliation Secondary affiliation ID Secondary affiliation ID type Li, Fangjun https://orcid.org/0000-0002-1109-62857 University of Leeds https://ror.org/024mrxd33 ror Hogg, David C. https://orcid.org/0000-0002-6125-95648 University of Leeds https://ror.org/024mrxd33 ror Cohn, Anthony G. https://orcid.org/0000-0002-7652-89079 University of Leeds https://ror.org/024mrxd33 ror Type of data: Dataset Publication date: 2024 Resource language: English Metadata language: English Publisher: University of Leeds Contact email: a.g.cohn@leeds.ac.uk10 Last Modified: 04 Sep 2024 11:40

Files

Documentation

README_Fangjun-etal_2024.txt [3kB] 11 README_Fangjun-etal_2024.txt [3kB] 12 README_Fangjun-etal_2024.txt [3kB] 13 Security: Anyone File name: README_Fangjun-etal_2024.txt14 Content type: Documentation Mime-Type: text/plain License: MIT License File size: 3kB

Data

data.zip [25MB] 15 data.zip [25MB] 16 data.zip [25MB] 17 Security: Anyone File name: data.zip18 Content type: Data Description: Data for the AAAI-24 paper, refining the StepGame dataset for accurate model evaluations Mime-Type: application/zip License: MIT License File size: 25MB

Program

correct.py [18kB] 19 correct.py [18kB] 20 correct.py [18kB] 21 Security: Anyone File name: correct.py22 Content type: Program License: MIT License File size: 18kB

• 1. https://doi.org/10.5518/1468
• 2. https://archive.researchdata.leeds.ac.uk/view/subjects/I400.html
• 3. https://archive.researchdata.leeds.ac.uk/view/divisions/SC/
• 4. https://doi.org/10.1609/aaai.v38i17.29811
• 5. https://eprints.whiterose.ac.uk/211546/
• 6. https://archive.researchdata.leeds.ac.uk/id/eprint/1321
• 7. https://orcid.org/0000-0002-1109-6285
• 8. https://orcid.org/0000-0002-6125-9564
• 9. https://orcid.org/0000-0002-7652-8907
• 10. mailto:a.g.cohn@leeds.ac.uk
• 11. https://archive.researchdata.leeds.ac.uk/1321/1/README_Fangjun-etal_2024.txt
• 12. https://archive.researchdata.leeds.ac.uk/1321/1/README_Fangjun-etal_2024.txt
• 13. https://archive.researchdata.leeds.ac.uk/1321/1/README_Fangjun-etal_2024.txt
• 14. https://archive.researchdata.leeds.ac.uk/1321/1/README_Fangjun-etal_2024.txt
• 15. https://archive.researchdata.leeds.ac.uk/1321/3/data.zip
• 16. https://archive.researchdata.leeds.ac.uk/1321/3/data.zip
• 17. https://archive.researchdata.leeds.ac.uk/1321/3/data.zip
• 18. https://archive.researchdata.leeds.ac.uk/1321/3/data.zip
• 19. https://archive.researchdata.leeds.ac.uk/1321/2/correct.py
• 20. https://archive.researchdata.leeds.ac.uk/1321/2/correct.py
• 21. https://archive.researchdata.leeds.ac.uk/1321/2/correct.py
• 22. https://archive.researchdata.leeds.ac.uk/1321/2/correct.py