Lorandi, Michela ORCID: 0000-0002-6131-8763 and Belz, Anya
ORCID: 0000-0002-0552-8096
(2023)
Data-to-text generation for severely under-resourced languages with GPT-3.5: a bit of help needed from Google Translate.
In: 16th International Natural Language Generation Conference - Workshop on Multimodal, Multilingual Natural Language Generation and Multilingual WebNLG Challenge, 11-12 Sep 2023, Prague, Czech Republic.
Popović, Maja ORCID: 0000-0001-8234-8745, Arvan, Mohammad, Parde, Natalie
ORCID: 0000-0003-0072-7499 and Belz, Anya
ORCID: 0000-0002-0552-8096
(2023)
Exploring variation of results from different experimental conditions.
In: ACL 2023, 9 - 14 July 2023, Toronto, Canada.
Lorandi, Michela and Belz, Anya (2023) How to control sentiment in text generation: a survey of the state-of-the-art in sentiment-control techniques. In: 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, 14 July 2023, Toronto, Canada.
Belz, Anya ORCID: 0000-0002-0552-8096, Thomson, Craig, Reiter, Ehud
ORCID: 0000-0002-7548-9504, Abercrombie, Gavin
ORCID: 0000-0002-6546-3562, Alonso-Moral, Jose M.
ORCID: 0000-0003-3673-421X, Arvan, Mohammad, Cheung, Jackie, Cieliebak, Mark
ORCID: 0009-0007-3059-8516, Clark, Elizabeth, van Deemter, Kees, Kelleher, John D.
ORCID: 0000-0001-6462-3248 and Klubička, Filip
ORCID: 0000-0001-9712-6141
(2023)
Missing information, unresponsive authors, experimental flaws: the impossibility of assessing the reproducibility of previous human evaluations in NLP.
In: Fourth Workshop on Insights from Negative Results in NLP, 2 May 2023, Dubrovnik, Croatia.
ISBN 978-1-959429-49-4
Mille, Simon ORCID: 0000-0002-8852-2764, Ricci, Josep, Shvets, Alexander
ORCID: 0000-0002-8370-2109 and Belz, Anya
ORCID: 0000-0002-0552-8096
(2023)
A pipeline for extracting abstract dependency templates for data-to-text natural language generation.
In: Seventh International Conference on Dependency Linguistics (Depling, GURT/SyntaxFest 2023), 9-12 Mar 2023, Washington, D.C..
Popović, Maja ORCID: 0000-0001-8234-8745 and Belz, Anya
ORCID: 0000-0002-0552-8096
(2022)
On reporting scores and agreement for error annotation tasks.
In: 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), 7 Dec 2022, Abu Dhabi, United Arab Emirates & Online.
Huidrom, Rudali ORCID: 0000-0003-0630-3603 and Belz, Anya
ORCID: 0000-0002-0552-8096
(2022)
A survey of recent error annotation schemes for automatically generated text.
In: 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), 7 Dec 2022, Abu Dhabi, UAE and Online.
Belz, Anya ORCID: 0000-0002-0552-8096
(2022)
A metrological perspective on reproducibility in nlp.
Computational Linguistics, 48
(4).
pp. 1125-1135.
ISSN 0891-2017
Savkov, Aleksandar ORCID: 0009-0009-6831-5563, Moramarco, Francesco, Korfiatis, Alex Papadopoulos, Perera, Mark, Belz, Anya
ORCID: 0000-0002-0552-8096 and Reiter, Ehud
ORCID: 0000-0002-7548-9504
(2022)
Consultation checklists: standardising the human evaluation of medical note generation.
In: EMNLP 2022 Industry Track, 9-11 Dec 2022, Abu Dhabi, UAE.
Belz, Anya ORCID: 0000-0002-0552-8096, Shimorina, Anastasia, Popović, Maja
ORCID: 0000-0001-8234-8745 and Reiter, Ehud
ORCID: 0000-0002-7548-9504
(2022)
The 2022 ReproGen shared task on reproducibility of evaluations in NLG: overview and results.
In: 15th International Conference on Natural Language Generation: Generation Challenges, 17-22 July 2022, Waterville, ME, USA.
Popović, Maja ORCID: 0000-0001-8234-8745, Castilho, Sheila
ORCID: 0000-0002-8416-6555, Huidrom, Rudali
ORCID: 0000-0003-0630-3603 and Belz, Anya
ORCID: 0000-0002-0552-8096
(2022)
Reproducing a manual evaluation of the simplicity of text simplification system outputs.
In: 15th International Conference on Natural Language Generation: Generation Challenges, 17-22 July 2022, Waterville, ME, USA.
Huidrom, Rudali ORCID: 0000-0003-0630-3603, Dušek, Ondřej
ORCID: 0000-0002-1415-1702, Kasner, Zdeněk
ORCID: 0000-0002-5753-5538, Castro Ferrera, Thiago
ORCID: 0000-0003-0200-3646 and Belz, Anya
ORCID: 0000-0002-0552-8096
(2022)
Two reproductions of a human-assessed comparative evaluation of a semantic error detection system.
In: International Natural Language Generation Conference, 18-22 July 2022, Waterville, Maine, USA + Online.
Knoll, Tom, Moramarco, Francesco, Korfiatis, Alex Papadopoulos, Young, Rachel, Ruffini, Claudia, Perera, Mark, Perstl, Christian, Reiter, Ehud, Belz, Anya ORCID: 0000-0002-0552-8096 and Savkov, Aleksandar
ORCID: 0009-0009-6831-5563
(2022)
User-driven development of a medical note generation system.
In: 20th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL'22), 10-15 July 2022, Seattle, Washington, USA.
Shimorina, Anastasia and Belz, Anya ORCID: 0000-0002-0552-8096
(2022)
The human evaluation datasheet: a template for recording details of human evaluation experiments in NLP.
In: 2nd Workshop on Human Evaluation of NLP Systems, 27 May 2022, Dublin, Ireland.
ISBN 9781713867395
Belz, Anya ORCID: 0000-0002-0552-8096, Popović, Maja
ORCID: 0000-0001-8234-8745 and Mille, Simon
ORCID: 0000-0002-8852-2764
(2022)
Quantified reproducibility assessment of NLP results.
In: 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 22-27 May 2022, Dublin, Ireland.
Moramarco, Francesco, Korfiatis, Alex Papadopoulos, Perera, Mark, Juric, Damir, Flann, Jack, Reiter, Ehud, Belz, Anya ORCID: 0000-0002-0552-8096 and Savkov, Aleksandar
ORCID: 0009-0009-6831-5563
(2022)
Human evaluation and correlation with automatic metrics in consultation note generation.
In: 60th Annual Meeting of the Association for Computational Linguistics, 22-27 May 2022, Dublin, Ireland.
Popović, Maja ORCID: 0000-0001-8234-8745 and Belz, Anya
ORCID: 0000-0002-0552-8096
(2021)
A reproduction study of an annotation-based human evaluation of MT outputs.
In: 14th International Conference on Natural Language Generation, 20-24 Sept 2021, Aberdeen, Scotland.
Mille, Simon ORCID: 0000-0002-8852-2764, Castro Ferreira, Thiago
ORCID: 0000-0003-0200-3646, Davis, Brian
ORCID: 0000-0002-5759-2655 and Belz, Anya
ORCID: 0000-0002-0552-8096
(2021)
Another PASS: a reproduction study of the human evaluation of a football report generation system.
In: 14th International Conference on Natural Language Generation (INLG 2021), 22-27 May 2022, Aberdeen, Scotland.
Belz, Anya ORCID: 0000-0002-0552-8096, Agarwal, Shubham, Shimorina, Anastasia and Reiter, Ehud
(2021)
A systematic review of reproducibility research in natural language processing.
In: 16th Conference of the European Chapter of the Association for Computational Linguistics: EACL'21, 19 - 23 Apr 2021, Online.
ISBN 978-1-954085-02-2
Howcroft, David ORCID: 0000-0002-0810-9065, Belz, Anya
ORCID: 0000-0002-0552-8096, Gkatzia, Dimitra, Clinciu, Miruna, Hasan, Sadid, Mahamood, Saad
ORCID: 0000-0003-2332-8749, Mille, Simon
ORCID: 0000-0002-8852-2764, van Miltenburg, Emiel
ORCID: 0000-0002-7143-8961, Santhanam, Sashank
ORCID: 0000-0002-9412-3495 and Rieser, Verena
ORCID: 0000-0001-6117-4395
(2020)
Twenty years of confusion in human evaluation: NLG needs evaluation sheets and standardised definitions.
In: 13th International Natural Language Generation Conference 2020 (INLG'20), 15-18 Dec 2020, Dublin, Ireland.
Belz, Anya ORCID: 0000-0002-0552-8096, Mille, Simon
ORCID: 0000-0002-8852-2764 and Howcroft, David M.
ORCID: 0000-0002-0810-9065
(2020)
Disentangling the properties of human evaluation methods: a classification system to support comparability, meta-evaluation and reproducibility testing.
In: 13th International Natural Language Generation Conference 2020 (INLG'20), 15-18 Dec 2020, Dublin, Ireland.
Belz, Anya ORCID: 0000-0002-0552-8096, Hoile, Richard
ORCID: 0000-0001-5003-1655, Ford, Elizabeth
ORCID: 0000-0001-5613-8509 and Mullick, Azam
(2019)
Conceptualisation and annotation of drug nonadherence Information for knowledge extraction from patient-generated texts.
In: 5th Workshop on Noisy User-generated Text (W-NUT 2019), 4 Nov 2019, Hong Kong, China.
Birmingham, Brandon ORCID: 0000-0002-3006-3526, Muscat, Adrian
ORCID: 0000-0002-9157-2818 and Belz, Anya
ORCID: 0000-0002-0552-8096
(2018)
Adding the third dimension to spatial relation detection in 2d images.
In: 11th International Conference on Natural Language Generation, 5-8 Nov 2018, Tilburg, The Netherlands.
Mille, Simon ORCID: 0000-0002-8852-2764, Belz, Anya
ORCID: 0000-0002-0552-8096, Bohnet, Bernd and Wanner, Leo
ORCID: 0000-0002-9446-3748
(2018)
Underspecified universal dependency structures as inputs for multilingual surface realisation.
In: 11th International Conference on Natural Language Generation, 5-8 Nov 2018, Tilburg, The Netherlands.
Belz, Anya, White, Mike, van Genabith, Josef ORCID: 0000-0003-1322-7944, Hogan, Deirdre and Stent, Amanda
(2010)
Finding common ground: towards a surface realisation shared task.
In: INLG 2010 - 6th International Natural Language Generation Conference, 7-9 July 2010, Trim, Ireland.
Belz, Anya ORCID: 0000-0002-0552-8096 and Kilgarriff, Adam
(2006)
Shared-task evaluations in HLT.
In: Fourth International Natural Language Generation Conference on-INLG'06, 15-16 July 2006, Sydney, Australia.
ISBN 9781617384981