ACL Anthology Google Scholar

Refereed Journal Articles

  1. Rebecca KNOWLES and Chi-kiu LO.
    Calibration and context in human evaluation of machine translation
    Natural Language Processing. Cambridge University Press. June 2024.

Refereed Conference and Workshop Papers (recent)

  1. Shriaya Vaagdevi CHIKATI, Samuel LARKIN, David MINICOLA and Chi-kiu LO.
    Challenges in Technical Regulatory Text Variation Detection
    In the Proceedings of the 1st Regulatory NLP Workshop (RegNLP 2025). Abu Dhabi, UAE. Jan 2025.
  2. Rebecca KNOWLES, Samuel LARKIN and Chi-kiu LO.
    MSLC24: Further Challenges for Metrics on a Wide Landscape of Translation Quality
    In the Ninth conference on Machine Translation (WMT 2024). Miami, Florida, USA. Nov 2024.
  3. Hillary DAWKINS, Isar NEJADGHOLI, and Chi-kiu LO.
    WMT24 Test Suite: Gender Resolution in Speaker-Listener Dialogue Roles
    In the Ninth conference on Machine Translation (WMT 2024). Miami, Florida, USA. Nov 2024.
  4. Samuel LARKIN, Chi-kiu LO and Rebecca KNOWLES.
    MSLC24 Submissions to the General Machine Translation Task
    In the Ninth conference on Machine Translation (WMT 2024). Miami, Florida, USA. Nov 2024.
  5. Ting LIU, Rebecca KNOWLES, Chi-kiu LO and Elizabeth MARSHMAN.
    Machine translation (MT) quality evaluation: What do MT experts have to learn from translators and translation scholars?
    In the Proceedings of Translating and the Computer 46 (TC46). Luxembourg. Nov 2024.
  6. Ting LIU, Chi-kiu LO, Elizabeth MARSHMAN and Rebecca KNOWLES.
    Evaluation Briefs: Drawing on Translation Studies for Human Evaluation of MT
    In the 16th Conference of the Association for Machine Translation in the Americas (AMTA 2024). Chicago, USA. Sep 2024.
  7. Rebecca KNOWLES, Samuel LARKIN, Michel SIMARD, Marc A TESSIER, Gabriel BERNIER-COLBORNE, Cyril GOUTTE and Chi-kiu LO.
    Some Tradeoffs in Continual Learning for Parliamentary Neural Machine Translation Systems
    In the 16th Conference of the Association for Machine Translation in the Americas (AMTA 2024). Chicago, USA. Sep 2024.
  8. Chi-kiu LO, Samuel LARKIN and Rebecca KNOWLES.
    Metric Score Landscape Challenge (MSLC23): Understanding Metrics’ Performance on a Wider Landscape of Translation Quality
    In the Eighth conference on Machine Translation (WMT 2023). Singapore. Dec 2023.
  9. Chi-kiu LO, Rebecca KNOWLES and Cyril GOUTTE.
    Beyond Correlation: Making Sense of the Score Differences of New MT Evaluation Metrics
    In the Machine Translation Summit XIX (MTSummit 2020). Macau, China. Sept 2020.
  10. Chi-kiu LO and Rebecca KNOWLES.
    Data Sampling and (In)stability in Machine Translation Evaluation
    In Findings of the Association for Computational Linguistics (ACL 2023). Toronto, Canada. Jul 2023.
  11. Rebecca KNOWLES and Chi-kiu LO.
    Test Set Sampling Affects System Rankings: Expanded Human Evaluation of WMT20 English-Inuktitut Systems
    In the Seventh conference on Machine Translation (WMT 2022). Abu Dhabi, United Arab Emirates. Dec 2022.
  12. Chi-kiu LO.
    Extended Study on Using Pretrained Language Models and YiSi-1 for Machine Translation Evaluation
    In the Fifth conference on Machine Translation (WMT 2020). Online. Nov 2020.
  13. Chi-kiu LO and Samuel LARKIN.
    Machine Translation Reference-less Evaluation using YiSi-2 with Bilingual Mappings of Massive Multilingual Language Model
    In the Fifth conference on Machine Translation (WMT 2020). Online. Nov 2020.
  14. Chi-kiu LO and Eric JOANIS.
    Improving Parallel Data Identification using Iteratively Refined Sentence Alignments and Bilingual Mappings of Pre-trained Language Models
    In the Fifth conference on Machine Translation (WMT 2020). Online. Nov 2020.
  15. Eric JOANIS, Rebecca KNOWLES, Roland KUHN, Samuel LARKIN, Patrick LITTELL, Chi-kiu LO, Darlene STEWART and Jeffrey MICHER.
    The Nunavut Hansard Inuktitut–English Parallel Corpus 3.0 with Preliminary Machine Translation Results
    In the 12th Language Resources and Evaluation Conference (LREC 2020). Marseille (Online). May 2020.
  16. Chi-kiu LO and Michel SIMARD.
    Fully Unsupervised Crosslingual Semantic Textual Similarity Metric Based on BERT for Identifying Parallel Data
    In the 23rd conference on Computational Natural Language Learning (CoNLL 2019). Hong Kong. Nov 2019.
  17. Chi-kiu LO.
    YiSi-A unified semantic MT quality evaluation and estimation metric for languages with different levels of available resources
    In the Fourth conference on Machine Translation (WMT 2019). Florence, Italy. Aug 2019.
  18. Patrick LITTELL, Chi-kiu LO, Samuel LARKIN and Darlene STEWART.
    Multi-Source Transformer for Kazakh-Russian-English Neural Machine Translation
    In the Fourth conference on Machine Translation (WMT 2019). Florence, Italy. Aug 2019.
  19. Gabriel BERNIER-COLBORNE and Chi-kiu LO.
    NRC Parallel Corpus Filtering System for WMT 2019
    In the Fourth conference on Machine Translation (WMT 2019). Florence, Italy. Aug 2019.
  20. Chi-kiu LO, Michel SIMARD, Darlene STEWART, Samuel LARKIN, Cyril GOUTTE and Patrick LITTELL.
    Accurate semantic textual similarity for cleaning noisy parallel corpora using semantic machine translation evaluation metric: The NRC supervised submissions to the parallel corpus filtering task
    In the Third conference on Machine Translation (WMT 2018). Brussels, Belgium. Nov 2018.
  21. Patrick LITTELL, Samuel LARKIN, Darlene STEWART, Michel SIMARD, Cyril GOUTTE and Chi-kiu LO.
    Measuring sentence parallelism using Mahalanobis distances: The NRC unsupervised submissions to the WMT18 Parallel Corpus Filtering shared task
    In the Third conference on Machine Translation (WMT 2018). Brussels, Belgium. Nov 2018.
  22. Chi-kiu LO.
    MEANT 2.0: Accurate semantic MT evaluation for any output language
    In the Second conference on Machine Translation (WMT 2017). Copenhagen, Denmark. Sep 2017.

Shared Task Overview Papers

  1. Markus FREITAG, Nitika MATHUR, Daniel DEUTSCH, Chi-kiu LO, Eleftherios AVRAMIDIS, Ricardo REI, Brian THOMPSON, Frédéric BLAIN, Tom KOCMI, Jiayi WANG, David Ifeoluwa ADELANI, Marianna BUCHICCHIO, Chrysoula ZERVA, and Alon LAVIE.
    Are LLMs Breaking MT Metrics? Results of the WMT24 Metrics Shared Task
    In the Ninth conference on Machine Translation (WMT 2024). Miami, Florida, USA. Nov 2024.
  2. Markus FREITAG, Nitika MATHUR, Chi-kiu LO, Eleftherios AVRAMIDIS, Ricardo REI, Brian THOMPSON, Tom KOCMI, Frédéric BLAIN, Daniel DEUTSCH, Craig STEWART, Chrysoula ZERVA, Sheila CASTILHO, Alon LAVIE and George FOSTER
    Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not Innocent
    In the Eighth conference on Machine Translation (WMT 2023). Singapore. Dec 2023.
  3. Markus FREITAG, Ricardo REI, Nitika MATHUR, Chi-kiu LO, Craig STEWART, Eleftherios AVRAMIDIS, Tom KOCMI, George FOSTER, Alon LAVIE and André FT MARTINS.
    Results of WMT22 Metrics Shared Task: Stop Using BLEU–Neural Metrics Are Better and More Robust
    In the Seventh conference on Machine Translation (WMT 2022). Abu Dhabi, United Arab Emirates. Dec 2022.
  4. Markus FREITAG, Ricardo REI, Nitika MATHUR, Chi-kiu LO, Craig STEWART, George FOSTER, Alon LAVIE and Ondřej BOJAR.
    Results of the WMT21 Metrics Shared Task: Evaluating Metrics with Expert-based Human Evaluations on TED and News Domain
    In the Sixth conference on Machine Translation (WMT 2021). Punta Cana, Dominican Republic. Nov 2021.
  5. Loïc BARRAULT, Magdalena BIESIALSKA, Ondřej BOJAR, Marta R. COSTA-JUSSÀ, Christian FEDERMANN, Yvette GRAHAM, Roman GRUNDKIEWICZ, Barry HADDOW, Matthias HUCK, Eric JOANIS, Tom KOCMI, Philipp KOEHN, Chi-kiu LO, Nikola LJUBEŠIĆ, Christof MONZ, Makoto MORISHITA, Masaaki NAGATA, Toshiaki NAKAZAWA, Santanu PAL, Matt POST and Marcos ZAMPIERI.
    Findings of the 2020 Conference on Machine Translation (WMT20)
    In the Fifth conference on Machine Translation (WMT 2020). Online. Nov 2020.

Articles

  1. Lane SCHWARTZ, Francis TYERS, Lori LEVIN, Christo KIROV, Patrick LITTELL, Chi-kiu LO, Emily PRUD’HOMMEAUX, Hyunji Hayley PARK, Kenneth STEIMEL, Rebecca KNOWLES, Jeffrey MICHER, Lonny STRUNK, Han LIU, Coleman HALEY, Katherine J ZHANG, Robbie JIMMERSON, Vasilisa ANDRIYANETS, Aldrian Obaja MUIS, Noaki OTANI, Jong Hyuk PARK and Zhisong ZHANG.
    Neural Polysynthetic Language Modelling
    Final Report of the Sixth Frederick Jelinek Memorial Summer Workshop on Neural Polysynthetic Language Modelling. In arXiv preprint arXiv:2005.05477. May 2020.
  2. Dekai WU, Pascale FUNG, Marine CARPUAT, Chi-kiu LO, Yongsheng YANG and Zhaojun WU.
    Lexical Semantics for Statistical Machine Translation
    Handbook of Natural Language Processing and Machine Translation: DARPA Global Autonomous Language Exploitation. (p. 236-252) Springer. 2011.

Theses

  1. Chi-kiu LO.
    MEANT: A highly accurate semantic frame based evaluation metric for improving machine translation utility
    Doctor of Philosophy in Computer Science and Engineering Thesis, The Hong Kong University of Science and Technology (HKUST). Aug 2018.
  2. Chi-kiu LO.
    Using semantic role labels to reorder statistical machine translation output
    Master of Philosophy in Computer Science and Engineering Thesis, The Hong Kong University of Science and Technology (HKUST). Aug 2009.

Others

  1. Chi-kiu LO.
    了解機器翻譯　分析錯譯原因
    眾新聞 CitizenNews 立場新聞 StandNews Jan 2021.
  2. Chi-kiu LO.
    Writing Cantonese with XeLaTeX on Overleaf
    Technical note. Jul 2020.