I am a postdoctoral fellow at ETH Zürich, where I work with the Rycolab in the Institute for Machine Learning, Department of Computer Science. I am also an associated researcher at the ETH AI Center and a member of the ELLIS Society. Previously, I was a PhD student at the University of Amsterdam in the Institute for Logic, Language and Computation. I study language processing in humans and machines using tools from machine learning, natural language processing, linguistics, and the computational cognitive science of language.

My research is currently concerned with:

• Language modelling (learning, inference, interpretability and evaluation)
• Computational psycholinguistics, semantics and pragmatics
• Computational modelling of language variation and change

It’s-a me, Mario

Born and raised in Italy, I spent three years in Germany as an undergraduate student of Computational Linguistics at the University of Tübingen and then moved to Amsterdam for a Master’s degree in Artificial Intelligence. During my Bachelor’s studies, I worked both as a teaching and as a research assistant for the Department of General and Computational Linguistics, and I served a five-month internship in the IBM department for social media analytics. As a Master’s student, I have collected more teaching and research experience, collaborating with an interdisciplinary set of scholars and students at the Institute for Logic, Language and Computation. I graduated with a thesis on the detection and analysis of lexical semantic change. As a PhD candidate at the University of Amsterdam, I worked in the ILLC’s Dialogue Modelling Group under the supervision of Raquel Fernández and together with many amazing colleagues—and I wrote a thesis on Neural Models of Language Use.

News

• Paper accepted at LREC-COLING with Jun Sen Yee and Arabella Sinclair: Efficiency and Effectiveness in Task‐Oriented Dialogue: On Construction Repetition, Information Rate, and Task Success.
• Paper published at the 1st Personalization of Generative AI (PERSONALIZE) workshop, @EACL2024. Great work by Ivar Frisch! LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Models
• Article published in the CLIN Journal, with Iris Luden and Raquel Fernández. Beyond Perplexity: Examining Temporal Generalization in Large Language Models via Definition Generation
• I am looking for semester projects and theses to supervise. Get in touch! (Here are some ideas, please ask for access permissions.)
• Very excited to have been awarded a competitive postdoctoral fellowship at ETH Zürich, where I will be hosted by the Rycolab!
• I kind of started a blog and my first blog post is out!
• I am giving a keynote at the EMNLP Workshop on Computational Approaches to Historical Language Change (LChange’23)
• New article on Nature Machine Intelligence: A taxonomy and review of generalization research in NLP
• Two papers accepted at EMNLP 2023 (Main), see you in Singapore!
  • What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability
  • Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives
• New paper accepted at CoNLL 2023 with Aron Molnar, Jaap Jumelet and Arabella Sinclair: Attribution and Alignment: Effects of Local Context Repetition on Utterance Production and Comprehension in Dialogue
• New paper at ACL 2023: Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis
• New paper at Findings of ACL 2023: Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind
• Solo paper at EMNLP 2022: Towards Pragmatic Production Strategies for Natural Language Generation
• Paper at AACL-IJCNLP 2022 with Arabella Sinclair and Raquel Fernández: Construction Repetition Reduces Information Rate in Dialogue

Invited talks and guest lectures

• Guest lecture, Seminar on Detection of Semantic Shift, University of Zürich. 21 March 2024.
• Text Technology & Digital Linguistics colloquium, University of Zürich. 5 March 2024.
• Department of Computer Science and Computational Linguistics, Saarland University, Germany. 26 February 2024.
• Keynote at the EMNLP Workshop on Computational Approaches to Historical Language Change (LChange’23), Singapore. 6 December 2023.
• Reatch nanoTalks. Zürich, Switzerland. 26 November 2023.
• Natural Language Processing Seminar. University of Groningen. 3 November.
• Explainable Machine Learning Lab, University of Tübingen. 26 July 2023.
• Cognitive Lexicon Laboratory, University of Toronto. Canada. 13 July 2023.
• COLT Seminar, Universitat Pompeu Fabra. Barcelona, Spain. 29 June 2023.
• ILCC Seminar, School of Informatics, University of Edinburgh. UK. 31 May 2023.
• Rycolab, ETH Zürich. Switzerland. 3 May 2023.
• Inria & Loria. 27 October 2022. Nancy, France. [Abstract]
• School of Natural and Computing Sciences, University of Aberdeen. UK. 12 October 2022. [Abstract]
• Guest lecture, Advanced Information Retrieval (MSc AI and Informatics). University of Amsterdam. Netherlands. 21 September 2022. [Slides]
• Language Technology Group Seminar. Language Technology Group. University of Oslo, Norway. 14 March 2022.
• NLPitch. Institute for Logic, Language and Computation. University of Amsterdam. Netherlands. 26 October 2020.
• AILC Lectures on Computational Linguistics. Università Tor Vergata. Rome, Italy. 17 June 2021.
• Cognitive Machine Learning Lab. Ecole Normale Supérieure. Paris, France. 29 June 2021.
• Symposium on Meaning Variation in Social Contexts. Institute for Logic, Language and Computation. Amsterdam, NL. 5 November 2020.
• Computational Linguistic Seminar. Institute for Logic, Language and Computation. University of Amsterdam, Netherlands. 5 October 2021.
• Guest lecture, NLP 1 (MSc AI course). University of Amsterdam, Netherlands. Online. 18 November 2020.
• Cognitive Science & Artificial Intelligence PhD Group. Tilburg, Netherlands. 1 May 2020.
• Cool Logic Seminar. Amsterdam, Netherlands. 1 February 2019.

My work

• Jun Sen Yee, Mario Giulianelli, and Arabella Sinclair. Efficiency and Effectiveness in Task‐Oriented Dialogue: On Construction Repetition, Information Rate, and Task Success. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC‐COLING 2024).
• [PDF] Ivar Frisch and Mario Giulianelli. 2024. LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Models In Proceedings of the 1st Personalization of Generative AI Workshop (EACL).
• [PDF] Iris Luden, Mario Giulianelli, and Raquel Fernández. 2024. Beyond Perplexity: Examining Temporal Generalization in Large Language Models via Definition Generation. Computational Linguistics in the Netherlands Journal 13 (CLIN).
• [PDF] Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, et al. 2023. A taxonomy and review of generalization research in NLP. Nature Machine Intelligence 5, 1161–1174.
• [PDF] Mario Giulianelli, Sarenne Wallbridge, Raquel Fernández. 2023. Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023).
• [PDF] Mario Giulianelli, Joris Baan, Wilker Aziz, Raquel Fernández, Barbara Plank. 2023. What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023).
• [PDF] Aron Molnar, Jaap Jumelet, Mario Giulianelli, Arabella Sinclair. 2023. Attribution and Alignment: Effects of Local Context Repetition on Utterance Production and Comprehension in Dialogue. In Proceedings of the 27th Conference on Computational Natural Language Learning (CONLL 2023).
• [PDF] Mario Giulianelli, Iris Luden, Raquel Fernández, Andrey Kutuzov. 2023. Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023).
• [PDF] Ece Takmaz, Nicolò Brandizzi, Mario Giulianelli, Sandro Pezzelle, Raquel Fernández. 2023. Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind. In Findings of the 61st Annual Meeting of the Association for Computational Linguistics (Findings of ACL 2023).
• [PDF] Mario Giulianelli. 2022. Towards Pragmatic Production Strategies for Natural Language Generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022)
• [PDF] Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artexte, Tiago Pimentel, et al. 2022. State-of-the-art generalisation research in NLP: A taxonomy and review. Preprint.
• [PDF] Mario Giulianelli, Arabella Sinclair, Raquel Fernández. 2022. Construction Repetition Reduces Information Rate in Dialogue. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (AACL-IJCNLP 2022).
• [PDF] Mario Giulianelli, Andrey Kutuzov, Lidia Pivovarova. 2022. Do Not Fire the Linguist: Grammatical Profiles Help Language Models Detect Semantic Change. In Proceedings of the 3rd Workshop on Computational Approaches to Historical Language Change.
• [PDF][Dataset] Samuel Ryb, Mario Giulianelli, Arabella Sinclair, Raquel Fernández. 2022. AnaLog: Testing Analytical and Deductive Logic Learnability in Language Models. In Proceedings of *SEM 2022: The 11th Joint Conference on Lexical and Computational Semantics.
• [PDF] With many collaborators :). 2022. Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models. In Transactions of Machine Learning Research.
• [PDF][Code] Mario Giulianelli, Arabella Sinclair, Raquel Fernández. 2021. Is Information Density Uniform in Task-Oriented Dialogues? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021).
• [PDF][Code] Mario Giulianelli and Raquel Fernández. 2021. Analysing Human Strategies of Information Transmission as a Function of Discourse Context. In Proceedings of the 25th Conference on Computational Natural Language Learning (CONLL 2021).
• [PDF][Code] Mario Giulianelli, Andrey Kutuzov, Lidia Pivovarova. 2021. Grammatical Profiling for Semantic Change Detection. In Proceedings of the 25th Conference on Computational Natural Language Learning (CONLL 2021).
• [PDF][Code] Mario Giulianelli, Marco Del Tredici, Raquel Fernández. 2020. Analysing Lexical Semantic Change with Contextualised Word Representations. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020).
• [PDF][Code] Andrey Kutuzov and Mario Giulianelli. 2020. UiO-UvA at SemEval-2020 Task 1: Contextualised Embeddings for Lexical Semantic Change Detection. In the Proceedings of the 14th International Workshop on Semantic Evaluation (SemEval 2020).
• [PDF][Code] Ece Takmaz, Mario Giulianelli, Sandro Pezzelle, Arabella Sinclair, Raquel Fernández. 2020. Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contexts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020).
• [PDF][Code] Mario Giulianelli, Jacqueline Harding, Florian Mohnert, Dieuwke Hupkes, Willem Zuidema. 2018. Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information. Best Paper Award at 1st Workshop on Analyzing and Interpreting Neural Networks for NLP (EMNLP 2018).
• [PDF][Code] Mario Giulianelli and Daniel de Kok. 2018. Semi-supervised emotion lexicon expansion with label propagation. Computational Linguistics in the Netherlands Journal 8 (CLIN).

Theses

• [PDF] Neural Models of Language Use. 2023. PhD thesis.
• [PDF] Lexical Semantic Change Analysis with Contextualised Word Representations. 2019. Master’s thesis.
• [PDF] Semi-supervised emotion lexicon expansion with label propagation and specialized word embeddings. 2017. Bachelor’s thesis.

The only way to predict the future is to invent it.