publications

2024

1. Self-supervised Speech Representations Still Struggle with African American Vernacular English

   Kalvin Chang, Yi-Hui Chou, Jiatong Shi, and 4 more authors

   In Proc. INTERSPEECH 2024, 2024
2. Can Large Language Models Code Like a Linguist?: A Case Study in Low Resource Sound Law Induction

   Atharva Naik, Kexun Zhang, Nathaniel Robinson, and 7 more authors

   2024
3. Zero-Shot Cross-Lingual NER Using Phonemic Representations for Low-Resource Languages

   Jimin Sohn, Haeji Jung, Alex Cheng, and 3 more authors

   2024
4. Semisupervised Neural Proto-Language Reconstruction

   Liang Lu, Peirong Xie, and David R. Mortensen

   2024
5. Neural Proto-Language Reconstruction

   Chenxuan Cui, Ying Chen, Qinxin Wang, and 1 more author

   2024
6. A Review of the Applications of Deep Learning-Based Emergent Communication

   Brendon Boldt, and David R. Mortensen

   Transactions on Machine Learning Research, 2024
7. Constructions Are So Difficult That Even Large Language Models Get Them Right for the Wrong Reasons

   Shijia Zhou, Leonie Weissweiler, Taiqi He, and 3 more authors

   In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), May 2024

   Abs

   In this paper, we make a contribution that can be understood from two perspectives: from an NLP perspective, we introduce a small challenge dataset for NLI with large lexical overlap, which minimises the possibility of models discerning entailment solely based on token distinctions, and show that GPT-4 and Llama 2 fail it with strong bias. We then create further challenging sub-tasks in an effort to explain this failure. From a Computational Linguistics perspective, we identify a group of constructions with three classes of adjectives which cannot be distinguished by surface features. This enables us to probe for LLM’s understanding of these constructions in various ways, and we find that they fail in a variety of ways to distinguish between them, suggesting that they don’t adequately represent their meaning or capture the lexical properties of phrasal heads.

8. Improved Neural Protoform Reconstruction via Reflex Prediction

   Liang Lu, Jingzhi Wang, and David R. Mortensen

   In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), May 2024

   Abs

   Protolanguage reconstruction is central to historical linguistics. The comparative method, one of the most influential theoretical and methodological frameworks in the history of the language sciences, allows linguists to infer protoforms (reconstructed ancestral words) from their reflexes (related modern words) based on the assumption of regular sound change. Not surprisingly, numerous computational linguists have attempted to operationalize comparative reconstruction through various computational models, the most successful of which have been supervised encoder-decoder models, which treat the problem of predicting protoforms given sets of reflexes as a sequence-to-sequence problem. We argue that this framework ignores one of the most important aspects of the comparative method: not only should protoforms be inferable from cognate sets (sets of related reflexes) but the reflexes should also be inferable from the protoforms. Leveraging another line of research—reflex prediction—we propose a system in which candidate protoforms from a reconstruction model are reranked by a reflex prediction model. We show that this more complete implementation of the comparative method allows us to surpass state-of-the-art protoform reconstruction methods on three of four Chinese and Romance datasets.

9. Phonotactic Complexity across Dialects

   Ryan Soh-Eun Shim, Kalvin Chang, and David R. Mortensen

   In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), May 2024

   Abs

   Received wisdom in linguistic typology holds that if the structure of a language becomes more complex in one dimension, it will simplify in another, building on the assumption that all languages are equally complex (Joseph and Newmeyer, 2012). We study this claim on a micro-level, using a tightly-controlled sample of Dutch dialects (across 366 collection sites) and Min dialects (across 60 sites), which enables a more fair comparison across varieties. Even at the dialect level, we find empirical evidence for a tradeoff between word length and a computational measure of phonotactic complexity from a LSTM-based phone-level language model—a result previously documented only at the language level. A generalized additive model (GAM) shows that dialects with low phonotactic complexity concentrate around the capital regions, which we hypothesize to correspond to prior hypotheses that language varieties of greater or more diverse populations show reduced phonotactic complexity. We also experiment with incorporating the auxiliary task of predicting syllable constituency, but do not find an increase in the strength of the negative correlation observed.

10. PWESuite: Phonetic Word Embeddings and Tasks They Facilitate

    Vilém Zouhar, Kalvin Chang, Chenxuan Cui, and 4 more authors

    In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), May 2024

    Abs

    Mapping words into a fixed-dimensional vector space is the backbone of modern NLP. While most word embedding methods successfully encode semantic information, they overlook phonetic information that is crucial for many tasks. We develop three methods that use articulatory features to build phonetically informed word embeddings. To address the inconsistent evaluation of existing phonetic word embedding methods, we also contribute a task suite to fairly evaluate past, current, and future methods. We evaluate both (1) intrinsic aspects of phonetic word embeddings, such as word retrieval and correlation with sound similarity, and (2) extrinsic performance on tasks such as rhyme and cognate detection and sound analogies. We hope our task suite will promote reproducibility and inspire future phonetic embedding research.

11. Verbing Weirds Language (Models): Evaluation of English Zero-Derivation in Five LLMs

    David R. Mortensen, Valentina Izrailevitch, Yunze Xiao, and 2 more authors

    In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), May 2024

    Abs

    Lexical-syntactic flexibility, in the form of conversion (or zero-derivation) is a hallmark of English morphology. In conversion, a word with one part of speech is placed in a non-prototypical context, where it is coerced to behave as if it had a different part of speech. However, while this process affects a large part of the English lexicon, little work has been done to establish the degree to which language models capture this type of generalization. This paper reports the first study on the behavior of large language models with reference to conversion. We design a task for testing lexical-syntactic flexibility—the degree to which models can generalize over words in a construction with a non-prototypical part of speech. This task is situated within a natural language inference paradigm. We test the abilities of five language models—two proprietary models (GPT-3.5 and GPT-4), three open source model (Mistral 7B, Falcon 40B, and Llama 2 70B). We find that GPT-4 performs best on the task, followed by GPT-3.5, but that the open source language models are also able to perform it and that the 7-billion parameter Mistral displays as little difference between its baseline performance on the natural language inference task and the non-prototypical syntactic category task, as the massive GPT-4.

12. XferBench: a Data-Driven Benchmark for Emergent Language

    Brendon Boldt, and David Mortensen

    In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Jun 2024

    Abs

    In this paper, we introduce a benchmark for evaluating the overall quality of emergent languages using data-driven methods. Specifically, we interpret the notion of the “quality” of an emergent language as its similarity to human language within a deep learning framework. We measure this by using the emergent language as pretraining data for a downstream NLP tasks in human language—the better the downstream performance, the better the emergent language. We implement this benchmark as an easy-to-use Python package that only requires a text file of utterances from the emergent language to be evaluated. Finally, we empirically test the benchmark’s validity using human, synthetic, and emergent language baselines.

2023

1. Kuki-Chin Phonology: An Overview

   David R Mortensen

   Himalayan Linguistics, Jun 2023
2. Evaluating self-supervised speech models on a Taiwanese Hokkien corpus

   Yi-Hui Chou, Kalvin Chang, Meng-Ju Wu, and 8 more authors

   In 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Jun 2023
3. African Substrates Rather Than European Lexifiers to Augment African-diaspora Creole Translation

   Nathaniel Romney Robinson, Matthew Dean Stutzman, Stephen D. Richardson, and 1 more author

   In 4th Workshop on African Natural Language Processing, Jun 2023
4. Automating Sound Change Prediction for Phylogenetic Inference: A Tukanoan Case Study

   Kalvin Chang, Nathaniel Robinson, Anna Cai, and 3 more authors

   In Proceedings of the 4th Workshop on Computational Approaches to Historical Language Change, Dec 2023

   Abs

   We describe a set of new methods to partially automate linguistic phylogenetic inference given (1) cognate sets with their respective protoforms and sound laws, (2) a mapping from phones to their articulatory features and (3) a typological database of sound changes.We train a neural network on these sound change data to weight articulatory distances between phones and predict intermediate sound change steps between historical protoforms and their modern descendants, replacing a linguistic expert in part of a parsimony-based phylogenetic inference algorithm. In our best experiments on Tukanoan languages, this method produces trees with a Generalized Quartet Distance of 0.12 from a tree that used expert annotations, a significant improvement over other semi-automated baselines. We discuss potential benefits and drawbacks to our neural approach and parsimony-based tree prediction. We also experiment with a minimal generalization learner for automatic sound law induction, finding it less effective than sound laws from expert annotation. Our code is publicly available.

5. ChatGPT MT: Competitive for High- (but Not Low-) Resource Languages

   Nathaniel Robinson, Perez Ogayo, David R. Mortensen, and 1 more author

   In Proceedings of the Eighth Conference on Machine Translation, Dec 2023

   Abs

   Large language models (LLMs) implicitly learn to perform a range of language tasks, including machine translation (MT). Previous studies explore aspects of LLMs’ MT capabilities. However, there exist a wide variety of languages for which recent LLM MT performance has never before been evaluated. Without published experimental evidence on the matter, it is difficult for speakers of the world’s diverse languages to know how and whether they can use LLMs for their languages. We present the first experimental evidence for an expansive set of 204 languages, along with MT cost analysis, using the FLORES-200 benchmark. Trends reveal that GPT models approach or exceed traditional MT model performance for some high-resource languages (HRLs) but consistently lag for low-resource languages (LRLs), under-performing traditional MT for 84.1% of languages we covered. Our analysis reveals that a language’s resource level is the most important feature in determining ChatGPT’s relative ability to translate it, and suggests that ChatGPT is especially disadvantaged for LRLs and African languages.

6. Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models

   Orevaoghene Ahia, Sachin Kumar, Hila Gonen, and 4 more authors

   In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Dec 2023

   Abs

   Language models have graduated from being research prototypes to commercialized products offered as web APIs, and recent works have highlighted the multilingual capabilities of these products. The API vendors charge their users based on usage, more specifically on the number of “tokens” processed or generated by the underlying language models. What constitutes a token, however, is training data and model dependent with a large variance in the number of tokens required to convey the same information in different languages. In this work, we analyze the effect of this non-uniformity on the fairness of an API’s pricing policy across languages. We conduct a systematic analysis of the cost and utility of OpenAI’s language model API on multilingual benchmarks in 22 typologically diverse languages. We show evidence that speakers of a large number of the supported languages are overcharged while obtaining poorer results. These speakers tend to also come from regions where the APIs are less affordable, to begin with. Through these analyses, we aim to increase transparency around language model APIs’ pricing policies and encourage the vendors to make them more equitable.

7. Counting the Bugs in ChatGPT’s Wugs: A Multilingual Investigation into the Morphological Capabilities of a Large Language Model

   Leonie Weissweiler, Valentin Hofmann, Anjali Kantharuban, and 10 more authors

   In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Dec 2023

   Abs

   Large language models (LLMs) have recently reached an impressive level of linguistic capability, prompting comparisons with human language skills. However, there have been relatively few systematic inquiries into the linguistic capabilities of the latest generation of LLMs, and those studies that do exist (i) ignore the remarkable ability of humans to generalize, (ii) focus only on English, and (iii) investigate syntax or semantics and overlook other capabilities that lie at the heart of human language, like morphology. Here, we close these gaps by conducting the first rigorous analysis of the morphological capabilities of ChatGPT in four typologically varied languages (specifically, English, German, Tamil, and Turkish). We apply a version of Berko’s (1958) wug test to ChatGPT, using novel, uncontaminated datasets for the four examined languages. We find that ChatGPT massively underperforms purpose-built systems, particularly in English. Overall, our results—through the lens of morphology—cast a new light on the linguistic capabilities of ChatGPT, suggesting that claims of human-like language skills are premature and misleading.

8. Calibrated Seq2seq Models for Efficient and Generalizable Ultra-fine Entity Typing

   Yanlin Feng, Adithya Pratapa, and David Mortensen

   In Findings of the Association for Computational Linguistics: EMNLP 2023, Dec 2023

   Abs

   Ultra-fine entity typing plays a crucial role in information extraction by predicting fine-grained semantic types for entity mentions in text. However, this task poses significant challenges due to the massive number of entity types in the output space. The current state-of-the-art approaches, based on standard multi-label classifiers or cross-encoder models, suffer from poor generalization performance or inefficient inference speed. In this paper, we present CASENT, a seq2seq model designed for ultra-fine entity typing that predicts ultra-fine types with calibrated confidence scores. Our model takes an entity mention as input and employs constrained beam search to generate multiple types autoregressively. The raw sequence probabilities associated with the predicted types are then transformed into confidence scores using a novel calibration method. We conduct extensive experiments on the UFET dataset which contains over 10k types. Our method outperforms the previous state-of-the-art in terms of F1 score and calibration error, while achieving an inference speedup of over 50 times. Additionally, we demonstrate the generalization capabilities of our model by evaluating it in zero-shot and few-shot settings on five specialized domain entity typing datasets that are unseen during training. Remarkably, our model outperforms large language models with 10 times more parameters in the zero-shot setting, and when fine-tuned on 50 examples, it significantly outperforms ChatGPT on all datasets.

9. Construction Grammar Provides Unique Insight into Neural Language Models

   Leonie Weissweiler, Taiqi He, Naoki Otani, and 3 more authors

   In Proceedings of the First International Workshop on Construction Grammars and NLP (CxGs+NLP, GURT/SyntaxFest 2023), Mar 2023

   Abs

   Construction Grammar (CxG) has recently been used as the basis for probing studies that have investigated the performance of large pretrained language models (PLMs) with respect to the structure and meaning of constructions. In this position paper, we make suggestions for the continuation and augmentation of this line of research. We look at probing methodology that was not designed with CxG in mind, as well as probing methodology that was designed for specific constructions. We analyse selected previous work in detail, and provide our view of the most important challenges and research questions that this promising new field faces.

10. Transformed Protoform Reconstruction

    Young Min Kim, Kalvin Chang, Chenxuan Cui, and 1 more author

    In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Jul 2023

    Abs

    Protoform reconstruction is the task of inferring what morphemes or words appeared like in the ancestral languages of a set of daughter languages. Meloni et al (2021) achieved the state-of-the-art on Latin protoform reconstruction with an RNN-based encoder-decoder with attention model. We update their model with the state-of-the-art seq2seq model: the Transformer. Our model outperforms their model on a suite of different metrics on two different datasets: their Romance data of 8,000 cognates spanning 5 languages and a Chinese dataset (Hou 2004) of 800+ cognates spanning 39 varieties. We also probe our model for potential phylogenetic signal contained in the model. Our code is publicly available at \urlhttps://github.com/cmu-llab/acl-2023.

11. Generalized Glossing Guidelines: An Explicit, Human- and Machine-Readable, Item-and-Process Convention for Morphological Annotation

    David R. Mortensen, Ela Gulsen, Taiqi He, and 4 more authors

    In Proceedings of the 20th SIGMORPHON workshop on Computational Research in Phonetics, Phonology, and Morphology, Jul 2023

    Abs

    Interlinear glossing provides a vital type of morphosyntactic annotation, both for linguists and language revitalists, and numerous conventions exist for representing it formally and computationally. Some of these formats are human readable; others are machine readable. Some are easy to edit with general-purpose tools. Few represent non-concatentative processes like infixation, reduplication, mutation, truncation, and tonal overwriting in a consistent and formally rigorous way (on par with affixation). We propose an annotation convention—Generalized Glossing Guidelines (GGG) that combines all of these positive properties using an Item-and-Process (IP) framework. We describe the format, demonstrate its linguistic adequacy, and compare it with two other interlinear glossed text annotation schemes.

12. Multilingual TTS Accent Impressions for Accented ASR

    Georgios Karakasidis, Nathaniel Robinson, Yaroslav Getman, and 6 more authors

    In International Conference on Text, Speech, and Dialogue, Jul 2023
13. SigMoreFun Submission to the SIGMORPHON Shared Task on Interlinear Glossing

    Taiqi He, Lindia Tjuatja, Nathaniel Robinson, and 4 more authors

    In Proceedings of the 20th SIGMORPHON workshop on Computational Research in Phonetics, Phonology, and Morphology, Jul 2023

    Abs

    In our submission to the SIGMORPHON 2023 Shared Task on interlinear glossing (IGT), we explore approaches to data augmentation and modeling across seven low-resource languages. For data augmentation, we explore two approaches: creating artificial data from the provided training data and utilizing existing IGT resources in other languages. On the modeling side, we test an enhanced version of the provided token classification baseline as well as a pretrained multilingual seq2seq model. Additionally, we apply post-correction using a dictionary for Gitksan, the language with the smallest amount of data. We find that our token classification models are the best performing, with the highest word-level accuracy for Arapaho and highest morpheme-level accuracy for Gitksan out of all submissions. We also show that data augmentation is an effective strategy, though applying artificial data pretraining has very different effects across both models tested.

2022

1. Data-adaptive Transfer Learning for Translation: A Case Study in Haitian and Jamaican

   Nathaniel Robinson, Cameron Hogan, Nancy Fulda, and 1 more author

   In Proceedings of the Fifth Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2022), Oct 2022

   Abs

   Multilingual transfer techniques often improve low-resource machine translation (MT). Many of these techniques are applied without considering data characteristics. We show in the context of Haitian-to-English translation that transfer effectiveness is correlated with amount of training data and relationships between knowledge-sharing languages. Our experiments suggest that for some languages beyond a threshold of authentic data, back-translation augmentation methods are counterproductive, while cross-lingual transfer from a sufficiently related language is preferred. We complement this finding by contributing a rule-based French-Haitian orthographic and syntactic engine and a novel method for phonological embedding. When used with multilingual techniques, orthographic transformation makes statistically significant improvements over conventional methods. And in very low-resource Jamaican MT, code-switching with a transfer language for orthographic resemblance yields a 6.63 BLEU point advantage.

2. WikiHan: A New Comparative Dataset for Chinese Languages

   Kalvin Chang, Chenxuan Cui, Youngmin Kim, and 1 more author

   In COLING 2022, Oct 2022
3. When Is TTS Augmentation Through a Pivot Language Useful?

   Nathaniel Romney Robinson, Perez Ogayo, Swetha R. Gangu, and 2 more authors

   In Interspeech 2022, Oct 2022
4. Speech Recognition for Around 2000 Languages without Audio

   Xinjian Li, Florian Metze, David R. Mortensen, and 2 more authors

   In Interspeech 2022, Oct 2022
5. A Hmong Corpus with Elaborate Expression Annotations

   David R. Mortensen, Xinyu Zhang, Chenxuan Cui, and 1 more author

   In Proceedings of the Thirteenth International Conference on Language Resources and Evaluation (LREC 2022), Oct 2022
6. Phone Inventories and Recognition for Every Language

   Xinjian Li, Florian Metze, David R. Mortensen, and 2 more authors

   In Proceedings of the Thirteenth International Conference on Language Resources and Evaluation (LREC 2022), Oct 2022
7. Large-Scale Computerized Forward Reconstruction Yields New Perspectives in French Diachronic Phonology

   Clayton Marr, and David Mortensen

   Diachronica, Oct 2022
8. Zero-shot Learning for Grapheme to Phoneme Conversion with Language Ensemble

   Xinjian Li, Florian Metze, David Mortensen, and 2 more authors

   In Findings of the Association for Computational Linguistics: ACL 2022, May 2022

   Abs

   Grapheme-to-Phoneme (G2P) has many applications in NLP and speech fields. Most existing work focuses heavily on languages with abundant training datasets, which limits the scope of target languages to less than 100 languages. This work attempts to apply zero-shot learning to approximate G2P models for all low-resource and endangered languages in Glottolog (about 8k languages). For any unseen target language, we first build the phylogenetic tree (i.e. language family tree) to identify top-k nearest languages for which we have training sets. Then we run models of those languages to obtain a hypothesis set, which we combine into a confusion network to propose a most likely hypothesis as an approximation to the target language. We test our approach on over 600 unseen languages and demonstrate it significantly outperforms baselines.

9. Learning the Ordering of Coordinate Compounds and Elaborate Expressions in Hmong, Lahu, and Chinese

   Chenxuan Cui, Katherine Zhang, and David Mortensen

   In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Jul 2022

   Abs

   Coordinate compounds (CCs) and elaborate expressions (EEs) are coordinate constructions common in languages of East and Southeast Asia. Mortensen (2006) claims that (1) the linear ordering of EEs and CCs in Hmong, Lahu, and Chinese can be predicted via phonological hierarchies and (2) that these phonological hierarchies lack a clear phonetic rationale. These claims are signficant because morphosyntax has often been seen as in a feed-forward relationship with phonology, and phonological generalizations have often been assumed to be phonetically “natural”. We investigate whether the ordering of CCs and EEs can be learned empirically and whether computational models (classifiers and sequence-labeling models) learn unnatural hierarchies similar to those posited by Mortensen (2006). We find that decision trees and SVMs learn to predict the order of CCs/EEs on the basis of phonology, beating strong baselines for all three languages, with DTs learning hierarchies strikingly similar to those proposed by Mortensen. However, we also find that a neural sequence labeling model is able to learn the ordering of elaborate expressions in Hmong very effectively without using any phonological information. We argue that EE ordering can be learned through two independent routes: phonology and lexical distribution, presenting a more nuanced picture than previous work.

10. Wfst4Str

    David R. Mortensen

    Jul 2022

    Rust/Python library for working with strings using weighted finite state transducers

2021

1. ASR2K: Speech Recognition for Around 2000 Languages without Audio

   In Interspeech 2022, Jul 2021
2. Quantifying Cognitive Factors in Lexical Decline

   David Francis, Ella Rabinovich, Farhan Samir, and 2 more authors

   Transactions of the Association for Computational Linguistics, Dec 2021

   Abs

   We adopt an evolutionary view on language change in which cognitive factors (in addition to social ones) affect the fitness of words and their success in the linguistic ecosystem. Specifically, we propose a variety of psycholinguistic factors—semantic, distributional, and phonological—that we hypothesize are predictive of lexical decline, in which words greatly decrease in frequency over time. Using historical data across three languages (English, French, and German), we find that most of our proposed factors show a significant difference in the expected direction between each curated set of declining words and their matched stable words. Moreover, logistic regression analyses show that semantic and distributional factors are significant in predicting declining words. Further diachronic analysis reveals that declining words tend to decrease in the diversity of their lexical contexts over time, gradually narrowing their ‘ecological niches’.

3. Multilingual phonetic dataset for low resource speech recognition

   Xinjian Li, David R Mortensen, Florian Metze, and 1 more author

   In ICASSP 2021, Dec 2021
4. East Tusom: A phonetic and phonological sketch of a largely undocumented Tangkhulic language

   David R. Mortensen, and Jordan Picone

   Dec 2021

   Abs

   East Tusom is a Tibeto-Burman language of Manipur, India, belonging to the Tangkhulic group. While it shares some innovations with the other Tangkhulic languages, it differs markedly from “Standard Tangkhul” (which is based on the speech of Ukhrul town). Past documentation is limited to a small set of hastily transcribed forms in a comparative reconstruction of Tangkhulic rhymes (Mortensen & Miller 2013; Mortensen 2012). This paper presents the first substantial sketch of an aspect of the language: its (descriptive) phonetics and phonology. The data are based on recordings of an extensive wordlist (730 items) and one short text, all from one fluent native speaker in her mid-twenties. We present the phonetic inventory of East Tusom and a phonemicization, with exhaustive examples. We also present an overview of the major phonological patterns and generalizations in the language. Of special interest are a “placeless nasal” that is realized as nasalization on the preceding vowel unless it is followed by a consonant, and numerous plosive-fricative clusters (where the fricative is roughly homorganic with the following vowel) that have developed from historical aspirated plosives. A complete wordlist, organized by gloss and semantic field, is provided as appendices.

5. Tusom2021: A Phonetically Transcribed Speech Dataset from an Endangered Language for Universal Phone Recognition Experiments

   David R. Mortensen, Jordan Picone, Xinjian Li, and 1 more author

   In Proc. Interspeech 2021, Dec 2021
6. Evaluating the Morphosyntactic Well-formedness of Generated Texts

   Adithya Pratapa, Antonios Anastasopoulos, Shruti Rijhwani, and 4 more authors

   In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Nov 2021

   Abs

   Text generation systems are ubiquitous in natural language processing applications. However, evaluation of these systems remains a challenge, especially in multilingual settings. In this paper, we propose L’AMBRE – a metric to evaluate the morphosyntactic well-formedness of text using its dependency parse and morphosyntactic rules of the language. We present a way to automatically extract various rules governing morphosyntax directly from dependency treebanks. To tackle the noisy outputs from text generation systems, we propose a simple methodology to train robust parsers. We show the effectiveness of our metric on the task of machine translation through a diachronic study of systems translating into morphologically-rich languages.

7. Phoneme Recognition Through Fine Tuning of Phonetic Representations: A Case Study on Luhya Language Varieties

   Kathleen Siminyu, Xinjian Li, Antonios Anastasopoulos, and 3 more authors

   In Proc. Interspeech 2021, Nov 2021
8. Ranking Transfer Languages with Pragmatically-Motivated Features for Multilingual Sentiment Analysis

   Jimin Sun, Hwijeen Ahn, Chan Young Park, and 2 more authors

   In EACL 2021, Nov 2021
9. Differentiable Allophone Graphs for Language-Universal Speech Recognition

   Brian Yan, Siddharth Dalmia, David R. Mortensen, and 2 more authors

   In Proc. Interspeech 2021, Nov 2021

2020

1. Automatic Extraction of Rules Governing Morphological Agreement

   Aditi Chaudhary, Antonios Anastasopoulos, Adithya Pratapa, and 4 more authors

   In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Nov 2020
2. Towards Zero-shot Learning for Automatic Phonemic Transcription

   Xinjian Li, Siddharth Dalmia, David R. Mortensen, and 3 more authors

   In Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, Nov 2020
3. Universal Phone Recognition with a Multilingual Allophone System

   Xinjian Li, Siddharth Dalmia, Juncheng Li, and 8 more authors

   In ICASSP 2020, Nov 2020
4. Computerized Forward Reconstruction for Analysis in Diachronic Phonology, and Latin to French Reflex Prediction

   Clayton Marr, and David R. Mortensen

   In 1st Workshop on Language Technologies for Historical and Ancient LAnguages (LT4HALA), Nov 2020
5. Characterizing Sociolinguistic Variation in the Competing Vaccination Communities

   Shahan Ali Memon, Aman Tyagi, David R. Mortensen, and 1 more author

   In Proceedings of the International Conference SBP-BRiMS 2020, Nov 2020
6. AlloVera: A Multilingual Allophone Database

   David R. Mortensen, Xinjian Li, Patrick Littell, and 6 more authors

   In Proceedings of the Twelfth International Conference on Language Resources and Evaluation (LREC 2020), Nov 2020
7. Where New Words Are Born: Distributional Semantic Analysis of Neologisms and Their Semantic Neighborhoods

   Maria Ryskina, Ella Rabinovich, Taylor Berg-Kirkpatrick, and 2 more authors

   In Proceedings of the Society for Computation in Linguistics, Nov 2020

2019

1. CMU-01 at the SIGMORPHON 2019 Shared Task on Crosslinguality and Context in Morphology

   Aditi Chaudhary, Elizabeth Salesky, Gayatri Bhat, and 3 more authors

   In Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology, Aug 2019

   Abs

   This paper presents the submission by the CMU-01 team to the SIGMORPHON 2019 task 2 of Morphological Analysis and Lemmatization in Context. This task requires us to produce the lemma and morpho-syntactic description of each token in a sequence, for 107 treebanks. We approach this task with a hierarchical neural conditional random field (CRF) model which predicts each coarse-grained feature (eg. POS, Case, etc.) independently. However, most treebanks are under-resourced, thus making it challenging to train deep neural models for them. Hence, we propose a multi-lingual transfer training regime where we transfer from multiple related languages that share similar typology.

2. Hmong (Mong Leng)

   David R. Mortensen

   In The Mainland Southeast Asia Linguistic Area, Aug 2019
3. IndoMorph

   David R. Mortensen, and Jong Hyuk Park

   Aug 2019

   Collection of Foma FST morphological analyzers for languages of the Indian subcontinent

2018

1. Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations

   Aditi Chaudhary, Chunting Zhou, Lori Levin, and 3 more authors

   In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Oct 2018

   Abs

   Much work in Natural Language Processing (NLP) has been for resource-rich languages, making generalization to new, less-resourced languages challenging. We present two approaches for improving generalization to low-resourced languages by adapting continuous word representations using linguistically motivated subword units: phonemes, morphemes and graphemes. Our method requires neither parallel corpora nor bilingual dictionaries and provides a significant gain in performance over previous methods relying on these resources. We demonstrate the effectiveness of our approaches on Named Entity Recognition for four languages, namely Uyghur, Turkish, Bengali and Hindi, of which Uyghur and Bengali are low resource languages, and also perform experiments on Machine Translation. Exploiting subwords with transfer learning gives us a boost of +15.2 NER F1 for Uyghur and +9.7 F1 for Bengali. We also show improvements in the monolingual setting where we achieve (avg.) +3 F1 and (avg.) +1.35 BLEU.

2. The ARIEL-CMU situation frame detection pipeline for LoReHLT16: a model translation approach

   Patrick Littell, Tian Tian, Ruochen Xu, and 12 more authors

   Machine Translation, Oct 2018
3. Parser combinators for Tigrinya and Oromo morphology

   Patrick Littell, Tom McCoy, Na-Rae Han, and 5 more authors

   In Proceedings of the 11th Language Resources and Evaluation Conference, May 2018
4. Epitran: Precision G2P for Many Languages

   David R. Mortensen, Siddharth Dalmia, and Patrick Littell

   In Proceedings of the 11th Language Resources and Evaluation Conference, May 2018
5. Epitran

   David R. Mortensen

   May 2018

   Precision orthography-to-IPA conversion for 65 languages
6. MStem

   David R. Mortensen

   May 2018

   Python multilingual morphological stemming framework and stemmer collection

2017

1. URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors

   Patrick Littell, David R. Mortensen, Ke Lin, and 3 more authors

   In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, Apr 2017

   Abs

   We introduce the URIEL knowledge base for massively multilingual NLP and thelang2vec utility, which provides information-rich vector identifications oflanguages drawn from typological, geographical, and phylogenetic databases andnormalized to have straightforward and consistent formats, naming, andsemantics. The goal of URIEL and lang2vec is to enable multilingual NLP,especially on less-resourced languages and make possible types of experiments(especially but not exclusively related to NLP tasks) that are otherwisedifficult or impossible due to the sparsity and incommensurability of the datasources. lang2vec vectors have been shown to reduce perplexity in multilinguallanguage modeling, when compared to one-hot language identification vectors.

2. Hmong-Mien Languages

   David R. Mortensen

   In Oxford Research Encyclopedia of Linguistics, May 2017
3. Hmong-Mien Languages

   David R. Mortensen

   In Oxford Bibliographies in Linguistics, May 2017

2016

1. Phonologically Aware Neural Model for Named Entity Recognition in Low Resource Transfer Settings

   Akash Bharadwaj, David Mortensen, Chris Dyer, and 1 more author

   In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Nov 2016
2. Bridge-Language Capitalization Inference in Western Iranian: Sorani, Kurmanji, Zazaki, and Tajik

   Patrick Littell, David R. Mortensen, Kartik Goyal, and 2 more authors

   In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), May 2016

   Abs

   In Sorani Kurdish, one of the most useful orthographic features in named-entity recognition – capitalization – is absent, as the language’s Perso-Arabic script does not make a distinction between uppercase and lowercase letters. We describe a system for deriving an inferred capitalization value from closely related languages by phonological similarity, and illustrate the system using several related Western Iranian languages.

3. Named Entity Recognition for Linguistic Rapid Response in Low-Resource Languages: Sorani Kurdish and Tajik

   Patrick Littell, Kartik Goyal, David R. Mortensen, and 3 more authors

   In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Dec 2016

   Abs

   This paper describes our construction of named-entity recognition (NER) systems in two Western Iranian languages, Sorani Kurdish and Tajik, as a part of a pilot study of “Linguistic Rapid Response” to potential emergency humanitarian relief situations. In the absence of large annotated corpora, parallel corpora, treebanks, bilingual lexica, etc., we found the following to be effective: exploiting distributional regularities in monolingual data, projecting information across closely related languages, and utilizing human linguist judgments. We show promising results on both a four-month exercise in Sorani and a two-day exercise in Tajik, achieved with minimal annotation costs.

4. PanPhon: A Resource for Mapping IPA Segments to Articulatory Feature Vectors

   David R. Mortensen, Patrick Littell, Akash Bharadwaj, and 3 more authors

   In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Dec 2016

   Abs

   This paper contributes to a growing body of evidence that—when coupled with appropriate machine-learning techniques–linguistically motivated, information-rich representations can outperform one-hot encodings of linguistic data. In particular, we show that phonological features outperform character-based models. PanPhon is a database relating over 5,000 IPA segments to 21 subsegmental articulatory features. We show that this database boosts performance in various NER-related tasks. Phonologically aware, neural CRF models built on PanPhon features are able to perform better on monolingual Spanish and Turkish NER tasks that character-based models. They have also been shown to work well in transfer models (as between Uzbek and Turkish). PanPhon features also contribute measurably to Orthography-to-IPA conversion tasks.

5. PanPhon

   David R. Mortensen

   Dec 2016

   Articulatory feature extractor and library
6. Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning

   Yulia Tsvetkov, Sunayana Sitaram, Manaal Faruqui, and 6 more authors

   In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Jun 2016

2013

1. Lexical Prefixes and Tibeto-Burman Laryngeal Contrasts

   David R. Mortensen

   In Proceedings of the Thirty-Seventh Annual Meeting of the Berkeley Linguistics Society (BLS 37), Jun 2013
2. A Reconstruction of Proto-Tangkhulic Rhymes

   David R. Mortensen, and James A. Miller

   Linguistics of the Tibeto-Burman Area , Jun 2013
3. Tonally Conditioned Vowel Raising in Shuijingping Mang

   David R. Mortensen

   Journal of East Asian Linguistics, Jun 2013

2012

1. Sorbung, an Undescribed Language of Manipur: Its Phonology and Place in Tibeto-Burman

   David R. Mortensen, and Jennifer Keogh

   Journal of the Southeast Asian Linguistics Society, Jun 2012
2. The Emergence of Obstruents after High Vowels

   David R. Mortensen

   Diachronica, Jun 2012
3. NetSPE

   David R. Mortensen

   Jun 2012

   A web-based application for exploring rule-based analyses of phonological data
4. A Classification of Compounding in American Sign Language: an Evaluation of the Bisetto and Scalise Framework

   Mary Lou Vercellotti, and David R. Mortensen

   Morphology, Jun 2012

2011

1. HsSPE

   David R. Mortensen

   Jun 2011

   Haskell library implementing SPE-style rule-based phonology
2. Web Comparator

   David R. Mortensen

   Jun 2011

   A web-based application for organizing and analyzing comparative lexical databases. Used in the production of “Emergence of Obstruents” and “Proto-Tangkhulic Rhymes”

2004

1. The emergence of dorsal stops after high vowels in Huishu

   David R. Mortensen

   In Proceedings of the Thirtieth Annual Meeting of the Berkeley Linguistics Society (BLS 30), Jun 2004

2003

1. Review of \emphBaheng Yu Yanjiu [research on the Pa-Hng language] by Mao Zongwu and Li Yunbing

   David R. Mortensen

   Linguistics of the Tibeto-Burman Area, Jun 2003

2002

1. Review of \emphLes langue Hmong-Mjen: phonologie historique by Barbara Niederer

   David R. Mortensen

   Linguistics of the Tibeto-Burman Area, Jun 2002