Statistical Speech Technology Group

University of Illinois

Phoneme/phone discovery in under-resourced languages:

In this paper, we showed that it's possible to define phonemes/phones by clustering the hidden units in an end-to-end speech recognizer. This paper showed that it's also possible to use voice conversion to separate the content embedding from speaker ID. In this paper, we showed that it's possible to automatically generate spoken descriptions of images in an unwritten language. We're now trying to combine these three ideas, using cycle-consistent speech recognition and synthesis to identify phonemes/phones in under-resourced languages.

Here's a reasonably accurate article about the problem of under-resourced languages, by Wired Magazine.

Real-time dynamics of infant-parent interaction:

LittleBeats provides a remote, unobtrusive method to simultaneously assess physiological (ECG) and behavioral (audio, motion) streams of data on a large scale among young children and their caregivers in their home environments. We're using LittleBeats to study the development of mother-child relationships. Our first publication on this work is here.