Sparsifying HMMs with a Renyi entropy prior

Sujeeth Bharadwaj, 11/1/2013, 2-3pm, BI 2369

The hidden Markov model (HMM) is a popular tool for tasks involving sequences. We show that sparsity in the HMM parameters, in addition to reducing model complexity, is especially useful for two different applications -- 1) clustering sequences, and 2) learning class-based language models. In both cases, we theoretically motivate the use of Renyi entropy as a prior and show that it can 1) directly maximize purity in the clustering problem, and 2) minimize perplexity of a class-based language model. Results on two different tasks -- 1) clustering non-speech acoustic events from the BBC sound effects corpus, and 2) learning a simple bigram model from the resource management (RM) corpus -- confirm that Renyi entropy priors are indeed useful, and are worth exploring for other HMM-based tasks.

[an error occurred while processing this directive]