Generalized tensor factorization

Abstract

This thesis proposes a unified probabilistic framework for modelling multiway data. Our approach establishes a novel link between probabilistic graphical models and tensor factorization, that allows us to design arbitrary factorization models utilizing major class of the cost functions while retaining simplicity. Using an expectationmaximization (EM) optimization for maximizing the likelihood (ML) and maximizing the posterior (MAP) of the exponential dispersions models (EDM), we obtain generalized iterative update equations for beta divergence with Euclidean (EU), Kullback- Leibler (KL), and Itakura-Saito (IS) costs as special cases. We then cast the update equations into multiplicative update rules (MUR) and alternating least square (ALS for Euclidean cost) for arbitrary structures besides the well-known models such as CP (PARAFAC) and TUCKER3. We, then, address the model selection issue for any arbitrary non-negative tensor factorization model with KL error by lower bounding the marginal likelihood via a factorized variational Bayes approximation. The bound equations are generic in nature such that they are capable of computing the bound for any arbitrary tensor factorization model with and without missing values. In addition, further the EM, by bounding the step size of the Fisher Scoring iteration of the generalized linear models (GLM), we obtain general factor update equations for real data and multiplicative updates for non-negative data. We, then, extend the framework to address the coupled models where multiple observed tensors are factorized simultaneously. We illustrate the results on synthetic data as well as on a musical audio restoration problem.