Model selection for relational data factorization models

Abstract

For relational data factorization, generative models provide a statistically principled approach that allows for extending the factorization task in the probabilistic framework of Bayesian statistics. The most well-known example of such models is the stochastic blockmodel which is a mixture of Bernoullis defineci for relational data. In this work, we propose the model BAM-MMSB which replicates the generative process of the mixed-membership stochastic block model (MMSB) within the generic allocation framework of Bayesian allocation model (BAM). In contrast to traditional blockmodels, BAM-MMSB considers the observations as Poisson counts generated by a base Poisson process and marked according to the generative process of MMSB. A considerable amount of algorithms have been proposed to factorize relational data. However, model selection for this task is still an open problem. In the sequel, we estimate the optimal number of communities for BAM-MMSB by computing the variational approximations of the marginal likelihood for each model order. Although we only perform the model order selection task in our work, we believe that the generic allocation perspective of BAM promises a generalized model selection solution where we not only select the model order but also choose the best factorization. We describe the proposed model and derive the inference algorithms. Next, we display the experimental setup where we represent relational data as Poisson counts of the allocation model. La ter, we assess our variational inference algorithm in terms of interpretability of the model output and block recovery and model selection performance by the experiments on synthetic and real-world datasets.