Fast high-dimensional temporal point processes with applications

Abstract

Large sets of continuous-time discrete event streams are often in the focus of seismology, neuroscience, finance, behavioral science among other scientific and engi neering disciplines. In this work, we explore a set of novel models and algorithms to learn from such data at scale, in the presence of a large number of events and event types. First, we develop two algorithms for estimating high-dimensional multivari ate Hawkes processes with a low-rank parameterization. The first approach leverages a novel connection to nonnegative matrix factorization, which we use to propose a stochastic gradient descent algorithm. We then demonstrate, via a moment-based ap proach, that we can reduce the parameter estimation problem to a single low-rank approximation. Notably, both approaches require only a few scans of the data, feature well-known matrix decompositions as subroutines, and yield fast parameter estimation. We also propose global-local temporal point processes (TPP), multidimensional TPP models that model self- and mutual-excitation patterns at different scales of time. One such model, FastPoint, relies on deep recurrent neural networks to approximate the mutual excitation pattern, and results in several orders of magnitude faster learning. Global-local TPPs also allow for substantially faster sequential Monte Carlo sampling, greatly accelerating the current state of the art in simulating temporal point patterns. Finally, we propose a novel application area for TPPs, applying ideas from renewal processes and deep learning to intermittent demand forecasting. Our contributions aim to remove both of the main challenges—scalable learning and inference—facing the adoption of high-dimensional TPP models.