Stein-ML · Stein’s method and functional inequalities in machine learning

Stein’s method and functional inequalities in machine learning

Mathematical statistics is a ubiquitous tool in modern data analysis. However, in typical applications, theoretically supported statistical frameworks cannot be used directly - for instance because they lack closed-form solutions or are too computationally expensive. Because of this, approximate inference techniques have been developed in the recent years as a way to speed up the learning process of algorithms based on statistics. However, the quality of the associated approximations is still not well-understood. Researchers and practitioners therefore look for tools to measure both the efficiency of the associated inference and sampling methods and the size of the error they generate when using approximations. Indeed, such tools are still not widely available in the literature for many of the commonly used models, therefore leaving researchers and practitioners unable to assess whether their methods are simultaneously fast and robust enough for their purposes. At the same time, due to the broad availability of probabilistic programming languages, approximate inference has been applied to a wide variety of problems, related to medical imaging, detection of gravitational waves or, recently, modelling of infectious diseases. Underestimation of uncertainty or inaccuracy of point estimates in those applications could undermine the reliability of the associated research or the quality of measures introduced as a result of it. The aim of this research project was to advance the development of quality measures for approximations in machine learning and statistics, using the rich theoretical machinery of mathematical analysis and probability. The project has been concluded with five papers, each targeting a specific approximation. Specifically, together with collaborators, we have achieved the following goals. We have constructed fully computable quality guarantees for the Laplace approximation of a Bayesian posterior, with respect to a variety of useful divergences. We have also constructed a new functional-data goodness-of-fit test for Gaussian Process targets and measures absolutely continuous with respect to Gaussians. Moreover, we have provided a novel targeted accuracy diagnostic for distributional approximations. Furthermore, in another paper, we have proved a functional version of the celebrated de Jong Theorem describing the asymptotic behaviour of U-statistics. Finally, we have proved new Berry-Esseen bounds for vector-valued statistics of binomial processes, with respect to the convex distance.

Текст от CORDIS, на английски · Данни: CORDIS, © Европейски съюз