Browsing by Author "Lode, Lauri"

Hamiltonian Monte Carlo is a powerful Markov Chain algorithm, which is able to traverse complex posterior distributions accurately. One of the method's disadvantages is it's reliance on gradient evaluations over the full data, which quickly becomes computationally costly when the data sets grow large. By mini-batching the data set for stochastic gradient approximations we can speed up the algorithm, albeit with a reduced posterior accuracy. We illustrate by using a toy example, that the stochastic version of the method is unable to explore the exact posterior, and we show how an added friction term greatly alleviates this, when the term is adjusted carefully. We use the added stochastic error to our advantage, by turning the results differentially private. The randomness in the results masks the appearance of any single data point in the used data set, creating a way to more secure handling of sensitive data. In the case of stochastic gradient Hamiltonian Monte Carlo, we are able to achieve reasonable privacy bounds with little to no decrease in optimization performance, although finding a good the differentially private approximation of the target posterior becomes harder. In addition, we compare the previously considered privacy accounting methods to assay the privacy bounds to a new privacy loss distribution method, which is able to determine a tighter privacy profile than, for example, the moments accountant method.