ACMD Seminar: Using Hamilton Jacobi PDEs in Optimization

Samy Wu Fung
Assistant Professor, Applied Mathematics and Statistics Dept., Colorado School of Mines

Tuesday, January 24, 2023, 1:00-2:00 PM MT (3:00-4:00 PM ET)

A video of this talk is available to NIST staff in the Math channel on NISTube, which is accessible from the NIST internal home page. It will be taken down from NISTube after 12 months at which point it can be requested by emailing the ACMD Seminar Chair.

Abstract: First-order optimization algorithms are widely used today. Two standard building blocks in these algorithms are proximal operators (proximals) and gradients. Although gradients can be computed for a wide array of functions, explicit proximal formulas are only known for limited classes of functions.  We provide an algorithm, HJ-Prox, for accurately approximating such proximals. This is derived from a collection of relations between proximals, Moreau envelopes, Hamilton-Jacobi (HJ) equations, heat equations, and importance sampling. In particular, HJ-Prox smoothly approximates the Moreau envelope and its gradient. The smoothness can be adjusted to act as a denoiser. Our approach applies even when functions are only accessible by (possibly noisy) blackbox samples. 

Our approach can also be embedded into a zero-order algorithm with guaranteed convergence to global minima, assuming continuity of the objective function; this is done by leveraging connections between the gradient of the Moreau envelope and the proximal operator. We show HJ-Prox is effective numerically via several examples. 

Bio: Samy is an Assistant Professor in the Department of Applied Mathematics and Statistics and the Department of Computer Science at Colorado School of Mines. Prior to joining Mines, he was an Assistant Adjunct Professor in the Department of Mathematics at UCLA. He received a PhD in applied mathematics from Emory University in 2019, where he worked under the guidance of Lars Ruthotto. Samy's research interests lie in the intersection of applied mathematics and data science. In particular, he is interested in inverse problems, optimization, optimal control, and deep learning.

Host: Zach Grey

Note: This talk will be recorded to provide access to NIST staff and associates who could not be present to the time of the seminar. The recording will be made available in the Math channel on NISTube, which is accessible only on the NIST internal network. This recording could be released to the public through a Freedom of Information Act (FOIA) request. Do not discuss or visually present any sensitive (CUI/PII/BII) material. Ensure that no inappropriate material or any minors are contained within the background of any recording. (To facilitate this, we request that cameras of attendees are muted except when asking questions.)

Note: Visitors from outside NIST must contact Lochi Orr (301) 975-3800; at least 24 hours in advance.