A Novel Quasi-Newton Method for Composite Convex Minimization

TY - JOUR

T1 - A Novel Quasi-Newton Method for Composite Convex Minimization

AU - Chai, W. H.

AU - Ho, S. S.

AU - Quek, H. C.

N1 - Funding Information: This work was conducted within the Rolls-Royce@NTU Corporate Lab with support from the National Research Foundation (NRF) Singapore un-der the Corp Lab@University Scheme and Energy Research Institute@NTU under Interdisciplinary Graduate School in Nanyang Technological University. Publisher Copyright: © 2021

PY - 2022/2

Y1 - 2022/2

N2 - A fast parallelable Jacobi iteration type optimization method for non-smooth convex composite optimization is presented. Traditional gradient-based techniques cannot solve the problem. Smooth approximate functions are attempted to be used as a replacement of those non-smooth terms without compromising the accuracy. Recently, proximal mapping concept has been introduced into this field. Techniques which utilize proximal average based proximal gradient have been used to solve the problem. The state-of-art methods only utilize first-order information of the smooth approximate function. We integrate both first and second-order techniques to use both first and second-order information to boost the convergence speed. A convergence rate with a lower bound of O([Formula presented]) is achieved by the proposed method and a super-linear convergence is enjoyed when there is proper second-order information. In experiments, the proposed method converges significantly better than the state of art methods which enjoy O([Formula presented]) convergence.

AB - A fast parallelable Jacobi iteration type optimization method for non-smooth convex composite optimization is presented. Traditional gradient-based techniques cannot solve the problem. Smooth approximate functions are attempted to be used as a replacement of those non-smooth terms without compromising the accuracy. Recently, proximal mapping concept has been introduced into this field. Techniques which utilize proximal average based proximal gradient have been used to solve the problem. The state-of-art methods only utilize first-order information of the smooth approximate function. We integrate both first and second-order techniques to use both first and second-order information to boost the convergence speed. A convergence rate with a lower bound of O([Formula presented]) is achieved by the proposed method and a super-linear convergence is enjoyed when there is proper second-order information. In experiments, the proposed method converges significantly better than the state of art methods which enjoy O([Formula presented]) convergence.

UR - http://www.scopus.com/inward/record.url?scp=85120963556&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85120963556&partnerID=8YFLogxK

U2 - https://doi.org/10.1016/j.patcog.2021.108281

DO - https://doi.org/10.1016/j.patcog.2021.108281

M3 - Article

SN - 0031-3203

VL - 122

JO - Pattern Recognition

JF - Pattern Recognition

M1 - 108281

ER -