SPGD: Steepest Perturbed Gradient Descent Optimization

Abstract

Abstract. Optimization algorithms are pivotal in advancing various scientific and industrial fields but often encounter obstacles such as trapping in local minima, saddle points, and plateaus (flat regions), which makes the convergence to reasonable or near-optimal solutions particularly challenging. This article presents the steepest perturbed gradient descent (SPGD), a novel algorithm that innovatively combines the principles of the gradient descent method with periodic uniform perturbation sampling to effectively circumvent these impediments and lead to better solutions whenever possible. SPGD is distinctively designed to generate a set of candidate solutions and select the one exhibiting the steepest loss difference relative to the current solution. It enhances the traditional gradient descent approach by integrating a strategic exploration mechanism that significantly increases the likelihood of escaping suboptimal local minima and navigating complex optimization landscapes effectively. Our approach not only retains the directed efficiency of gradient descent but also leverages the exploratory benefits of stochastic perturbations, thus enabling a more comprehensive search for global optima across diverse problem spaces. We demonstrate the efficacy of SPGD in solving the 3D component packing problem, an NP-hard challenge. Preliminary results show a substantial improvement over six established methods, particularly on response surfaces with complex topographies and in multidimensional nonconvex continuous optimization problems. Comparative analyses with established 2D benchmark functions over 30 randomized initial points highlight SPGD’s robustness and reliability in nonconvex optimization. These results emphasize SPGD’s potential as a versatile tool for a wide range of optimization problems.