Convex Optimization of Deep Polynomial and ReLU Activation Neural Networks
Burak Bartan (Stanford University); Mert Pilanci (Stanford University)
-
Members: FreeSPS
IEEE Members: $11.00
Non-members: $15.00
07 Jun 2023
We consider training multi-layer neural networks with polynomial and ReLU activation functions. We develop exact convex optimization formulations for three-layer and deeper architectures. Our formulations are based on semidefinite lifting and recent results on the hidden convexity of two-layer ReLU networks. We show that certain deep neural network architectures can be trained to global optimality in polynomial time by solving equivalent convex optimization problems.
