This WP will aim to overcome the computational bottleneck of training DNNs. DL training involves huge-scale nonconvex optimisation problems. DNNs have relatively simple structures, which most existing algorithms ignore. The main outcome of this WP is efficient and provably convergent augmented Lagrangian based algorithms for DL training, including more general saddle-point type DL models.