Illustration of Newton's method for minimizing a 1d function. (a) The solid curve is the function $\mathcal{L}(x)$. The dotted line $\mathcal{L}_{\mathrm{quad}}(\theta )$ is its second order approximation at $\theta _t$. The Newton step $d_t$ is what must be added to $\theta _t$ to get to the minimum of $\mathcal{L}_{\mathrm{quad}}(\theta )$. Adapted from Figure 13.4 of \citep{Vandenberghe06}. Generated by newtonsMethodMinQuad.ipynb . (b) Illustration of Newton's method applied to a nonconvex function. We fit a quadratic function around the current point $\theta _t$ and move to its stationary point, $\theta _{t+1} = \theta _t+ d_t$. Unfortunately, this takes us near a local maximum of $f$, not minimum. This means we need to be careful about the extent of our quadratic approximation. Adapted from Figure 13.11 of \citep{Vandenberghe06}. Generated by newtonsMethodNonConvex.ipynb .