Rootfinding and optimization.ipynb

Kernel: Python 3 (Ubuntu Linux)

Rootfinding and optimization

In many applications we would like to find a root or minimum of a function in one or more variables. We will learn more about these methods later on, but will show some black-box methods in Python here.

# import scipy module
import scipy.optimize as opt

Rootfinding in one variable

We want to find $x\in\mathbb{R}$ for which $f(x) = 0$. Two well-known methods are Bisection and Newton's method.

# define function
f = lambda x : x**2 - 2

# bisection
root = opt.bisect(f,0,2)
print('found root : ', root)

# Newton
root = opt.newton(f,0)
print('found root : ', root)

found root :  1.4142135623715149
found root :  1.414213562373095

Exercise

Try both methods on the function $x^2$, what is going on?

Exercise

Find all roots of the functions $3.1078 - 8.09404 x + 3.25688 x^2 + x^3$ and $-1.61051 + 7.3205 x - 13.31 x^2 + 12.1 x^3 - 5.5 x^4 + x^5$.

Optimization in one variable

Findig the minimum of a function in one variable is a common task. Various methods are available via a common interface:

f = lambda x : x**2 - 2*x + 5

xm = opt.minimize_scalar(f)
print(xm)

     fun: 4.0
    nfev: 5
     nit: 4
 success: True
       x: 0.99999998519

Exercise

Check out the documentation of minimize_scalar and try different methods for finding the minimum of $(x-1)^8$. Which one works best?

Optimization in multiple variables

# Define quadratic function
f  = lambda x : (2*x[0] + x[1] - 2)**2 + (5*x[0] - x[1] + 3)**2

# find minimum using Nelder-Mead method
xm = opt.minimize(f,[0,0],method='Nelder-Mead')
print(xm)

 final_simplex: (array([[-0.14285386,  2.28575115],
       [-0.14286149,  2.28567636],
       [-0.14284659,  2.28569444]]), array([2.30419251e-09, 2.43575720e-09, 5.27603680e-09]))
           fun: 2.304192514827093e-09
       message: 'Optimization terminated successfully.'
          nfev: 136
           nit: 71
        status: 0
       success: True
             x: array([-0.14285386,  2.28575115])

Exercise

Find the minimum of the function in the example above by hand and verify the solution found by the algorithm. Hint: at a minimum of $f(x,y) = 0$ we have $\partial_x f(x,y) = \partial_y f(x,y) = 0$.