decorators.ipynb

Kernel: Python 3

Table of Contents

• 1  Decorators
  • 1.1  Decorators Example
  • 1.2  Practical Use Cases
• 2  Reference

# code for loading the format for the notebook
import os

# path : store the current path to convert back to it later
path = os.getcwd()
os.chdir(os.path.join('..', '..', 'notebook_format'))

from formats import load_style
load_style(plot_style = False)

os.chdir(path)

# magic to print version
%load_ext watermark
%watermark -a 'Ethen' -d -t -v

Ethen 2017-09-06 08:25:08 

CPython 3.5.2
IPython 6.1.0

Decorators

Basic concepts:

• In python, functions are First class functions allow us to treat functions like any other object, so for example: we can pass functions as arguments to another function, return functions and we can assign functions to variables.

• Closures allows us to take advantage of First class functions that returns an inner function that remembers and has access to variables local to the scope in which they were created.

More detailed description in the following link. Youtube: Programming Terms: First-Class Functions.

# we can assign functions to variables
def greet(name) :
    return 'Hello ' + name

greet_someone = greet
print(greet_someone('John'))


# we can define functions inside other functions
def greet(name) :
    def get_message() :
        return 'Hello '

    result = get_message() + name
    return result

print(greet('John'))


# functions can be passed as parameters to other functions
def greet(name):
    return 'Hello ' + name 

def call_func(func):
    other_name = 'John'
    return func(other_name)  

print(call_func(greet))


# functions can return other functions 
def compose_greet_func():
    def get_message():
        return 'Hello John'

    return get_message

greet = compose_greet_func()
print(greet())


# scoping, access the inner functions.
# note that python only allows read access to the outer scope and not assignment
def compose_greet_func(name):
    def get_message():
        return 'Hello ' + name

    return get_message

greet = compose_greet_func('John')
print(greet())

Hello John
Hello John
Hello John
Hello John
Hello John

Decorators Example

A decorator is basically a function that takes another function as an argument, adds some kind of functionality and then returns another function. So why would we want to do something like this? Well, it's because this allows us to easily add or alter the functionality to our existing function method or class without having to directly use its subclasses. In short, decorators are simply wrappers to existing functions.

In the example below, the p_decorate function takes another function as an argument and generates a new function which augments the work of the original function, and returning the generated function so we can use it anywhere else.

def get_text(name):
    return 'Hello {}'.format(name)

def p_decorate(func):
    def func_wrapper(name):
        return '<p>{0}</p>'.format(func(name))
    
    return func_wrapper

# pass in the function to the decorator function,
# and note that the decorator function returns
# the wrapper function that's waiting to be executed
my_get_text = p_decorate(get_text)

print(my_get_text('John'))

<p>Hello John</p>

And in python, there's neat shortcut for that, which is to mention the name of the decorating function before the function to be decorated. and perpend the decorator with an @ symbol.

So instead of calling p_decorate(get_text). It becomes:

@p_decorate
def get_text(name):
    return 'Hello {}'.format(name)

# adding the decorator alters the functionality of 
# `get_text` without having to change any piece of code
# in `get_text`
print(get_text('John'))

<p>Hello John</p>

Using multiple decorators. Note the ordering matters.

def get_text(name):
    return 'Hello {}'.format(name)

def p_decorate(func):
    def func_wrapper(name):
        return '<p>{0}</p>'.format(func(name))
    
    return func_wrapper

def strong_decorate(func):
    def func_wrapper(name):
        return '<strong>{0}</strong>'.format(func(name))
    
    return func_wrapper

def div_decorate(func):
    def func_wrapper(name):
        return '<div>{0}</div>'.format(func(name))
    
    return func_wrapper


# the original way 
get_text1 = div_decorate(p_decorate(strong_decorate(get_text)))
print(get_text1('John'))


# the decorator way
@div_decorate
@p_decorate
@strong_decorate
def get_text(name):
    return 'Hello {}'.format(name)

print(get_text('John'))

<div><p><strong>Hello John</strong></p></div>
<div><p><strong>Hello John</strong></p></div>

We can build decorators for class's methods in a similar fashion. We can do this by putting *args and **kwargs as parameters for the wrapper, then it can accept any arbitrary number of arguments and keyword arguments. This approach will make our decorators work for both functions and methods alike.

# You can also use it with class's methods.
def p_decorate(func):
    def func_wrapper(*args, **kwargs):
        return '<p>{0}</p>'.format(func(*args, **kwargs))
    
    return func_wrapper


class Person:
    def __init__(self):
        self.name = 'John'
        self.family = 'Doe'

    @p_decorate
    def get_fullname(self):
        return self.name + ' ' + self.family


my_person = Person()
print(my_person.get_fullname())

<p>John Doe</p>

# Another example of using *args and **kwargs when working the function
def decorator_function1(original_function):
    def wrapper_function(*args, **kwargs):
        print('wrapper executed this before {}'.format(original_function.__name__))
        return original_function(*args, **kwargs)
    return wrapper_function


@decorator_function1
def display_info(name, age):
    print('display info ran with arguments ({}, {})'.format(name, age))
    

display_info('John', 25)

wrapper executed this before display_info
display info ran with arguments (John, 25)

Looking back at the example above, you can notice how redundant the decorators in the example are. 3 decorators, div_decorate, p_decorate, strong_decorate each with the same functionality but wrapping the string with different tags. We can definitely do much better than that. Why not have a more general implementation for one that takes the tag to wrap with as a argument? Yes please!

def tags(tag_name):
    def tags_decorator(func):
        def func_wrapper(name):
            return '<{0}>{1}</{0}>'.format(tag_name, func(name))
        
        return func_wrapper
    
    return tags_decorator


@tags('p')
def get_text(name):
    return 'Hello ' + name

print(get_text('John'))

<p>Hello John</p>

In this case, our decorators expect to receive an additional argument, that is why we will have to build a function that takes those extra arguments and generate our decorator on the fly.

At the end of the day decorators are just wrapping our functions, in case of debugging that can be problematic since the wrapper function does not carry the name, module and docstring of the original function.

# it prints out the name of the decorator instead 
# of the original function because it got overridden
print(get_text.__name__)

func_wrapper

Fortunately, python includes the functools module which contains wraps. Wraps is a decorator for updating the attributes of the wrapping function(func_wrapper) to those of the original function(get_text). This is as simple as decorating func_wrapper by @wraps(func). Here is the updated example:

from functools import wraps


def tags(tag_name):
    def tags_decorator(func):      
        
        @wraps(func)
        def func_wrapper(name):
            return '<{0}>{1}</{0}>'.format(tag_name, func(name))
        
        return func_wrapper
    
    return tags_decorator


@tags('p')
def get_text(name):
    """returns some text"""
    return 'Hello ' + name

print(get_text('John'))
print(get_text.__name__)  # get_text
print(get_text.__doc__ )  # returns some text

<p>Hello John</p>
get_text
returns some text

Practical Use Cases

Adding timing and logging to functions.

import time
import logging


def logger(func):
    """
    create logging for the function,
    re-define the format to add specific logging time
    """
    @wraps(func)
    def wrapper(*args, **kwargs):
        logging.basicConfig(
            filename = '{}.log'.format( func.__name__ ),
            format = '%(asctime)s -- %(levelname)s:%(name)s: %(message)s',
            datefmt = '%Y/%m/%d-%H:%M:%S',
            level = logging.INFO)
        
        # custom the logging information
        logging.info('Ran with args: {} and kwargs: {}'.format(args, kwargs))
        return func(*args, **kwargs)

    return wrapper


def timer(func):
    """time the running time of the passed in function"""
    @wraps(func)
    def wrapper(*args, **kwargs):
        t1 = time.time()
        result = func(*args, **kwargs)
        t2 = time.time() - t1
        print('{} ran in: {} sec'.format(func.__name__, t2))
        return result
    
    return wrapper

@timer
def display_info(name, age):
    time.sleep(1)  # manually add a second to the timing
    print('display info ran with arguments ({}, {})'.format(name, age))

display_info('John', 25)

display info ran with arguments (John, 25)
display_info ran in: 1.002450942993164 sec

# http://stackoverflow.com/questions/18786912/get-output-from-the-logging-module-in-ipython-notebook
# ipython notebook already call basicConfig somewhere, thus reload the logging
from importlib import reload
reload(logging)


@logger
def display_info(name, age):
    print('display info ran with arguments ({}, {})'.format(name, age))
    

display_info('John', 25)

Both time and log the function

@logger
@timer
def display_info(name, age):
    time.sleep(1) # manually add a second to the timing
    print('display info ran with arguments ({}, {})'.format(name, age))
    
display_info('Tom', 22)

display info ran with arguments (Tom, 22)
display_info ran in: 1.0016038417816162 sec

# good idea to add @wraps to your decorator for debugging
display_info.__name__

'display_info'

The good thing about writing the timing and logging function as decorators is that we can use them without having the re-write the logic every time we wish to use timing or logging functionality.

Reference

• Blog: Guide to python function decorators

• Youtube: Decorators - Dynamically Alter The Functionality Of Your Functions