GitHub Repository: gmolveau/python_full_course
Path: blob/master/python_full_course.ipynb
³⁰⁵ views

Kernel: venv

Python Full Course

Basics

History of Python

Python was conceived in the late 1980s by Guido van Rossum. Guido was considered a Benevolent dictator for life (BDFL).

Python 3.0 final was released on December 3rd, 2008.

So please use python3 😉 #nomorepython2

Its name is based on the "Monty Python".

What is Python

Python is :

an interpreted language (in opposition to a compiled language)
source code is converted into bytecode that is then executed
strongly typed
can't perform operations inappropriate to the type, eg. in Python you can't add a number typed variable with a string typed variable
- Wat Gary Bernhardt JS Takj 2012 - youtube.com
dynamically typed
you can write a = 2 and python will know that a is an Integer
variables, parameters, and return values of a function can be any type. Also, the types of variables can change while the program runs.
dynamic typing makes it easy to cause unexpected errors that you can only discover until the program runs (runtime).
- note : but python can now be statically typed through type hinting
see also
garbage collected

What does Python looks like ?

An example from netflix.

Why choose python then ?

/!\ opinions - be careful /!\

sweet syntax
good standard library
big community
various profesionnal applications (IT, space, maths, IA, games...)
does the job

"Our clients"

Google / Youtube
Vine (#rip)
Instagram
Reddit
Spotify
Netflix
NASA
Amazon
Every math teacher ever

ZEN of python

A collection of 19 principles that influence the conception and usage of the language.

>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

CPython

CPython ? What is that ?

CPython is the reference implementation of the Python programming language. Written in C and Python, CPython is the default and most widely used implementation of the Python language.

CPython can be defined as both an interpreter and a compiler as it compiles Python code into bytecode before interpreting it.

mise-en-jambe

écrire un programme pseudo.py qui vous demande votre pseudo et l'affiche verticalement

exemple :

$ python3 pseudo.py
Entrez votre pseudo: greg
g
r
e
g

correction

pseudo = input("please enter your pseudo ?\n")
for l in pseudo:
    print(l)

Comments

Single line comments start with a # : # this is a comment

Multiline comments can be written using triple single-quotes ''' or double-quotes """.

We usually use triple-quotes """ to write documentation, also called docstrings.

In [ ]:

# a single line comment
username = "greg" # an in-line comment
print(username)

''' ok so this is
    a multiline comment
'''

""" and this is
    another one
    dj khaled
"""

In [ ]:

def print_it(text:str) -> None:
    """
    print_it prints the given text

    Args:
        text (str): a text to be printed
    """
    print(text)

doc = """a very long string with "double quotes" and 'single' quotes in it"""
print_it(doc)

Variables

You don't need to declare a variable before using it, nor its type.

A variable stores a piece of data, and gives it a specific name.

username = "greg"

Memory management

Naming

The name of a variable should use the snake_case convention.

# wrong
allTheHashes = ['abcedf123' ...]
# right
all_hashes = ['abcedf123' ...]

Reading a variable name should give you a hint about its meaning.

# wrong
a = "https://myapi.heroku.com/api/v1/register"
# right
register_url = "https://myapi.heroku.com/api/v1/register"

You should not put the type of the variable in its name.

# wrong
list_of_username = ["greg", "alex"]
# right
usernames = ["greg", "alex"]

Finally, you should NOT use reserved keywords for a variable name (eg. dict, from)

Primitive/native data types

/!\ even if it won't makes sense right now, remember that everything in python is an Object /!\

The most common types are :

numbers : integers ; floating point numbers (floats) ; complex
string
list
dict
tuple
boolean (bool) : True and False
- None, 0, and empty strings/lists/dicts/tuples all evaluate to False

but there's a lot of other special ones :

iterators
range
bytes ; bytearray ; memoryview
frozenset

remember when we said that everything in python is an object ? all these types are objects, represented by a class.

In [ ]:

b = True
print(type(b))
i = 1
print(type(i))

more about what's a class later 😉

Note : all native types are available here

Casting / converting a type to another one

a variable can be converted to another type, for example :

In [ ]:

num = "3"
print(type(num))
num = int(num)
print(type(num))

For custom types (classes that you designed), you will have to code the operations necessary to convert your class to another one.

Operators

Python comes with a lot of operators that you can use and combine.

You can use them with variables of the same type, and also with variables of different types.

But be careful, some operations don't exist and will raise an Exception and crash your app.

In [ ]:

a = 1
b = 2
d = a / b # a division always gives a float even if the division is perfect
print(type(d))

In [ ]:

print(3 // 2) # euclidean division rounds down the result of the division

In [ ]:

print(7 % 3) # modulo operator

In [ ]:

print(2 ** 6) # exponential operator (also called power)

In [ ]:

c = a + b
c += 1 # equivalent to c = c + 1 # not all languages have this notation
print(c)

In [ ]:

s = "username"
p = "greg"
print(s + ": " + p)

In [ ]:

o = p * 3
print(o)

In [ ]:

l = [1,2,3]
ll = l * 2
print(ll)

In [ ]:

lll = l + ll
print(lll)

In [ ]:

# boolean operators
print(a == b)
print(a != b)

In [ ]:

a = not True # negation operation
print(a)

In [ ]:

# python let's you chain operators nicely for example for range
1 == 1 and 2 == 2 or 2 != 3

In [ ]:

1 < 2 < 3

`is` keyword

is keyword can be a little confusing and can lead to error later if not properly used.

is checks that two variables refer to the same object (memory address).

== checks if two objects have the same value.

See the difference here ?

In [ ]:

a = [1, 2, 3, 4]
b = a # make `b` points to the same 'place' as `a`
b is a
print(id(a), "!=", id(b))

In [ ]:

b = [1, 2, 3, 4]  # re-use `b` to create a new list
b is a

even if the two lists have the same values, they dont have the same address

In [ ]:

b == a

We can dig deeper with the help of the id function, which returns the memory address of an object.

In [ ]:

print(id(a), '!=', id(b))

None

None is an object. Always use is to compare a variable to None.

Mutability

some sequences (tuples and strings for example) are immutable which means that you can't edit them after creation.

In [ ]:

s = "hello"
s[0] = "a"

In [ ]:

t = (1, 2, 3)
t[0] = 4

If you wan't to edit them, you need to reassign them.

In [ ]:

s = "hello"
s = s + " " + "greg"
print(s)

Type hinting

Python’s type hints (PEP 484) provide you with optional static typing to leverage the best of both static and dynamic typing.

In [ ]:

def add_three(i):
    res = i + 3
    return res

y = add_three(4)
print(y)
z = add_three("ok")

In [ ]:

def add_three(i:int) -> int:
    res = i + 3
    return res

y = add_three("ok")

Functions (intro)

functions vs procedure

A function returns a value and a procedure just executes commands.

The name function comes from math. It is used to calculate a value based on input.

A procedure is a set of commands which can be executed in order.

In most programming languages, even functions can have a set of commands. Hence the difference is only returning a value.

QUESTION : how can we declare a procedure in Python ?

we can't (lol). Procedures don't exist in Python. If a function doesn't have a return statement in Python, it returns None.

In [ ]:

def print_it(text: str):
    print(text)

res = print_it("test")
type(res)

Multi return

In C, you can only return one value. In python you can return multiple values.

But ... if the most popular implementation of Python is in C (remember CPython), how did they do it ?

Let's take a look at the following code :

In [3]:

def give_numbers():
    return "ok", 44, [3,4]

res = give_numbers()
type(res)

Out[3]:

tuple

In fact, when you return multiple values, in reality, a Tuple object is created (reminder: a tuple is like an immutable list object).

We can also choose to unpack those values in different variables if we need to.

In [4]:

def give_numbers():
    return 1, 2, 3, 4

a, b, c, d = give_numbers()
print(type(a))

a, *b, c = give_numbers()
print(type(b))

Out[4]:

<class 'int'>
<class 'list'>

declaration

A function is declared via the keywork def then the name of the function (follow the snake_case naming convention please), followed by its argument(s) between parentheses (they can be positional or keywords), eventually their types and the type of the return of the function.

Note : there is no correlation between the name of the variable inside the function and the name of the variable that you need to give during the function call.

The following concat function expects a string as an argument, this string will then be named text inside the function only.

In [ ]:

def concat(text: str, separator: str = "\n") -> str:
    end = text + separator
    return end

We can then call this function, by using positional arguments :

In [ ]:

lorem_ipsum = "lorem ipsum dolor sit amet"
res = concat(lorem_ipsum)

or we can call this function by naming its arguments :

In [ ]:

lorem_ipsum = "lorem ipsum dolor sit amet"
res = concat(text=lorem_ipsum, separator="x")
res = concat(separator="x", text=lorem_ipsum)

QUIZZ :

what would the call of the following function looks like ?

def analyze_twitter_account(username):
    verified = False
    timezone = “Europe/Paris”
    numtweets_yesterday = 50
    return verified, timezone, numtweets

correction

obama = "@obama"
a, b, c = analyze_twitter_account(obama)

Default arguments

A function can have multiple arguments with a default value already set.

This allows you to design your function both for casual usage and advanced usage.

In [ ]:

def print_and_concat(text: str, end:str = "!!!") -> str:
    a = 1
    a: int = get_a_random_number()
    newline = text + end
    print(newline)
    return newline

print_and_concat("hello")
print_and_concat(text="hello")
print_and_concat("hello", "???")
print_and_concat("hello", end="???")
print_and_concat(text="hello", end="???")

But you can't use keywords arguments any way you want.

In [ ]:

print_and_concat(text="hello", "???")

Be careful of default mutable arguments

Be careful when using default mutable arguments ...

Can you feel what's gonna be wrong with the follo code ?

visualize memory here : https://pythontutor.com/render.html#code=def oops(l %3D [])%3A l.append("ok") return l print(oops()) print(oops())&cumulative=true&curInstr=11&heapPrimitives=true&mode=display&origin=opt-frontend.js&py=3&rawInputLstJSON=[]&textReferences=false

In [5]:

def oops(text, l = []):
    print(text)
    l.append("ok")
    return l

res = oops("test")
print(res)

Out[5]:

test
['ok']

In [6]:

abc = oops("test")
print(abc)

Out[6]:

test
['ok', 'ok']

QUIZZ : why is this code wrong ?

because the default argument is mutated anytime that value is mutated !

You can prevent this behaviour by setting the default argument to None.

In [ ]:

def oops(text, l : None | list[str] = None):
    print(text)
    if l is None:
        l = []
    l.append("ok")
    return l

res1 = oops("test")
print(res)
res2 = oops("test")
print(res)

test
['ok']
test
['ok']

In [ ]:

def oops(text, l = None):
    print(text)
    l = l or [] # be aware that this will also work if l == False and l == 0
    l.append("ok")
    return l

mal = []
a = oops("ok", mal)

This feature of Python is so nasty that the PEP 505 was submitted to create a new operator ??=.

This operator would replace a if ... is None.

def oops(text, l = None):
    print(text)
    l ??= []
    l.append("ok")
    return l

Fun debugging times :<

Signature

The signature of a function is the list of its arguments and their types + the return type.

Here, the function f takes one argument that is a string and returns None. The argument will be named text in the function.

QUIZZ : find the signature of the built-in `print` function

use the help function :

>>> help(print)
Help on built-in function print in module builtins:

print(...)
    print(value, ..., sep=' ', end='\n', file=sys.stdout, flush=False)

    Prints the values to a stream, or to sys.stdout by default.
    Optional keyword arguments:
    file:  a file-like object (stream); defaults to the current sys.stdout.
    sep:   string inserted between values, default a space.
    end:   string appended after the last value, default a newline.
    flush: whether to forcibly flush the stream.

LGI

When Python encounters a variable, it first looks if this variable is local to the function.

def print_it(text:str):
    res = text + "\n"
    return res

In this example, when the line res = text + "\n" will be interpreted, python will try to find the text variable. Here text is defined in the function so it is a local variable.

If this variable is not found locally, python will then look for it globally.

lines = []
def add_to_lines(text:str):
    lines.append(text)

Here when the lines.append(text) will be interpreted, the lines variable won't be found locally, but is defined before the definition of the function and so is accessible inside the function. lines is a defined as a global variable in this case.

Finally, if a variable is not local nor global, python will search internally if it is one of its own variable.

Good Naming

Imagine that you're reading this line of code :

ts("obama", False, 20, True)

What about this ?

# we query twitter to get the last 20 tweets of obama without retweets, unicode encoded
ts("obama", False, 20, True)

Is the comment really necessary ?

How about this ?

twitter_search(username=‘@obama’, retweets=False, numtweets=20, unicode=True)

Much better 😃 All we had to do is rename the function and rename + use keywords arguments.

Your code shouldn't need comment to explain this kind of behaviour.

This kind of code modification is called a refactoring.

Famous Acronyms

TODO bouger cette partie

One of the many times those terms will be mentioned in this course.

DRY means Don't Repeat Yourself. Don't write the same stuff in multiple places, or you will have to keep them synchronized at every change.

YAGNI means You Aren't Gonna Need It. Complexity introduced in order to solve a possible future problem that you won't necessarily have.

KISS means Keep It Simple, Stupid (the last word is actually an insult, but yeah). Avoiding unnecessary complexity will make your system more robust.

Built-in functions

Python comes with built-in functions for various purposes 😃

Here's a shortlist :

dir(obj) returns an alphabetized list of the attributes
help(obj) returns the docstring
vars(obj) returns an object's (writable) attributes
range(int) builds a sequence (is in fact a class 😉 )
len(obj) returns the length of an object

Classes (intro)

wait, isn't it only the beginning of the course ? sorry again, we need to address this subject too 😃

A class is a way to encapsulate data (named fields or attributes) and functionalities (methods).

A class is a way to store a state and a behaviour.

An object is an instance of a class. For example a Car would be a class, and a peugeot_208 would be an object instance of the Car class.

The object is responsible for modifying its own internal states, the way an object refers to itself is via the keyword self.

Objects then interact with each others.

In [ ]:

class Phone:

    def __init__(self, ringtone: str) -> None:
        self.ringtone = ringtone

    def ring(self):
        print(self.ringtone)

iphone = Phone(ringtone="coucou")
iphone.ring()
print(iphone.ringtone)

In [16]:

class Malware:
    def __init__(self, name:str, c2_domain:str):
        # attributes / fields (state, data)
        self.name = name
        self.c2_domain = c2_domain

    # method (functionalities, behaviour)
    def run(self, host:str):
        print(f"{self.name} is poutring {host} ...")
        pass

In [17]:

emotet = Malware('emotet', "m1cr0s0ft.com")
stuxnet = Malware('stuxnet', "youtuberouge.com")

targets = ['sadserver.com', "esna.com"]
for target in targets:
    emotet.run(target)

Out[17]:

emotet is poutring sadserver.com ...
emotet is poutring esna.com ...

In [ ]:

dico = {
    "name": "greg",
    "age" :29
}
dico["name"]
dico["age"]
for key, value in dico.items():
    print(key, value)

for key in dico:
    print(key, dico[key])

for key in dico.keys():
    print(key, dico[key])

for value in dico.values():
    print(value)

print(dico)

name greg
age 29
name greg
age 29
name greg
age 29
{'name': 'greg', 'age': 29}

Magic methods

Also called dunder for double under methods (because they begin and end with 2 underscores _).

They're special methods that you can define to add "magic" to your classes.

Those methods are used for all kind of usage in Python.

Ever wondered how len([1,2,3]) worked ? Or why you can iterate on a dict ? How the with statement works like with open("f.txt") as f: ?

Let's dive in 😃

Here's some "famous" methods :

__repr__
__str__
__len__
__eq__

(Note: a huge list can be found here in plain english)

In [26]:

# what happens if we print an object ?
emotet = Malware('emotet', "m1cr0s0ft.com")
print(emotet)

Out[26]:

<__main__.Malware object at 0x1184c8510>

string representation (repr + str) example

<__main__.Malware object at 0x113323df0> what a weird print, isn't it ?

if you're interested in knowing why this line is formated like that, here's the source code

Scenario : imagine that we are printing some logs to debug, is this information useful ? How could we improve it ?

We could find a way to add useful and critical infos in the string representation of an instance of the Malware class.

How can we change and improve the string representation then ?

Behind the scenes, the print function is in reality calling the __str__ method of the object. If this method is not defined, the __repr__ method is called.

What's the difference between repr and str ?

__repr__ goal is to be unambiguous, developer oriented, evaluable / REPRoduce
__str__ goal is to be readable, user friendly

In [27]:

# quick example

import datetime
now = datetime.datetime.now()
now = datetime.datetime(2023, 10, 11, 11, 51, 23, 30381)
print(now)

Out[27]:

2023-10-11 11:51:23.030381

In [29]:

repr(now) , now.__repr__()

Out[29]:

('datetime.datetime(2023, 10, 11, 11, 51, 23, 30381)',
 'datetime.datetime(2023, 10, 11, 11, 51, 23, 30381)')

Let's define the different string representations of our Malware class 😃

In [30]:

class MalwareVerbose:
    def __init__(self, name:str, domain:str):
        # attributes / fields (state, data)
        self.name = name
        self.domain = domain

    def __repr__(self):
        return f"MalwareVerbose({self.name!r}, {self.domain!r})"

    def __str__(self):
        return f"Malware: {self.name}"

    # method (functionalities, behaviour)
    def run(self, host:str):
        print(f"{self.name} is poutring {host} ...")
        pass

stuxnet = MalwareVerbose("stuxnet", "m1crosoft.com")

In [31]:

repr(stuxnet)

Out[31]:

"MalwareVerbose('stuxnet', 'm1crosoft.com')"

In [32]:

print(stuxnet)

Out[32]:

Malware: stuxnet

Functions are objects ?

wait... what ?!

Yes, you can assign an attribute to a function, let's check out its __dict__ to confirm that.

In [33]:

def one_function():
    pass

one_function.temp = 1

print(one_function.__dict__)

Out[33]:

{'temp': 1}

String interpolation

Python supports multiple ways to format text strings. (string interpolation, also named variable substitution)

String modulo operator

In [ ]:

# % interpolation
print("%s, %s!" %('Hello','world',))

name = 'world'
print("Hello, %s!" %(name))

The modulo operator uses format indicators (eg. %s for string).

A shortlist of format indicators :

%s: String (performed using the str() function)
%d: Integer
%f: Floating point
%e: Lowercase exponent
%E: Uppercase exponent
%x: Lowercase hexadecimal
%X: Uppercase hexadecimal
%o: Octal
%r: Raw (performed using the repr() function)
%g: Floating point for smaller numbers, lowercase exponent for larger numbers
%G: Floating point for smaller numbers, uppercase exponent for larger numbers
%a: ASCII (performed using the ascii() function)
%c: Converts an int or a char to a character, such as 65 to the letter A

Those format indicators can be combined with flags to modify the output.

In [ ]:

price = 2.2
print("%05d - %.2f" %(price, price))

In this example, the format indicator %05d, uses the 5 flag to indicate that the integers should be formatted with 5 spaces and the 0 flag is used to pad those with zeros. Since the d portion converts the floating-point numbers to integers, four 0 are placed in front of the integers. The format indicator %.2f indicates that the float should have only 2 decimals.

str.format() method

In [ ]:

s = "greg"
number_of_g = s.count("g")
print(number_of_g)

The string format method can also use format indicators and flags !

In [ ]:

price = 2.2
print("{:.3f}".format(price))

f-string

f-string or literal string interpolation was introduced in python 3.6 and is characterized by a new prefix f.

f-strings are really powerful. In fact, they are an expression evaluated at run time, not a constant value, making them quicker than others string interpolations.

In [ ]:

name = "world"
say = f"Hello, {name}!"
print(say)
print(f"{3 * 2}")
# escaping {}
print(f'My name is {{{name}}}')

In [34]:

quantity = 357568.12312
print(f"{quantity: >+20_.4f}")

Out[34]:

       +357_568.1231

Exercice : explain the result of the previous print :]

From the specs :

> is an align format specifier, right align
+ is a sign format specifier, indicates that the + and - sign should be printed
20 is the width reserved for printing the number
_ is a group format specifier, indicates the caracter that should be printed to separate every 3 numbers
.4 is a precision format specifier, here indicates that the number should have only 4 decimals
f is a type format specifier, here for a non scientific notation

In [ ]:

# others f-string magic
username = 'leo'
# expression
print(f"{2 * 2}")
print(f"user has a {'short' if len(username) < 5 else 'long'} username")

# function
def connect_status(username):
    return "connected"

log = f"user: {username} is {connect_status(username)}"
print(log)

# multiline print
print(
    f"""1
2
3"""
)

print(
    '1'
    '2'
    '3'
)

# using single and double quotes
print(f'''je fais ce 'que' je "veux" ok''')

# raw f-string
print(f'this is a not a phase \nmom')
print(fr'this is a not a phase \n mom')

# cool trick using the = operator
print(f"username={username}")
print(f"{username=}")

# the ! operator
face = "hmmm 🤔"
print(f"{face}")
print(f"{face!a}") # == convert to ascii
print(f"{face!r}") # == repr(face)

4
user has a short username
user: leo is connected
1
    2
    3
123
je fais ce 'que' je "veux" ok
this is a not a phase 
mom
this is a not a phase \n mom
username=leo
username='leo'
hmmm 🤔
'hmmm \U0001f914'
'hmmm 🤔'

limitations

If your format strings are provided by the user, use Template Strings to avoid security issues.
Can’t use backslashes to escape in the expression part of an f-string.
Expressions should not include comments using the # symbol

Lists

A list is a sequence object that stores an ordered sequence of objects, which can be of any types (even lists, yes).

In [39]:

random_stuff = ["lion", 1, False, ["o", "k"]]

In [40]:

print(random_stuff[0])
print(random_stuff[3])

Out[40]:

lion
['o', 'k']

Trying to access an invalid index results in an Exception IndexError being raised.

In [41]:

random_stuff[4]

Out[41]:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[41], line 1
----> 1 random_stuff[4]
IndexError: list index out of range

One of the under-rated feature of python is its negative index ! If you want to access the last element of a list without having to calculate its size, use the -1 index !

In [42]:

numbers = [1, 2, 3]
#    0  1  2   ---> positive index
#   -3 -2 -1   ---> negative index

numbers[::-1]

Out[42]:

[3, 2, 1]

also works for string : (but string are not lists, how does it work ? more on that later)

+---+---+---+---+---+---+
| P | y | t | h | o | n |
+---+---+---+---+---+---+
  0   1   2   3   4   5   6
 -6  -5  -4  -3  -2  -1

Slicing

Slicing is used to get a part (a slice) of a sequence. A lot of tricks can be done with slicing.

The syntax of a slice is as followed : [<start_index>:<stop_index>:<step>].

In [49]:

numbers = [1, 2, 3, 4]
print(numbers[0:3:1])
print(numbers[4:0:-1])

Out[49]:

[1, 2, 3]
[4, 3, 2]

In [50]:

# how NOT to copy a list
copy_of_numbers = numbers
print(copy_of_numbers)
numbers.append(5)
print(copy_of_numbers)

Out[50]:

[1, 2, 3, 4]
[1, 2, 3, 4, 5]

QUIZZ : why is this happening ?

TODO

pythontutor

In [51]:

# how to correctly copy a list
copy_of_numbers = numbers[:]
numbers.append(6)
print(copy_of_numbers)

Out[51]:

[1, 2, 3, 4, 5]

In [52]:

numbers = [1,2,[3,4,5]]
numbers2 = numbers[:]
print(numbers2)
numbers[2].append(6)
print(numbers2)

Out[52]:

[1, 2, [3, 4, 5]]
[1, 2, [3, 4, 5, 6]]

In [ ]:

reversed_numbers = numbers[::-1]
print(reversed_numbers)

In [ ]:

n = [1,2,3,[5,6]]
o = n.copy()
print(o, n)
n[3].append(8)
print(o, n)

Common list methods

we can enumerate all the methods of the list object by using the dir built-in function.

In [53]:

l = []
dir(l)

Out[53]:

['__add__',
 '__class__',
 '__class_getitem__',
 '__contains__',
 '__delattr__',
 '__delitem__',
 '__dir__',
 '__doc__',
 '__eq__',
 '__format__',
 '__ge__',
 '__getattribute__',
 '__getitem__',
 '__getstate__',
 '__gt__',
 '__hash__',
 '__iadd__',
 '__imul__',
 '__init__',
 '__init_subclass__',
 '__iter__',
 '__le__',
 '__len__',
 '__lt__',
 '__mul__',
 '__ne__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__reversed__',
 '__rmul__',
 '__setattr__',
 '__setitem__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 'append',
 'clear',
 'copy',
 'count',
 'extend',
 'index',
 'insert',
 'pop',
 'remove',
 'reverse',
 'sort']

But here's some commonly used :

.append(<value>) adds an element at the end of the list
.pop() removes the last element of the list and returns it
.remove(<value>) removes the first occurence of a value (or raises a ValueError if the value can't be found)
.insert(<index>, <value>) adds an element at the given index
.index(<value>) returns the index of the first occurence of a value (or raises a ValueError if the value can't be found)
.sort() modify the list to sort it ascendingly
.reverse() modify the list to put it in reverse order
.count(<value>) returns the number of occurence of a value in the list

In [ ]:

l = [1,2,3]
l.append(4)
print(l)
l.pop()
print(l)
l.remove(1)
print(l)
l.insert(0, 1)
print(l)
l.index(2)
l + [0]
print(l)
l.reverse()
print(l)
l.count(1)
l.sort()
print(l)

`list` built-in function

the list function converts an object to a list.

In [ ]:

msg = "hello"
l_msg = list(msg)
print(l_msg)

`in` keyword

The in keyword has two purposes:

check if a value is present in a sequence (list, range, string etc.), returns a Bool
iterate through a sequence in a for loop

In [57]:

# checking the presence of a value (present)
l = [1,2,3]
3 in l

Out[57]:

True

In [ ]:

msg = "hello"
"ll" in msg

In [ ]:

# checking the presence of a value (absent)
l = [1, 2, 3]
55 in l

In [ ]:

# iterate
l = [1, 2, 3]
for elem in l:
    print(elem)

In [ ]:

# NO
l = [1, 2, 3]
for i in range(len(l)):
    print(l[i])

# YES
for i, val in enumerate(l):
    print(i, val)

`len` built-in function

the len function returns the length of a sequence

In [ ]:

l = [1,2,3,4]
len(l)

`del` keyword

use the del keyword to delete an element with its index

In [ ]:

l = [1, 2 ,3]
del l[1]
print(l)

In [ ]:

del l[49]  # deleting an index that does not exist for this list raises an Exception

Common built-in functions used with lists

max(<list>) returns the maximum value in a list
min(<list>) returns the minimum value in a list

In [ ]:

l = [1, 22, 333333]
print(min(l))
print(max(l))

Strings

/!\ aparté /!\

speaking of text, do you use the best font FOR YOU ?

Please check codingfont.com to find out 😃

Pssss, mine is Roboto Mono 😉

a string can be seen as a list of characters (because it is a sequence) ! You can access each of its characters via their index. You can also loop on it.

In [ ]:

msg = "hello"
print(msg[0])

In [ ]:

for letter in msg:
    print(letter)

Two or more string literals (i.e. the ones enclosed between quotes) next to each other are automatically concatenated.

In [ ]:

s = 'Py' 'thon'
print(s)

In [ ]:

# This feature is particularly useful when you want to break long strings:
text = ('Put several strings within parentheses '
        'to have them joined together.')
print(text)

Common string methods

use dir to list the methods of this class

In [ ]:

dir(str)

<string>.join(<list>) creates a new string with each element of <list>, separated by <string>
.upper() returns the string entirely in ALL CAPS
.lower() returns the string entirely in lowercase
.capitalize() returns the string with a Uppercase at the beginning
.split(<separator>) splits the string using the separator and returns a list of string
.startswith(<value>) checks if the string starts with the value
.endswith(<value>) checks if the string ends with the value
.replace(<old>, <new>, [count=-1]) returns a copy of the string where the <old> part is replaced by the <new> part. By default, count is at -1 meaning that all occurences of <old> will be replaced. If you want only the first occurence of old to be replaced, then set count as 1
.find(<sub>) returns the first index where the sub string is found

In [61]:

s = ".".join(["hello", "world"])
print(s)
s = "hello"
print(s.upper())
print(s.lower())
print(s.capitalize())
sp = "127.0.0.1"
print(sp.split("."))
print(sp.startswith("127"))
print(sp.endswith("8"))
print(sp.find('0'))

Out[61]:

hello.world
HELLO
hello
Hello
['127', '0', '0', '1']
True
False
4

Ellipsis

... in python is called ellipsis. It is a string literal.

One of its main purpose is placeholding (just like pass). For example if you declare a class without attribute (like a custome Exception for example), you can use the ellipsis as a placeholder.

class CustomException:
    ...

class CustomException:
    pass

But the no-op instruction pass is prefered and much older than the ellipsis so use pass instead 😃

Tuples

Tuples are like lists but are immutable and are written with parenthesis.

In [ ]:

immut = (1, 2, 3)
type(immut)

In [ ]:

# if you want to create a tuple of one element, you need to add a comma after the element
immut = (1,)
len(immut)

In [62]:

# if we try to edit a tuple, we'll get an error
immut = (1, 2, 3)
immut[0] = 35

Out[62]:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[62], line 3
      1 # if we try to edit a tuple, we'll get an error
      2 immut = (1, 2, 3)
----> 3 immut[0] = 35
TypeError: 'tuple' object does not support item assignment

Really immutable ?

/!\ a tuple is immutable, but one of its item CAN be mutable

In [65]:

immut = (1, 2, [3, 4])
print(immut)
immut[2].append(5)
print(immut)

Out[65]:

(1, 2, [3, 4])
(1, 2, [3, 4, 5])

Unpacking

a sequence can be unpacked into variables.

Unpacking is a powerful tool !

In [ ]:

a, b, c = (1, 2, 3)
print(a, b ,c)

def give_numbers():
    return 4, 5, 6

d, e, f = give_numbers()
print(d, e ,f)

In [ ]:

# extended unpacking is defined by an asterisk
a, *b, c = (1, 2, 3, 4)
print(a, b, c)

In [ ]:

# you can also use unpacking to init some variables
d, e = 1, 2

# a neat trick with unpacking is to swap variables (commonly used in math algorithm implementation)
# other programming languages would have required temporary variables
e, d = d, e
print(d, e)

Dictionaries

A dict is a key/value datastructure, created by placing a sequence of elements within curly braces {}.

Keys of a dict needs to be immutable (int, float, string, tuple). But a dict is not immutable.

You can access an element using square brackets and the key. [<key>]

When the element is nested, combined multiple square brackets.

To add or edit an element, also use square brackets + the = assign operator.

In [ ]:

empty_dict = {}

In [ ]:

d = {"one": 1, "two": 2}
d["one"]
d["three"] = 3
print(d)

In [ ]:

d["four"] = ["rowenta", "bosch"]
print(d["four"][0])

In [ ]:

d["unknown"] # accessing an unknown key raises a KeyError exception

Common dict methods

In [ ]:

dir(dict)

.keys() returns a list of all the keys
.values() returns a list of all the values
.items() returns a list of tuples which are the couples ,
.get(<key>, [default]) returns an element for a key, if the key does not exist, None is returned, except if a default is provided
.setdefault(<key>, <value>) set a default value for a key if it's not already set
.update({"four":4}) merges two dictionaries

In [67]:

d = {"one": 1, "two": 2}

In [ ]:

d.keys()

In [ ]:

d.values()

In [ ]:

d.items()

In [68]:

d.get("one")
d["blabla"]

Out[68]:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[68], line 2
      1 d.get("one")
----> 2 d["blabla"]
KeyError: 'blabla'

In [ ]:

print(d.get("three"))

In [ ]:

d.get("three", "default data 3")

In [ ]:

d.setdefault("three", 3) # "three" key is not configured, so the default value is set
d["three"]

In [ ]:

d.setdefault("three", "default 3") # "three" key is already configured
d["three"]

In [ ]:

d.update({"four":4})
d

`in` keyword

the in keyword can be used to check if an element is a key of a dict.

In [ ]:

d = {"one": 1, "two": 2}
"one" in d # is equivalent to "one" in d.keys()

`del` keyword

you can use the del keyword to delete a key in a dict.

trying to delete a key that doesn't exist raises a KeyError exception.

In [ ]:

d = {"one": 1, "two": 2}
del d["one"]
d

In [ ]:

del d["one"]

unpacking (again!)

Since python 3.5, you can now unpack elements in a dict, resulting in a merge or an addition !

In [ ]:

d = {'a': 1, **{'b': 2}} # addition because the key 'b' is not set
print(d)

In [ ]:

d = {'a': 1, **{'a': 2}} # merge because the key 'a' is already defined so its overwritten
print(d)

Sets

a set is a collection of unique elements. You can add multiple times the same element, but there will be only one occurence of this element in the set.

a set can be instancied with the function set or with curly braces (like a dict) {}.

a set can contain only immutable elements (strings, tuples, ints...) (like the keys of a dict). Trying to add a mutable item will result in a TypeError unhashable type 👀

In [70]:

empty_set = set()
s = {1,2,3,4}
s.add(1) # here we try to add 1 to the set
print(s)
# as we can see, 1 was already in the set so the addition didn't do anything

Out[70]:

{1, 2, 3, 4}

In [ ]:

names = ["greg", "julien", "julien"]
print(names)
names = list(set(names))
print(names)
names

['greg', 'julien', 'julien']
['greg', 'julien']

In [ ]:

empty_set = set()
empty_set.add([1,2,3,4])

Like a mathematical set, we can do mathematical operations such as intersection (&), union (|), difference (-) ...

In [72]:

s1 = {1,2,3,4}
s2 = {2,3}
s1 & s2 # intersection

Out[72]:

{2, 3}

In [73]:

s3 = {5,6}
s1 | s3 # union

Out[73]:

{1, 2, 3, 4, 5, 6}

In [74]:

{1, 2, 3, 4, 5, 6}
s1 - s2 # difference

Out[74]:

{1, 4}

`in` keyword

the in keyword can be used to check if an element exists in a set, and to iterate.

In [ ]:

unique_numbers = {1,2,3,4}
1 in unique_numbers

In [ ]:

5 in unique_numbers

In [ ]:

for number in unique_numbers: # iterating
    print(numbers)

pointers, pass-by-value, pass-by-reference

A little aparté to talk about how python variables are passed between functions.

Python is "pass-by-object-reference"

This is equivalent to “object references are passed by value".

The variable is not the object itself.

source : https://stackoverflow.com/a/986145

If you pass a mutable object into a method, the method gets a reference to that same object and you can mutate it as you want, but if you rebind the reference in the method, the outer scope will know nothing about it, and after you're done, the outer reference will still point at the original object.

If you pass an immutable object to a method, you still can't rebind the outer reference, and you can't even mutate the object.

Some examples of memory usage

Let's dig deeper. Take a look at the following examples and try to guess the answer

>>> a = [1,2,3]
>>> b = [1,2,3]
>>> a == b # ? True or False ?

answer

True

In [ ]:

a = [1,2,3]
b = [1,2,3]
# a == b # uncomment me for the solution

>>> a = [1,2,3]
>>> b = [1,2,3]
>>> a is b # ? True or False ?

answer

False

remember, the is keywork check if both sides are the same objects (if they have the same address)

we can verify this claim by using the id function

In [ ]:

a = [1,2,3]
b = [1,2,3]
# a is b # uncomment me for the solution

In [ ]:

print(id(a), id(b))

now what about this example :

>>> a = [1,2,3]
>>> b = a
>>> a == b # ? True or False ?
True
>>> a is b # ? True or False ?

answer

True

In [ ]:

a = [1,2,3]
b = a
# print(a == b) # uncomment me for the solution
# print(a is b) # uncomment me for the solution
# print(id(a), id(b)) # uncomment me for the solution

but what if we want b to be a copy of a and not a reference to it ? 😉

we can use the .copy() method (or use a trick with slicing)

In [ ]:

a = [1,2,3]
b = a.copy()
a is b
print(id(a), id(b))

In [ ]:

b = a[:]
a is b
print(id(a), id(b))

but be careful, with a "simple" copy like this, we don't clone the elements in the list, we just make new pointers (we can call them labels) to them.

if you modify the new list, elements in the old list will be affected too ! this is called a shallow copy.

Let's see two examples !

In [ ]:

# first example
a = [1,2,3,4]
b = a.copy()
print(b)
# now we edit b, lets change its first element
b[0] = "RIP"
print(b)
print(a)

In this first example, we see that editing b, the copy of a, doesn't change a. But in the second example ...

In [ ]:

a = [[1,2],[3,4]]
b = a.copy()
print(b)
# now we edit b, lets change its first element
b[0][0] = "RIP"
print(b)
print(a)

Do you know why this happens ?

the <source>.copy() method iterate over the <source> and create a new reference for each of its element (so only for the first dimension). This behaviour (problem?) arises when those elements are mutable, like lists ! Because if we mutate those lists in the copy, it is in fact the same objects that are being edited.

See this pythontutor example to better visualize !

If we really want to clone the entire list, we must create a deep copy with the copy module and its function deepcopy()

In [ ]:

import copy
a = [[1,2,3],[4,5,6],[7,8,9]]
b = copy.deepcopy(a)
b[0][0] = "RIP"
print(a)
print(b)

Here's another exercice (trap ?)

def reassign(l):
    l = [0, 1]

l = [1,2,3]
reassign(l)

What's the value of `some_numbers` ?

[1,2,3]

the variable some_numbers is passed by reference and linked to the l local variable. But this local variable l is reassigned to something else, so this doesn't affect some_numbers.

pythontutor explanation

In [75]:

def reassign(l):
    l = [0, 1]

l = [1,2,3]
reassign(l)

print(l)

Out[75]:

[1, 2, 3]

the previous example can be confusing because the local variable of the function has the same name as the other variable l.

we can rewrite it for more clarity.

In [ ]:

def reassign(local_l):
  local_l = [0, 1]

l = [1,2,3]
reassign(l)

Let's make sure it's understood 😃

def do_smth(local_l):
  local_l.append(4)

>>> l = [1,2,3]
>>> do_smth(l) # here we pass the `l` variable to the function `do_smth`
>>> print(l)

# what's the value of `l` now ?
# answer 1. [1, 2, 3, 4]
# answer 2. [1, 2, 3]

answer

you choosed answer 1, well done !
answer 2 ? yikes ! please read this chapter again 😃 (or drop an email)

In [ ]:

def do_smth(local_l):
  local_l.append(4)

l = [1,2,3]
do_smth(l)
print(l)

[1, 2, 3, 4]

`range`

range is a powerful function (actually its not a function but that's for another time), which can be used in various ways :

generate a sequence of numbers from 0 to <stop>-1 with range(<stop>)
generate a sequence of numbers from <start> to <stop> (with an optional <step>) with range(<start>, <stop>, [step])

(Want to see its source code ? https://github.com/python/cpython/blob/5fcfdd87c9b5066a581d3ccb4b2fede938f343ec/Objects/rangeobject.c#L76)

In [ ]:

list(range(0, 5)) # will generate a list of numbers from 0 to 4 (5-1)

In [ ]:

list(range(5, 10, 2)) # will generate a list of numbers from 5 to 9 (10-1) 2-by-2

In [ ]:

list(range(10, -1, -1)) # will generate a list of numbers from 10 to 0

In [ ]:

for i in range(10):
    print(i)

Want to see its C source code ?

conditions

TODO if / elif / else break continue

= < <=

1 < 3 < 5

loops

for

exercice 1 : create a program that prints this "graph" with n = 4 (number of lines).

****
***
**
*

solution exercice 1

n = 4
for i in range(n):
    print('*' * (n - i))

n = 4
for i in range(n, 0, -1):
    print('*' * i)

In [ ]:

# try your code here :)
n = 4
for i in range(n, 0, -1):
    print("*" * i)

for i in range(n+1):
    print('*' * (n - i))

exercice 2 : create a program that print this "pyramid" with height = 4.

   *         3 espaces 1 etoile ligne 0 hauteur 4
  ***        2 espaces 3 etoile ligne 1 hauteur 4
 *****       1 espace  5 etoile ligne 2 hauteur 4
*******      0 espace  7 etoile ligne 3 hauteur 4

In [ ]:

def print_pyramid(height: int):
    # code here plz
    pass

height = 4
print_pyramid(height)

solution exercice 2

def print_pyramid(height: int):
    ''' in this example for height = 4, we see that
       *    > 3 spaces, 1 stone (3 spaces optional)
      ***   > 2 spaces, 3 stones (2 spaces optional)
     *****  > 1 space, 5 stones (1 space optional)
    ******* > 0 space, 7 stones
    with current_height starting at 0
    we add 2 'stones' for each new line, so 2 * current_height + 1
    those stones have (height - current_height - 1) * spaces on the left
    '''
    for current_height in range(height):
        stones = '*' * (2 * current_height + 1)
        space = ' ' * (height - current_height - 1)
        print(f"{space}{stones}{space}")

height = 4
print_pyramid(height)

`else` clause

the else clause in a for loop is used when for loop DOES NOT break.

It is equivalent to a if no break.

In [ ]:

# first a method using a boolean
def find_with_bool(seq, target):
    found = False
    for i, value in enumerate(seq):
        if value == target:
            found = True
            break
    if not found:
        return -1
    return i

find_with_bool([1,2,3], 3)

In [ ]:

# but we can now use the else clause
def find_with_else(seq, target):
    for i, value in enumerate(seq):
        if value == seq:
            break
    else:
        return -1
    return i

find_with_else([1,2,3], 3)

using index to loop

If you're using the index to access an element, you're doing it (probably) wrong.

In [ ]:

# BEURK
users = ['greg', 'leo']
for i in range(len(users)):
    print(users[i])

In [ ]:

# good :)
users = ['greg', 'leo']
for user in users:
    print(user)

But I still want the index for some reasons !!

enumerate

the enumerate function returns a pair of elements (index, value)

In [ ]:

users = ['greg', 'leo']
for index, user in enumerate(users):
    print(index, ' - ', user)

But what if I want only to use one element every three ?

In [ ]:

# then use a slice ;-)
users = ['greg', 'leo', 'alban', 'zoe']
for user in users[::3]:
    print(user)

# and please NEVER do this :
users = ['greg', 'leo', 'alban', 'zoe']
for i in range(len(users)):
    if i % 3 == 0:
        print(users[i])

while

while <condition> is executing as long as its <condition> is True
continue is used to go to the next iteration (like a skip)
break is used to stop the loop
the else block is executed when the <condition> is no longer True

In [ ]:

i = 0
while True: #infinite loop
    if i != 4:
        i += 1
        continue
    else:
        break

print(i) # should be 4

In [ ]:

i = 0
while i < 6:
    print(i)
    i += 1
else:
    print("here > i is no longer less than 6")

Looping over a dict

In [ ]:

store = {"apple": 3, "ananas": 1}
for fruit in store:
    print(fruit)

In [ ]:

store = {"apple": 3, "ananas": 1}
for fruit, quantity in store.items():
    print(fruit, quantity)

Exercice, try to edit the dict while iterating over it !
- You will encounter a RuntimeError

In [ ]:

# naive implem
store = {"apple": 3, "ananas": 1}
for fruit in store:
    if fruit == "apple":
        del store[fruit]

In [ ]:

# To edit a dict while iterating over it, we can convert the dict to a list to get a list of keys that doesn't "lock" the dict
store = {"apple": 3, "ananas": 1}
for fruit in list(store):
    if fruit == "apple":
        del store[fruit]

print(store)

/!\ But remember, be careful when mutating a variable while iterating over it /!\

functions (again!)

`*args` and `**kwargs`

When a final formal parameter of the form **name is present, it receives a dictionary (see Mapping Types — dict) containing all keyword arguments except for those corresponding to a formal parameter. This may be combined with a formal parameter of the form *name (described in the next subsection) which receives a tuple containing the positional arguments beyond the formal parameter list. (*name must occur before **name.)

In [ ]:

def dynamic(bar, *args, **kwargs):
    print(bar)
    print("-" * 20)
    for arg in args:
        print(f"arg : {arg}")
    print("-" * 20)
    for k in kwargs:
        print(f"kwarg: {k} => {kwargs[k]}")

dynamic("a", "b", "c", k1="v1", k2="v2")

# TODO trouver un cas concret

a
--------------------
arg : b
arg : c
--------------------
kwarg: k1 => v1
kwarg: k2 => v2

type annotation

Function annotations are optional metadata information about the types used by user-defined functions.

Annotations are stored in the __annotations__ attribute of the function as a dictionary and have no effect on any other part of the function.

In [ ]:

# here we specify that the function expects a string and returns a string
def foo(bar: str) -> str:
    print(bar)
    return "ok"

print(foo.__annotations__)

comprehensions

A comprehension is a short syntax to construct a sequence (such as lists, set, dictionary etc).

Usually on one-line (these are called oneliners) a comprehension "can" replace multiple lines of code. (But should it ??)

In [ ]:

# "classic"
numbers = []
for i in range(10):
    numbers.append(i)
print(numbers)

# comprehension
numbers = [i for i in range(10)]
print(numbers)

In [ ]:

# let's create a dictionary that has x for key and x^2 for value (x->10)

numbers_squared = {}
for i in range(1, 10):
    numbers_squared[i] = i**2
print(numbers_squared)

numbers_squared = {k:k**2 for k in range(1,10)}
print(numbers_squared)

In [ ]:

# we can also add conditions in a comprehension
firstnames = ["Linus", "Bill", "Steve"]

# if we want a sublist without the firstnames that begin with a 'B'
firstnames_no_b = [f for f in firstnames if not f.startswith('B')]

print(firstnames_no_b)

In [ ]:

# more complex example, we have a dict of malwares and we want a sub-dict without windows malwares

malwares = {"linux": ["mirai"], "windows": ["njrat"], "android": ["vikinghorde"]}

malwares_no_windows = {p:m for p,m in malwares.items() if p != "windows" }

print(malwares_no_windows)

Exercice :

with a list of quantities and a list of corresponding fruits, combine those two lists into one dictionary named fruits_and_quantities using a comprehension, with the fruit name as key and the quantity as value., only if the quantity is not 0.

In [ ]:

quantities = [2, 5, 0]
fruits = ['apples', 'bananas', 'kiwis']

#
# code here
#

assert fruits_and_quantities == {'apples':2, 'bananas':5}

print(fruits_and_quantities)

answer

fruits_and_quantities = {f:q for f,q in zip(fruits, quantities) if q > 0}

/!\ BUT, comprehensions as you can see can become very complex and narrow.

A good practice when coding comprehensions is "your comprehension should be equal to a simple sentence".

For example, a list of names without 'f'.

A bad example would be a dict where the key is in the database and the value is not a in the GET query of the RSS url provided by the user if this URL is ...

Google in its python styling guide has some interesting recommendations

modules/packages/libraries/script

a script is a python file designed to be executed
a module is a python file designed to be imported
a package is a directory of modules along with a __init__.py
a library doesn't really exist in python, but can be understood as a collection of packages

example : the module name A.B designates a submodule named B in a package named A

importing

when you import a module, you change the global namespace.

Using dir() without any argument shows what’s in the global namespace.

In [ ]:

dir()

When we import for example math this way import math, we place (or replace if it was already defined) math in the global namespace.

In [ ]:

import math

dir()

If we don't want to "pollute" our global namespace, we can select only certain elements of a module. For example we can only import pi from math with from math import pi. Out global namespace will now contain pi and not math.

If what we want to import has a name that is too generic for our usecase, for example now, we can rename our import with the as statement : from math import pi as math_pi.

Prefer importing the module entirely rateher than selecting only certain things from the module.

Also, use a tool like isort to automaticcaly sort your imports.

Relative imports

Do not use relative names in imports, use absolute imports.

That is, if your app has multiple folders (modules), do not use from ..app.database import db.

Python has some mechanisms to import from the top of your app. (PYTHONPATH, etc ...)

execution

ever wondered what this code does ?

if __name__ == '__main__':
    ...

Whenever the Python interpreter reads a source file, it does two things:

it sets a few special variables like name, and then
it executes all of the code found in the file.

Here's a script example :

# calc.py
def add(a, b):
    return a+b

print("a+b")
a = int(input("enter a: "))
b = int(input("enter b: "))
print(add(a, b))

Now, if I import this file calc.py in another file main.py, and try to execute the main.py file

# main.py
from calc import add

... # do stuff

here's what happens :

$ python3 m.py
a+b
enter a:

The code in the calc.py file is executed too !

The if __name__ == "__main__": code is used to tell python to execute this if block only if the script is called directly.

In our case, our calc.py file can be both a module and a script if we move the interactive code in a function or in the if block :

# calc.py
def add(a, b):
    return a+b

if __name__ == "__main__":
    print("a+b")
    a = int(input("enter a: "))
    b = int(input("enter b: "))
    print(add(a, b))

File handling

/!\ use the with statement, except if you really know what you're doing.

The with statement is a context manager (more on that later). Basically it creates a block-scope where code can be executed (like a if block), and as soon as the execution exits this block (during an error for example), some logic is executed.

For example, a naive way to open and close a text file in python is :

f = open('big-leak.txt','r')
... # do stuff with the file
f.close()

The problem with this naive way, is that if during the "do stuff" code, something bad happens and an exception is thrown, the file will not be closed !

To avoid this scenario, use the with statement.

with (
    open('the-zen-of-python.txt','r') as f,
    open("output.txt", "wb") as o,
):
    # do stuff with the file

.read() method

Be careful when using the .read() method on a file, if you don't specify a size (an optional argument of the read method), the entire content of the file will be returned in a single string. If this file is multiple gigabytes, your RAM won't survive.

There are multiple ways to read a file chunk by chunk to avoid memory exhaustion and slow performance.

Other kind of files

Now that you learned about the with statement, other kind of files can be used with this with statement.

In [ ]:

import tempfile

# temporary file
with tempfile.NamedTemporaryFile() as tmpfile:
    # this file will be destroyed as soon as it is closed
    print('created temporary file', tmpfile.name)

# temporary directory
with tempfile.TemporaryDirectory() as tmpdir_path:
    # this folder will be destroyed as soon as it is closed
    print('created temporary directory', tmpdir_path)

Good practices

(I prefer the term good practices than best practices ¯\_(ツ)_/¯)

Data Structures

https://www.hackerrank.com/domains/data-structures

Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming. -- Rob PIKE (https://users.ece.utexas.edu/~adnan/pike.html)

TODO

exemple evolutif avec dabord des listes, puis dico puis classe puis dataclass

In [ ]:

address = "Greg\n24 rue Paradis\n35000 Rennes\n France"

Context Manager

A context manager is an object that sets up a context for code to run in, enters the context, runs the code and then exits the context.

A context manager is usually called via the with statement.

Not using a context manager

Let's take a look at the following code.

f = open('afileidontknow.txt')
# do stuff
f.close()

What happens if during the do stuff an error occurs and the execution is stopped ? The file won't be closed !

Another alternative could be :

try:
    f = open('afileidontknow.txt')
    # do stuff
except OSError as e:
    # handle exception with open
except XXXError as e:
    # handle exception with do stuff
finally:
    f.close()

with open('afileidontknow.txt') as f:
    # do stuff

How to implement a contextmanager

A context manager is an object that implements the __enter__(self) and __exit__(self, exc_type, exc_value, exc_tb) methods.

In [ ]:

class HelloContextManager:
    def __init__(self, msg, raising=False):
        print("__init__ method called")
        self.msg = msg
        self.raising = raising

    def __enter__(self):
        print("__enter__ method called")
        return self.msg

    def __exit__(self, exc_type, exc_value, exc_tb):
        print("__exit__ method called")
        print(f"{exc_type=} | {exc_value=} | {exc_tb=}")
        if isinstance(exc_value, IndexError):
            # Handle IndexError here...
            return True
        return self.raising

with HelloContextManager("Hello, World !") as msg:
    print(msg)

with open("myfile.txt") as f:
    f.read()

__init__ method called
__enter__ method called
Hello, World !
__exit__ method called
exc_type=None | exc_value=None | exc_tb=None

In [ ]:

with HelloContextManager({"Hello, World !"}) as msg:
    # trigger an error
    msg[0] = "error"

In [ ]:

with HelloContextManager({"Hello, World !"}, True) as msg:
    # trigger an error, which will be ignored
    msg[0] = "error"

The __exit__() method has the following arguments :

exc_type is the exception class.
exc_value is the exception instance.
exc_tb is the traceback object.

The exit method should return a boolean. If True any exception raised during the with statement will be ignored. If False (the default) the exception will be re-raised.

@contextmanager

A @contextmanager decorator is present in the contextlib lib. It can be used to turn a generator (a function that yields something ) into a context manager.

In [81]:

from contextlib import contextmanager

@contextmanager
def simple_context():
    print("__init__ and __enter__")
    yield "hello" # return __enter__
    print("__exit__")

with simple_context() as s:
    print("during the context")
    print(s)

Out[81]:

__init__ and __enter__
during the context
hello
__exit__

In [ ]:

from contextlib import contextmanager

@contextmanager
def open2(filepath: str):
    print("__init__ and __enter__")
    f = open(filepath)
    yield f # return __enter__
    print("__exit__")
    f.close()

with open2("idontknow.txt") as f:
    f.read()

In [ ]:

from contextlib import contextmanager

@contextmanager
def open_db():
    print("enter")
    try:
        yield "a connection to a database"
    except OSError as err:
        # handle error
        pass
    finally:
        print("exit")

with open_db() as db:
    print("during the context")
    print(db)

Decorator

A decorator is a function that takes in a function and extend its behaviour, for example it can check its arguments or its return value, measure its execution time ...

In [82]:

def simple_decorator(func):
    def inner():
        print("inside the decorator")
        return func()
    return inner

def hello_no_deco():
    print("Hello, World!")

@simple_decorator
def hello():
    print("Hello, World !")

hello_no_deco()

Out[82]:

Hello, World!

In [83]:

hello()

Out[83]:

inside the decorator
Hello, World !

This simple decorator has a flaw, it will mess with the introspection of the object !

We must use another decorator, wraps, from the functools standard package.

In [84]:

print(hello.__name__)

Out[84]:

inner

In [85]:

import functools

def decorator_adding_argument(func):
    @functools.wraps(func)
    def inner():
        msg = "world"
        return func(msg)
    return inner

@decorator_adding_argument
def print_it(x):
    print("Hello", x)

print_it()
print(print_it.__name__)

Out[85]:

Hello world
print_it

In [ ]:

import functools

def decorator_interceptor(func):
    @functools.wraps(func)
    def inner(*args, **kwargs):
        print("inside the decorator")
        print(f"{args=}")
        print(f"{kwargs=}")
        if kwargs.get("user") == "Greg":
            print("NON !")
            return
        return func(*args, **kwargs)
    return inner

@decorator_interceptor
def print_it(a, b, user=""):
    print("Hello")

print_it(1, 2, user="Greg")

In [86]:

# https://realpython.com/primer-on-python-decorators/#a-few-real-world-examples
import functools
import time

def timer(func):
    @functools.wraps(func)
    def wrapper_timer(*args, **kwargs):
        start_time = time.perf_counter()
        value = func(*args, **kwargs)
        end_time = time.perf_counter()
        run_time = end_time - start_time
        print(f"Finished {func.__name__!r} in {run_time:.4f} secs")
        return value
    return wrapper_timer

@timer
def waste_some_time(num_times):
    for _ in range(num_times):
        sum([i**2 for i in range(10000)])

waste_some_time(999)

Out[86]:

Finished 'waste_some_time' in 0.4210 secs

Generators

TODO xxx

In [ ]:

import time
import random

def generate_data():
    time.sleep(1)
    return random.randint(1, 6)

ten_elements = [generate_data() for _ in range (10)]
for e in ten_elements:
    print(e)

In [ ]:

generator_ten_elements = (generate_data() for _ in range (10))
for e in generator_ten_elements:
    print(e)

Exceptions

Analogy : Imagine that you order something on the web, and during the delivery you're not at home. Error handling in this context means that the delivery company should be able to deliver your package another time/place so you can have it.

TODO

Protocol

In Python a protocol is an informal interface. Protocols are either known as an accepted truth or defined in documentation and not strictly in code1. For example, any class that implements the __container__() special method is said to follow the "container protocol." While this is an accepted truth, there is no syntax in the Python language that declares a class as following a protocol. Python classes can also implement multiple protocols.

TODO

Design patterns

TODO

a design pattern is a general repeatable solution to a commonly occurring problem in software design. A design pattern isn't a finished design that can be transformed directly into code. It is a description or template for how to solve a problem that can be used in many different situations.

Iterator Pattern

TODO

Decorator Pattern

TODO

Strategy Pattern

TODO

State Pattern

TODO

Singleton Pattern

TODO

Template Pattern

TODO

Adapter Pattern

TODO

Facade Pattern

TODO

Flyweight Pattern

TODO

Command Pattern

TODO

Abstract Factory Pattern

TODO

Composite Pattern

TODO

Advanced

OOP

TODO

functional

TODO

Threading

TODO

GIL

Testing

TODO

env

TODO

todo cours recopies pas fini ou deja fait etc

affichage print signature d'une fonction help() function __dir__ method fstring

conditions if /elif / else == != > >= < <= continue saute à l’itération suivante break stoppe la boucle

exercice :
    jours = ["lundi", "mardi", "mercredi", "jeudi", "vendredi", "samedi", "dimanche"]
        de lundi a jeudi : travail
        vendredi : soon le w-e
        samedi dimanche : BBQ

boucle si vous utilisez des index, cest que vous le faites probablement pas bien for for i in range(len(colors)): print(colors[i])

    for i in range(len(colors)-1, -1, -1)
        print(colors[i])

    enumerate + for/else

        def find(seq, target):
            found = False
            for i, value in enumerate(seq):
                if value == target:
                    found = True
                    break
            if not found:
                return -1
            return i

        def find(seq, target):
            for i, value in enumerate(seq):
                if value == seq:
                    break
            else:
                return -1
            return i
    zip
        2 listes

    for n in range(2, 10):
        for x in range(2, n):
            if n % x == 0:
                print(n, 'equals', x, '*', n//x)
                break
        else:
            # loop fell through without finding a factor
            print(n, 'is a prime number')

valeur sentinel

    from functools import partial
    from random import randint

    pull_trigger = partial(randint, 1, 6)

    print('Starting a game of Russian Roulette...')
    print('--------------------------------------')

    for i in iter(pull_trigger, 6): #iter prend en argument une fonction ne renvo
        print('I am still alive, selected', i)
    print('Oops, game over, I am dead! :(')

    #partial() est utilisé pour une application de fonction partielle
    qui "gèle" une portion des arguments et/ou mots-clés d'une fonction
    donnant un nouvel objet avec une signature simplifiée.

while
range
iter

dir() help() - how to use it + how does it work (docstrings) sorted() - sorted(, key)

fonction (vs procedure) signature portée LGI anonyme DRY passage d'arguments renvoi de résultats arguments positionnels vs. arguments par mots clés twitter_search("@obama", False, 20, True) twitter_search("@obama", retweets=False, numtweets=20, popular=True)

tuples NamedTuples (subclass of tuple) unpacking : p = 'pommes', 30, 1 produit, quantite, prix = p

dictionnaires fundamental relationships, linking, counting, grouping

for k in d:
    print(k)

cant mutate a dict while youre iterating over it

for k in d.keys():
    if k.startswith('non'):
        del d[k]

for k in d: #non
    print(f"{k} --> {d[k]}")

for k, v in d.items():
    print(f"{k} --> {v}")

colors = ["red", "green"]
d = {}
for color in colors:
    if color not in d: #raise an error but its a not so no problem
        d[color] = 0
    d[color] += 1

d = {}
for color in colors:
    d[color] = d.get(color, 0) + 1

defaultdict # too advanded but it exists ;)

grouping :
    d = {}
    for name in names:
        key = len(name)
        d.setdefault(key, []).append(name)

    d = defaultdict(list)
    for name in names:
        key = len(name)
        d[key].append(name)

linking :
    collections.ChainMap

fichier binaire modules / package docstring

iterators

context manager with

try: #wrong way to do it
    os.remove('somefile.tmp')
except OSError:
    pass

from contextlib import suppress
with suppress(FileNotFoundError):
    os.remove('somefile.tmp')

from contextlib import contextmanager

@contextmanager
def open_db(filename: str):
    try:
        con = sqlite3.connect(filename)
        yield con
    except sqlite3.DatabaseError as err:
        logger.error(err    )
    finally:
        con.close()

one liner : 1 ligne de code = 1 phrase en francais

bonnes pratiques pep8 (syntaxe) pep257 pep20

python : everything is an object

decorateurs try except comprehension generateurs magic methods

DRY YAGNI

classes interfaces conventions (snake case, globals in all-caps) "pythonic" code https://www.python.org/dev/peps/pep-0008/ https://google.github.io/styleguide/pyguide.html

Voici quelques conseils pour vous aider à concevoir un script Python.

Réfléchissez avec un papier, un crayon... et un cerveau (voire même plusieurs) ! Reformulez avec des mots en français (ou en anglais) les consignes qui vous ont été données ou le cahier des charges qui vous a été communiqué. Dessinez ou construisez des schémas si cela vous aide.
Découpez en fonctions chaque élément de votre programme. Vous pourrez ainsi tester chaque élément indépendamment du reste. Pensez à écrire les docstrings en même temps que vous écrivez vos fonctions.
Quand l’algorithme estc omplexe, commentez votre code pour expliquer votre raisonnement. Utiliser des fonctions(ou méthodes) encore plus petites peut aussi être une solution.
Documentez-vous. L’algorithme dont vous avez besoin existe-t-il déjà dans un autre module ? Existe-t-il sous la forme de pseudo-code ? De quels outils mathématiques avez-vous besoin dans votre algorithme ?
Si vous créez ou manipulez une entité cohérente avec des propriétés propres, essayez de construire une classe.
Utilisez des noms de variables explicites, qui signifient quelquechose. En lisant votre code, on doit comprendre ce que vous faites. Choisir des noms de variables pertinents permet aussi de réduire les commentaires.
Quand vous construisez une structure de données complexe (par exemple une liste de dictionnaires contenant d’autres objets), documentez et illustrez l’organisation de cette structure de données sur un exemple simple.
Testez toujours votre code sur un jeu de données simple pour pouvoir comprendre rapidement ce qui se passe. Par exemple, une séquence de 1000 bases est plus facile à gérer que le génome humain ! Cela vous permettra également de retrouver plus facilement une erreur lorsque votre programme ne fait pas ce que vous souhaitez.
Lorsque votre programme « plante », lisez le message d’erreur. Python tente de vous expliquer ce qui ne va pas. Le numéro de la ligne qui pose problème est aussi indiqué.
Discutez avec des gens. Faites tester votre programme par d’autres. Les instructions d’utilisation sont-elles claires ?
Si vous distribuez votre code :
Rédigez une documentation claire.
Testez votre programme (jetez un œil aux tests unitaires 10).
Précisez une licence d’utilisation. Voir par exemple le site Choose an open source license

If you are ever wondering how these structures work, have a look at the following source: https://github.com/python/cpython/blob/main/Lib/collections/__init__.py

Python Full Course

Basics

History of Python

What is Python

Why choose python then ?

"Our clients"

ZEN of python

CPython

mise-en-jambe

Comments

Variables

Memory management

Naming

Primitive/native data types

Casting / converting a type to another one

Operators

is keyword

None

Mutability

Type hinting

Functions (intro)

functions vs procedure

Multi return

declaration

Default arguments

Be careful of default mutable arguments

Signature

LGI

Good Naming

Famous Acronyms

Built-in functions

Classes (intro)

Magic methods

string representation (repr + str) example

Functions are objects ?

String interpolation

String modulo operator

str.format() method

f-string

limitations

Lists

Slicing

Common list methods

list built-in function

in keyword

len built-in function

del keyword

Common built-in functions used with lists

Strings

Common string methods

Ellipsis

Tuples

Really immutable ?

Unpacking

Dictionaries

Common dict methods

in keyword

del keyword

unpacking (again!)

Sets

in keyword

pointers, pass-by-value, pass-by-reference

Some examples of memory usage

range

conditions

TODO if / elif / else break continue

loops

for

else clause

using index to loop

enumerate

while

Looping over a dict

functions (again!)

*args and **kwargs

type annotation

comprehensions

modules/packages/libraries/script

importing

Relative imports

`is` keyword

`list` built-in function

`in` keyword

`len` built-in function

`del` keyword

`in` keyword

`del` keyword

`in` keyword

`range`

`else` clause

`*args` and `**kwargs`