**Views:**

^{32}

**Visibility:**Unlisted (only visible to those who know the link)

**Image:**ubuntu2004

**Kernel:**SageMath 9.1

# Lecture 9: Modules, packages and SageMath core development

**References:**

**Summary:**

In this lecture we discuss how (and why) to put code into **Python modules** and **Python packages**, and how to install the latter on your system. In the second part, I show a practical example of the steps necessary to contribute to the code of SageMath itself.

## Python modules and packages

In the previous lecture we saw that one way to share code is to save it to a `.sage`

-file, and run it using `load`

. This is quick to set up and works reasonably in many use cases, but it has some drawbacks (that we discuss below). These can be avoided using **modules** and **packages** in Python.

### Python modules

One disadvantage when using `load`

is that it transfers all variable and function names from the loaded file into the current session (of Python or SageMath). In particular, this overwrites existing functions, which can create problems e.g. if you want to use multiple code files using the same function name.

To avoid this, you can instead store code inside a file with ending `.py`

(which defines a **Python module**), and then **import** this file. Instead of loading all function names into the current session, this will keep them bundled up inside a module, and you can then access them as shown below.

#### Exercise

Create a file

`basics.py`

in the folder of the current notebook, and save the following text inside (similar to the`.sage`

files we had before). If you feel lazy, you can also download the file here.

def is_prime(n): for m in range(2, n): if n % m == 0: return False return True def one_third(): return 1/3

Import it as a module using

import basics

Run the function

`is_prime`

in an example of your choice, accessing it via`basics.is_prime`

.

## Solution (click to expand)

import basics basics.is_prime(8) > False

Some remarks about the example above:

First, note that the

`is_prime`

function from the modulie`basics`

has not overwritten the default`is_prime`

function in SageMath.

is_prime == basics.is_prime

If you

*do*want to import some function into the namespace of the current session, you can do it as follows:

from basics import is_prime

is_prime == basics.is_prime

Finally, it is both possible to import a function (or class) giving it a custom name, or to import

*all*names from a given module.

from basics import is_prime as is_p

is_p(7)

from basics import *

One issue to be aware of is that code stored in a `.py`

file is interpreted as *Python* code. A first consequence is that e.g. numbers you type in this code will be interpreted as Python `int`

or `float`

values. Therefore, the function `one_third`

above, which returns `1/3`

, will actually return `int(1)/int(3)`

which gives a `float`

:

one_third()

Similarly, you need to use e.g. `**`

instead of `^`

for exponentiation. Moreover, functions (like `factorial`

) that are part of SageMath will not automatically be available and need to be imported *inside the new Python module*. To get the line of code that you need to add for this (traditionally at the beginning of the `.py`

file) you can use the function `import_statements`

in your SageMath session:

import_statements(factorial)

#### Exercise

Say we want to have a function computing `1/3`

as a `Rational`

. The following code is a first attempt:

def better_one_third(): return Integer(1)/Integer(3)

Add the code above to the file

`basics.py`

. Restart the kernel of the notebook (via`Kernel -> Restart`

in the menu) and import basics again. Check that when trying to call this function, we obtain an error message since`Integer`

is not known.

*Remark:*Note that restarting the kernel is really necessary: the`import`

will not overwrite a module that was imported before, even if its code changed in the meantime.Find the correct line for importing the missing name

`Integer`

and add it to`basics.py`

. Then restart the kernel, import the module and check that now the function`better_one_third`

works.

## Solution (click to expand)

Before adding the right import:

import basics basics.better_one_third() > --------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-1-9ae9ca737176> in <module>() 1 import basics ----> 2 basics.better_one_third() /home/sage/Desktop/Computer algebra/Notebooks/basics.py in better_one_third() 11 12 def better_one_third(): ---> 13 return Integer(1)/Integer(3) NameError: name 'Integer' is not defined

Finding the right import:

import_statements(Integer) > from sage.rings.integer import Integer

After adding this line to the file `basics.py`

:

import basics basics.better_one_third() > 1/3

While it is a bit cumbersome having to import the relevant SageMath-functions as above, it is (as far as I know) the only way to use the strengths of Python modules (and the Python packages below) in SageMath. One tool making it easier is to use the **automatic preparser** of SageMath. This means, you can create a file `basics.sage`

and write normal Sage-code (that would work inside a SageMath notebook). Then, open a terminal (not a SageMath session!) and type

sage --preparse basics.sage

The resulting code is correct, but has the disadvantage of containing some inconvenient abbreviations (e.g. for constants).

## Click here for the preparsed version of the original code block above

# This file was *autogenerated* from the file basics_for_preparsing.sage from sage.all_cmdline import * # import sage library _sage_const_2 = Integer(2); _sage_const_0 = Integer(0); _sage_const_1 = Integer(1); _sage_const_3 = Integer(3) def is_prime(n): for m in range(_sage_const_2 , n): if n % m == _sage_const_0 : return False return True def one_third(): return _sage_const_1 /_sage_const_3

For more information see this guide.

### Python packages

For bigger projects, it is often convenient to be able to split the code in multiple files (containing different parts of the project). To bundle these, one can combine them into a **Python package**.

To get our hands on an example, you can download this zip file and unpack it (in all common operating systems: do a right-click and select "Extract all" or something similar). You should get a folder structure as follows:

testpackage/ ├── LICENSE ├── README.md ├── setup.py └── testpack ├── __init__.py ├── advanced.py └── basics.py

Ignoring the files `LICENSE`

etc for now, the actual Python package is `testpack`

. In general, such a package is essentially a folder containing

a file

`__init__.py`

some Python modules, which can either be

`.py`

-files or further sub-packages

Here the `__init__.py`

-file mostly has a dummy function (allowing the folder to be recognized as a package). However, it can contain some Python code, which is executed when `testpack`

is first imported. Let's try it out and make some comments.

Make sure that the folder `testpackage`

is contained in the same folder as the current notebook. Then we can load the package `testpack`

as follows:

import testpackage.testpack as testpack

type(testpack)

When the package is *first* imported, all code contained in the `__init__.py`

file is run (inside the module). In our case, the `__init__.py`

file contains the following lines:

from .advanced import my_favorite_function as myfavfunc from .basics import scalar_product

Note that above we do a **relative import**: the `.`

stands for the current directory of `__init__.py`

, and so `.advanced`

stands for the file `advanced.py`

located in the same folder as `__init__.py`

.

Typing `testpack.`

+ `Tab`

you can check that the module `testpack`

now contains

the sub-modules

`advanced`

and`basic`

, andthe functions

`myfavfunc`

and`scalar_product`

which were imported in the`__init__.py`

file.

testpack.

So in practice, you can use the `__init__.py`

file to determine which functions will be loaded into the session when the user types

from testpack import *

#### Exercise

Look at the code contained in `advanced`

and `basics`

, test some of the functions below and try to understand what each line does.

## Solution (click to expand)

There is not a well-defined solution to this exercise, but here are some more phenomena to notice:

As before we have that the code is interpreted as Python code, so

`testpack.advanced.one_third()`

returns`0.3333333333333333`

.Both

`advanced.py`

and`basics.py`

contain a definition of a function`my_favorite_function`

. Using the structure of modules, these stay nice and separate:

testpack.advanced.my_favorite_function() > 4/3 testpack.basics.my_favorite_function() > 0

The different files can import from each other (with some care!):

the module

`advanced`

imports the function`one_third`

from`basics`

inside the function

`scalar_product`

we import`multiply`

from`advanced`

Here it's important that we*cannot*import`multiply`

directly at the start of`advanced.py`

. This leads to a**circular import**, which we have to avoid. Since the code inside the function definitions is not executed, we are good.

### Installing and distributing packages

#### Getting the package ready for installation

For the Python package `testpack`

above, there is one drawback: to import it, it is most convenient to run the SageMath console or notebook inside the folder containing `testpack`

(otherwise the command `import testpack`

will give an error). This can be solved by **installing the package**, effectively adding it permanently to the list of "known modules" of your SageMath installation.

To make this possible, we put the folder `testpack`

into a directory `testpackage`

with several other files. In more detail, these are:

`setup.py`

: It contains instructions for the setuptools module of Python, which takes care of the installation. In the file we provide some basic information about the package (such as name, author, a short description etc). An important variable is`packages`

which includes the path (relative to`setup.py`

) of the actual Python package we want to install.`README.md`

: For a longer description, it is customary to provide such a readme file. Here`.md`

stands for Markdown, the same language we use for the text/formulas/pictures in Jupyter notebooks. This description is also input in the setup instructions above via a short script in`setup.py`

loading its text into the parameter`long_description`

.`LICENSE`

: This file contains the text of a possible software license that you want to use when distributing your code (see below).

#### The Python package manager pip

With these preparations in place, we can install `testpack`

via the Python package manager pip. For this, navigate with a terminal into the folder `testpackage`

(not `testpack`

!) and type

sage -pip install .

Here we use `sage -pip`

to use the version of `pip`

associated with the Python installation used by SageMath, and `.`

again stands for the current directory (containing `setup.py`

).

#### Exercise

Install the package

`testpack`

as described above.Navigate with your terminal into a folder

*not*containing the`testpack`

folder.Open a SageMath session, run the command

`import testpack`

and execute one of the functions from that package.

Some variants of the command above:

sage -pip install -e .

Installs the package in **editable mode**. This means, if you change the code inside `testpack`

it will change the behaviour of the module (with code `sage -pip install .`

above you save a copy of the module at the time of installation, which is not changed when you modify the original code). This editable option is particularly useful if you are still actively working on the code of the package.

sage -pip install . --upgrade

If you already have a previous version installed, you can use this to upgrade to the new one.

#### Revisiting doctests

Once you installed the package, it is also possible to run any of the doctests that you included. You should run the `sage -t`

command on the folder `testpack`

containing the code files.

sage -t testpackage/testpack

Note that to make this work, your doctests should always start by importing the relevant functions from the package, like the following doctest of `scalar_product`

:

EXAMPLES:: sage: from testpack.basics import scalar_product sage: scalar_product([1,2], [3,4]) 11

#### The Python package indey PyPI

One final great feature about pip is that it is by default connected to the Python package index (PyPI). This is an online database of Python packages, and to install a package `foo`

from there, you just need to type

sage -pip install foo

For our beloved test package above, I did not upload it to PyPI itself, but (following these instructions) I uploaded it to a test-verion of PyPI. Thus, from a console anywhere you can install it using the command

sage -pip install -i https://test.pypi.org/simple/ testpackmath007

*Remark:* As you see I had to change the name to `testpackmat007`

since, maybe not unsurprisingly, the very creative name `testpack`

was already taken ...

## Summary: the right format for your code

Here we provide again an overview over the strengths and weaknesses of various ways of storing your code:

**Jupyter notebooks**are nice for short pieces of code with some additional explanatory text and examples**Sage files**are great for projects of medium size, where the user wants to load all relevant functions into their session**Python files**allow to create a separate namespace, avoiding collisions with user-defined functions, but they require that you write pure Python code and import relevant SageMath functions by hand**Python packages**are the professional standard for projects of greater complexity and can use the Python package index for easy distribution and installation

## A panorama of further tools for development

When editing your code files, you can use the versioning software git to

keep a record of previous versions of your code

synchronize your local files with some online repository (like gitlab) to allow multiple people to work on them

On platforms like gitlab, you also have access to tools for continuous integration, e.g. a script which runs all doctests of your code whenever you create a new version. In addition, there are tools like airspeed velocity which automatically check the speed of the code on a set of example computations, so that you see whether your changes made things faster or slower.

You can use sphinx to create nice documentation pages for your code (such as this one). They are created by putting the docstrings of your functions into some nice, readable format.

## SageMath core development - a practical example (a.k.a. showing my homework)

### The task

Recall from the first lecture the following disappointing performance of SageMath:

M = matrix([[17,2,-4,9],[-13,2,8,3],[6,1,-1,4],[5,5,-2,2]]) P = M.characteristic_polynomial() P

S = ZZ[x].quotient_ring(P); S

S.is_integral_domain()

The problem was that the class of `S`

did not have a good implementation of the function `is_integral_domain`

. I gave myself the exercise to write a suitable version of the `is_integral_domain`

function, and get it added to SageMath.

### My solution

You can see the results of my effort under the following trac ticket of SageMath. To see the actual code changes I proposed, you can click on the link to the branch which I created.

Below is the relevant function. Note that in this case `self`

will be the ring in question, which is of the form $R[x]/(f(x))$ where $R$ (=`self.base_ring()`

) is any commutative ring with $1$ and $f$ (=`self.modulus()`

) is a nonzero element of $R[x]$ with leading coefficient which is a unit in $R$.

After executing the cell below, the rings will know this better function `is_integral_domain`

, so when you run the cell above again, it will give the correct result.

def is_integral_domain(self, proof = True): """ Return whether or not this quotient ring is an integral domain. EXAMPLES:: sage: R.<z> = PolynomialRing(ZZ) sage: S = R.quotient(z^2 - z) sage: S.is_integral_domain() False sage: T = R.quotient(z^2 + 1) sage: T.is_integral_domain() True sage: U = R.quotient(-1) sage: U.is_integral_domain() False sage: R2.<y> = PolynomialRing(R) sage: S2 = R2.quotient(z^2 - y^3) sage: S2.is_integral_domain() True sage: S3 = R2.quotient(z^2 - 2*y*z + y^2) sage: S3.is_integral_domain() False sage: R.<z> = PolynomialRing(ZZ.quotient(4)) sage: S = R.quotient(z-1) sage: S.is_integral_domain() False TESTS: Here is an example of a quotient ring which is not an integral domain, even though the base ring is integral and the modulus is irreducible:: sage: B = ZZ.extension(x^2 - 5, 'a') sage: R.<y> = PolynomialRing(B) sage: S = R.quotient(y^2 - y - 1) sage: S.is_integral_domain() Traceback (most recent call last): ... NotImplementedError sage: S.is_integral_domain(proof = False) False The reason that the modulus y^2 - y -1 is not prime is that it divides the product (2*y-(1+a))*(2*y-(1-a)) = 4*y^2 - 4*y - 4. Unfortunately, the program above is already unable to determine that the modulus is irreducible. """ from sage.categories.all import IntegralDomains if self.category().is_subcategory(IntegralDomains()): return True ret = self.base_ring().is_integral_domain(proof) if ret: try: irr = self.modulus().is_irreducible() if not irr: # since the modulus is nonzero, the condition of the base ring being an # integral domain and the modulus being irreducible are # necessary but not sufficient ret = False else: from sage.categories.gcd_domains import GcdDomains if self.base_ring() in GcdDomains(): # if the base ring is a GCD domain, the conditions are sufficient ret = True else: raise NotImplementedError except NotImplementedError: if proof: raise else: ret = False if ret: self._refine_category_(IntegralDomains()) return ret # Adding this function to existing class via monkey patching from sage.rings.polynomial.polynomial_quotient_ring import PolynomialQuotientRing_generic PolynomialQuotientRing_generic.is_integral_domain = is_integral_domain

R.<z> = PolynomialRing(ZZ) S = R.quotient(z^2 - z) S.is_integral_domain()

T = R.quotient(z^2 + 1) T.is_integral_domain()

### Solution steps

The abstract process for contributing code to SageMath is described in the official developer guide. Below I give a summary of the concrete steps I had to take:

To change the source code of SageMath, one needs to first download this code, and compile SageMath from it. Many of the usual installations will give you access to a pre-compiled version, which is not enough to do SageMath development. For me, working on Windows, the easiest way to do this compilation was to work with the Windows subsystem for Linux, which is an installation of Linux running inside Windows.

I made changes to the source code (mostly in the file

`/src/sage/rings/polynomial/polynomial_quotient_ring.py`

), adding the function`is_integral_domain`

above. Note that the function has a documentation with several examples.Using this updated version of SageMath, I ran all doctests in the source files of SageMath itself, and encountered an error! It turns out that my changes broke some expected behaviour in an obscure little corner of the software dealing with splitting algebras (whatever that is). This was fixed using the code

from sage.categories.all import IntegralDomains if self.category().is_subcategory(IntegralDomains()): return True

Note that I would not have found this in a million years without doctests!

I pushed my changes to a new

**branch**on the trac server of SageMath where the development happens. For this I needed to create an account there, and install the git-trac software on my computer.I got some helpful feedback from Vincent Delecroix, one of the main developers of SageMath, and I changed the code accordingly.

Once this review was finished, the code was marked ready to be merged into the main (development) version of SageMath (this happened a few days later).

This means that starting from the next version of SageMath, which is scheduled to release sometime later this year, it will be possible to use the function above.

## Assignments

#### Exercise

Recall that in a previous lecture we had some code for a class `Rectangle`

implementing rectangles with sides parallel to the $x$ and $y$-axes. Similarly, one could write a class `Circle`

implementing circles in the plane (e.g. with given center and radius).

Write a small Python package, containing files

`rectangles.py`

and`circles.py`

which contain classes for such rectangles and circles. Add methods so that for any rectangle, one can compute the circumscribed circle and so that for any circle one can compute the (unique) "insquare", i.e. the unique square with corners on the circle (and sides parallel to $x,y$-directions as before). Make sure that at least some of your functions contain documentation with doctests.Install the package in your system.

Run the doctests.

## Solution (click to expand)

You find one possible solution for this package here. Below is the code for the files `rectangles.py`

:

from sage.functions.other import sqrt class Rectangle: def __init__(self, xmin, xmax, ymin, ymax): self.xmin = xmin self.xmax = xmax self.ymin = ymin self.ymax = ymax def __repr__(self): return 'Rectangle [{}, {}] x [{}, {}]'.format(self.xmin, self.xmax, self.ymin, self.ymax) def area(self): r""" Compute the area of the rectangle. EXAMPLES:: sage: from shapespack import Rectangle sage: R = Rectangle(0, 4, 2, 3) sage: R.area() 4 sage: R = Rectangle(0, 0, 2, 3) sage: R.area() 0 """ return (self.xmax-self.xmin)*(self.ymax-self.ymin) def is_square(self): r""" Check whether the rectangle is a square. EXAMPLES:: sage: from shapespack import Rectangle sage: R = Rectangle(2, 4, 3, 5) sage: R.is_square() True sage: R = Rectangle(2, 4, 3, 6) sage: R.is_square() False TESTS:: In the first implementation, the following example caused a problem, since it returned ``pi == pi`` instead of ``True``. sage: from shapespack import * sage: R = Rectangle(0, pi, 0, pi) sage: R.is_square() True """ return bool((self.xmax-self.xmin) == (self.ymax-self.ymin)) def center(self): r""" Compute the center of mass of the rectangle. EXAMPLES:: sage: from shapespack import Rectangle sage: R = Rectangle(-3, -2, 0, 8) sage: R.center() (-5/2, 4) """ return ((self.xmin + self.xmax)/2, (self.ymin + self.ymax)/2) def circumcircle(self): r""" Compute the circle through the four vertices of the rectangle. EXAMPLES:: sage: from shapespack import Rectangle sage: R = Rectangle(-3, -2, 0, 8) sage: R.circumcircle() Circle around (-5/2, 4) of radius 1/2*sqrt(65) """ from .circles import Circle center = self.center() x, y = center radius = sqrt((x-self.xmin)**2 + (y-self.ymin)**2) return Circle(center, radius)

and `circles.py`

:

from sage.symbolic.ring import SR from sage.functions.other import sqrt class Circle: def __init__(self, center, radius): self.center = center self.radius = radius def __repr__(self): return f'Circle around {self.center} of radius {self.radius}' def area(self): r""" Computes the area of the circle. EXAMPLES:: sage: from shapespack import Circle sage: C = Circle((2,3), 5) sage: C Circle around (2, 3) of radius 5 sage: C.area() 25*pi """ return SR.pi() * self.radius**2 def insquare(self): r""" Compute the unique square with vertices on the circle and sides parallel to the x,y axes. EXAMPLES:: sage: from shapespack import Circle sage: C = Circle((2,3), 3*sqrt(2)); C Circle around (2, 3) of radius 3*sqrt(2) sage: R = C.insquare(); R Rectangle [-1, 5] x [0, 6] sage: R.circumcircle() Circle around (2, 3) of radius 3*sqrt(2) """ from .rectangles import Rectangle x,y = self.center r = self.radius sq2 = sqrt(2) return Rectangle(x-r/sq2, x+r/sq2, y-r/sq2, y+r/sq2)

If you have a terminal inside the folder `shapespackage`

, you can install it and run the doctests as follows:

sage -pip install -e . sage -t shapespack > too few successful tests, not using stored timings Running doctests with ID 2022-05-17-07-23-46-a49dd1e2. Using --optional=bliss,ccache,cmake,coxeter3,dochtml,mcqd,primecount,sage,tdlib Doctesting 3 files. sage -t shapespack/circles.py [8 tests, 0.02 s] sage -t shapespack/rectangles.py [19 tests, 0.02 s] sage -t shapespack/__init__.py [0 tests, 0.00 s] ---------------------------------------------------------------------- All tests passed! ---------------------------------------------------------------------- Total time for all tests: 1.4 seconds cpu time: 0.0 seconds cumulative wall time: 0.0 seconds