Lecture 9: Modules, packages and SageMath core development
In this lecture we discuss how (and why) to put code into Python modules and Python packages, and how to install the latter on your system. In the second part, I show a practical example of the steps necessary to contribute to the code of SageMath itself.
Python modules and packages
In the previous lecture we saw that one way to share code is to save it to a
.sage-file, and run it using
load. This is quick to set up and works reasonably in many use cases, but it has some drawbacks (that we discuss below). These can be avoided using modules and packages in Python.
One disadvantage when using
load is that it transfers all variable and function names from the loaded file into the current session (of Python or SageMath). In particular, this overwrites existing functions, which can create problems e.g. if you want to use multiple code files using the same function name.
To avoid this, you can instead store code inside a file with ending
.py (which defines a Python module), and then import this file. Instead of loading all function names into the current session, this will keep them bundled up inside a module, and you can then access them as shown below.
Create a file
basics.pyin the folder of the current notebook, and save the following text inside (similar to the
.sagefiles we had before). If you feel lazy, you can also download the file here.
Import it as a module using
Run the function
is_primein an example of your choice, accessing it via
Some remarks about the example above:
First, note that the
is_primefunction from the modulie
basicshas not overwritten the default
is_primefunction in SageMath.
If you do want to import some function into the namespace of the current session, you can do it as follows:
Finally, it is both possible to import a function (or class) giving it a custom name, or to import all names from a given module.
One issue to be aware of is that code stored in a
.py file is interpreted as Python code. A first consequence is that e.g. numbers you type in this code will be interpreted as Python
float values. Therefore, the function
one_third above, which returns
1/3, will actually return
int(1)/int(3) which gives a
Similarly, you need to use e.g.
** instead of
^ for exponentiation. Moreover, functions (like
factorial) that are part of SageMath will not automatically be available and need to be imported inside the new Python module. To get the line of code that you need to add for this (traditionally at the beginning of the
.py file) you can use the function
import_statements in your SageMath session:
Say we want to have a function computing
1/3 as a
Rational. The following code is a first attempt:
Add the code above to the file
basics.py. Restart the kernel of the notebook (via
Kernel -> Restartin the menu) and import basics again. Check that when trying to call this function, we obtain an error message since
Integeris not known.
Remark: Note that restarting the kernel is really necessary: the
importwill not overwrite a module that was imported before, even if its code changed in the meantime.
Find the correct line for importing the missing name
Integerand add it to
basics.py. Then restart the kernel, import the module and check that now the function
While it is a bit cumbersome having to import the relevant SageMath-functions as above, it is (as far as I know) the only way to use the strengths of Python modules (and the Python packages below) in SageMath. One tool making it easier is to use the automatic preparser of SageMath. This means, you can create a file
basics.sage and write normal Sage-code (that would work inside a SageMath notebook). Then, open a terminal (not a SageMath session!) and type
The resulting code is correct, but has the disadvantage of containing some inconvenient abbreviations (e.g. for constants).
For more information see this guide.
For bigger projects, it is often convenient to be able to split the code in multiple files (containing different parts of the project). To bundle these, one can combine them into a Python package.
To get our hands on an example, you can download this zip file and unpack it (in all common operating systems: do a right-click and select "Extract all" or something similar). You should get a folder structure as follows:
Ignoring the files
LICENSE etc for now, the actual Python package is
testpack. In general, such a package is essentially a folder containing
some Python modules, which can either be
.py-files or further sub-packages
__init__.py-file mostly has a dummy function (allowing the folder to be recognized as a package). However, it can contain some Python code, which is executed when
testpack is first imported. Let's try it out and make some comments.
Make sure that the folder
testpackage is contained in the same folder as the current notebook. Then we can load the package
testpack as follows:
When the package is first imported, all code contained in the
__init__.py file is run (inside the module). In our case, the
__init__.py file contains the following lines:
Note that above we do a relative import: the
. stands for the current directory of
__init__.py, and so
.advanced stands for the file
advanced.py located in the same folder as
Tab you can check that the module
testpack now contains
scalar_productwhich were imported in the
So in practice, you can use the
__init__.pyfile to determine which functions will be loaded into the session when the user types
Look at the code contained in
basics, test some of the functions below and try to understand what each line does.
Installing and distributing packages
Getting the package ready for installation
For the Python package
testpack above, there is one drawback: to import it, it is most convenient to run the SageMath console or notebook inside the folder containing
testpack (otherwise the command
import testpack will give an error). This can be solved by installing the package, effectively adding it permanently to the list of "known modules" of your SageMath installation.
To make this possible, we put the folder
testpack into a directory
testpackage with several other files. In more detail, these are:
setup.py: It contains instructions for the setuptools module of Python, which takes care of the installation. In the file we provide some basic information about the package (such as name, author, a short description etc). An important variable is
packageswhich includes the path (relative to
setup.py) of the actual Python package we want to install.
README.md: For a longer description, it is customary to provide such a readme file. Here
.mdstands for Markdown, the same language we use for the text/formulas/pictures in Jupyter notebooks. This description is also input in the setup instructions above via a short script in
setup.pyloading its text into the parameter
LICENSE: This file contains the text of a possible software license that you want to use when distributing your code (see below).
The Python package manager pip
With these preparations in place, we can install
testpack via the Python package manager pip. For this, navigate with a terminal into the folder
testpack!) and type
Here we use
sage -pip to use the version of
pip associated with the Python installation used by SageMath, and
. again stands for the current directory (containing
Install the package
testpackas described above.
Navigate with your terminal into a folder not containing the
Open a SageMath session, run the command
import testpackand execute one of the functions from that package.
Some variants of the command above:
Installs the package in editable mode. This means, if you change the code inside
testpack it will change the behaviour of the module (with code
sage -pip install . above you save a copy of the module at the time of installation, which is not changed when you modify the original code). This editable option is particularly useful if you are still actively working on the code of the package.
If you already have a previous version installed, you can use this to upgrade to the new one.
Once you installed the package, it is also possible to run any of the doctests that you included. You should run the
sage -t command on the folder
testpack containing the code files.
Note that to make this work, your doctests should always start by importing the relevant functions from the package, like the following doctest of
The Python package indey PyPI
One final great feature about pip is that it is by default connected to the Python package index (PyPI). This is an online database of Python packages, and to install a package
foo from there, you just need to type
For our beloved test package above, I did not upload it to PyPI itself, but (following these instructions) I uploaded it to a test-verion of PyPI. Thus, from a console anywhere you can install it using the command
Remark: As you see I had to change the name to
testpackmat007 since, maybe not unsurprisingly, the very creative name
testpack was already taken ...
Summary: the right format for your code
Here we provide again an overview over the strengths and weaknesses of various ways of storing your code:
Jupyter notebooks are nice for short pieces of code with some additional explanatory text and examples
Sage files are great for projects of medium size, where the user wants to load all relevant functions into their session
Python files allow to create a separate namespace, avoiding collisions with user-defined functions, but they require that you write pure Python code and import relevant SageMath functions by hand
Python packages are the professional standard for projects of greater complexity and can use the Python package index for easy distribution and installation
A panorama of further tools for development
When editing your code files, you can use the versioning software git to
keep a record of previous versions of your code
synchronize your local files with some online repository (like gitlab) to allow multiple people to work on them
On platforms like gitlab, you also have access to tools for continuous integration, e.g. a script which runs all doctests of your code whenever you create a new version. In addition, there are tools like airspeed velocity which automatically check the speed of the code on a set of example computations, so that you see whether your changes made things faster or slower.
SageMath core development - a practical example (a.k.a. showing my homework)
Recall from the first lecture the following disappointing performance of SageMath:
The problem was that the class of
S did not have a good implementation of the function
is_integral_domain. I gave myself the exercise to write a suitable version of the
is_integral_domain function, and get it added to SageMath.
Below is the relevant function. Note that in this case
self will be the ring in question, which is of the form where (=
self.base_ring()) is any commutative ring with and (=
self.modulus()) is a nonzero element of with leading coefficient which is a unit in .
After executing the cell below, the rings will know this better function
is_integral_domain, so when you run the cell above again, it will give the correct result.
The abstract process for contributing code to SageMath is described in the official developer guide. Below I give a summary of the concrete steps I had to take:
To change the source code of SageMath, one needs to first download this code, and compile SageMath from it. Many of the usual installations will give you access to a pre-compiled version, which is not enough to do SageMath development. For me, working on Windows, the easiest way to do this compilation was to work with the Windows subsystem for Linux, which is an installation of Linux running inside Windows.
I made changes to the source code (mostly in the file
/src/sage/rings/polynomial/polynomial_quotient_ring.py), adding the function
is_integral_domainabove. Note that the function has a documentation with several examples.
Using this updated version of SageMath, I ran all doctests in the source files of SageMath itself, and encountered an error! It turns out that my changes broke some expected behaviour in an obscure little corner of the software dealing with splitting algebras (whatever that is). This was fixed using the code
Note that I would not have found this in a million years without doctests!
I got some helpful feedback from Vincent Delecroix, one of the main developers of SageMath, and I changed the code accordingly.
Once this review was finished, the code was marked ready to be merged into the main (development) version of SageMath (this happened a few days later).
This means that starting from the next version of SageMath, which is scheduled to release sometime later this year, it will be possible to use the function above.
Recall that in a previous lecture we had some code for a class
Rectangle implementing rectangles with sides parallel to the and -axes. Similarly, one could write a class
Circle implementing circles in the plane (e.g. with given center and radius).
Write a small Python package, containing files
circles.pywhich contain classes for such rectangles and circles. Add methods so that for any rectangle, one can compute the circumscribed circle and so that for any circle one can compute the (unique) "insquare", i.e. the unique square with corners on the circle (and sides parallel to -directions as before). Make sure that at least some of your functions contain documentation with doctests.
Install the package in your system.
Run the doctests.