IPython - an improved shell for Python

IPython is an alternative to the default Python interpreter. It was created to make scripting and prototyping in the programs in the Python language more easy and has gradually developed new technologies to improve Python, as well as other programming languages. Some of its key features are syntax highlighting and tab completion for variables, modules and files in an interactive Python session, and notebooks that allow a mixture of code, output, and descriptions (see for instance this one).

You can install IPython through your package manager (e.g. pip or conda). IPython has loads of options. Although it is well documented, the manual is so long I postponed reading it for years. Now that I did, I decided it was time to share the highlights that improved my workflow.

Basic features

Tab completion

IPython has a well-designed auto-complete feature. By pressing <Tab> while typing a variable name, it will try to complete the variable, giving you a list of options if possible. I mostly use it to see what functions and variables are available in a module I am unfamiliar with.

To obtain the documentation for a function, append ?:

In [1]: np.absolute?

Call signature:  np.absolute(*args, **kwargs)
Type:            ufunc
String form:     <ufunc 'absolute'>
File:            /usr/lib/python3/dist-packages/numpy/__init__.py
Docstring:      
absolute(x[, out])

Calculate the absolute value element-wise.

Parameters
----------
x : array_like
    Input array.

Returns
-------
absolute : ndarray
    An ndarray containing the absolute value of
    each element in `x`.  For complex input, ``a + ib``, the
    absolute value is :math:`\sqrt{ a^2 + b^2 }`.

Examples
(...)

Of course, this requires that the code has been documented. Luckily, for Numpy/Scipy, this is usually the case. I especially like the ‘Examples’ part. Often, I find my solution even faster than with StackOverflow.

Notebooks

The developers of IPython are also responsible for the Jupyter notebook technology, a way to develop and run Python scripts in-browser. By installing and running jupyter notebook in the command line, you launch a graphical user interface in your browser allowing you to create and edit Python notebooks.

Embedding

Another possibility is to use IPython in a regular Python program for ad-hoc debugging. The IPython module provides an embed function. Insert it where you want to examine the program state, run the program (using any Python interpreter), and the script pauses after embed() and launches a standard IPython session, loaded with current state of the program. You can explore and modify the variables and their contents, and when you exit the interactive session, the program continues from where it left off. It is no replacement for advanced debuggers, but it provides a lot of flexibility for experimentation.

Check for instance this example program test_embed.py:

from IPython import embed
import numpy as np
def symmetric_part(array):
    array = (array + array.T)/2
    embed()
    return

arr = np.array([[1,2.5],[2,3]])
symmetric_part(arr)

If we run this program in the command line, we get

python3 test_embed.py
In [1]: print(array)
[[ 1.    2.25]
 [ 2.25  3.  ]]

Configuration options

IPython has many, many configuration options. By default, it uses a standard configuration profile, found in $HOME/.ipython/profile_default/ipython_config.py. ($HOME is your home folder). If there is no file present, or you want to reset your configurations, run

ipython profile create default

If there is something about IPython that is bothering you, it is worth the time to check out the list and see what alternatives are available. Three important ones:

Auto-exit on EOF

When you exit the shell by pressing <Ctrl-D>, by default IPython asks for a confirmation. To disable this (and yes, save a single keystroke), set c.TerminalInteractiveShell.confirm_exit = False.

Load script before prompt

IPython provides the option to load Python scripts every time IPython starts. I have one that contains my most used imports and variables:

 import numpy as np
 import scipy as sp
 import matplotlib.pyplot as plt

l = np.linspace(0,10)
f = np.random.random([4,4])

Set editor in IPython

If you find yourself dealing with a particularly difficult (multiline) command, you can open your favorite editor to simplify your tinkering. Set c.TerminalInteractiveShell.editor='editor', where ‘editor’ is the name (executable) of the editor of choice. Vim, gedit, Atom, Sublime are all possible options. After saving and closing the editor, the command is run in your session.

Magic functions

IPython comes with a set of ‘magic’ functions that simplify certain commonly encountered problems They are preceded with one or two %, depending whether they work on one line or on an entire cell. Most of the magic functions work both in the interactive shell as the notebook environment

%paste

Paste and run the text in your clipboard. Convenient, because the indentation can get messed up if you use manual pasting.

%run script.py

Runs the file script.py in the interactive session. All variables and loaded modules remain in the namespace. If you are testing new modules or functions, it works better then import script.

%timeit function(arg)

Runs the method function with arguments arg many times and reports on the execution time. An example:

 In [1]: import numpy as np;import math

In [2]: %timeit np.exp(2)
973 ns ± 40.9 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [3]: %timeit math.exp(2)
116 ns ± 0.633 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [4]:

%matplotlib

Allows for interactive plotting with matplotlib. Normally, after plotting a graph, a call to plt.show() is necessary to display the graph. Also, the Python program/session will be suspended until the window is closed. By calling %matplotlib, both of these are no longer the case.