Debugging Python programs in Spyder

This article gives a brief demonstration of how to use Python's debugger in Spyder. The sample program we will be debugging is a program that draws cards from a deck, with replacement, and counts the number of unique cards drawn.

The main function of our program is the test_with_replacement() function, which uses some helper functions:

• generate_deck(): Return a standard, 52-card deck.
• draw_random(): Return a random card from the specified deck, without making assumptions about the number of cards in the deck.
• draw_card(): Return a specific card from the deck.

The program is designed to illustrate some of the tools available in the debugger, and is not a good example of drawing cards from a deck. The NumPy function numpy.random.choice can sample with and without replacement, and would be better suited in a real world program. It may also look weird that draw_card() is a separate function, but we'll soon see why.

Full program source (download by clicking the deck.py link):

deck.py

# -*- coding: utf-8 -*-
 
import random
 
 
def generate_deck():
    """Return a list containing the cards in a standard, 52-card deck."""
    suits = ["Spades", "Hearts", "Diamonds", "Clubs"]
    ranks = ["2", "3", "4", "5", "6", "7", "8", "9", "10",
             "Jack", "Queen", "King", "Ace"]
    deck = []
    for s in suits:
        for r in ranks:
            name = "%s of %s" % (r, s)
            deck.append(name)
    return deck
 
 
def draw_random(deck):
    """Draw a random card from the given deck."""
    n = len(deck)
    i = random.randint(0, n)
    k = draw_card(deck, i)
    return k
 
 
def draw_card(deck, x):
    """Return card no. x from the deck."""
    return deck[x]
 
 
def test_with_replacement():
    print("Generating deck")
    deck = generate_deck()
 
    number = 10
    print("Drawing", number, "random cards with replacement:")
    drawn = set()
    for x in range(number):
        card = draw_random(deck)
        print("  You drew", card)
        drawn.add(card)
    unique = len(drawn)
    print("Drew", unique, "different cards")
 
 
test_with_replacement()

When run, the program prints:

Generating deck
Drawing 10 random cards with replacement:
  You drew King of Clubs
  You drew 7 of Hearts
  You drew King of Clubs
  You drew King of Hearts
  You drew 6 of Diamonds
  You drew Ace of Diamonds
  You drew 9 of Spades
  You drew 5 of Clubs
  You drew 7 of Diamonds
  You drew King of Hearts
Drew 8 different cards

However, if we run it a few more times, it will crash:

Generating deck
Drawing 10 random cards with replacement:
  You drew Queen of Hearts
  You drew 9 of Diamonds
  You drew 4 of Spades
Traceback (most recent call last):

  File "<ipython-input-23-6d734866f1a9>", line 1, in <module>
    runfile('/Users/perhov/Documents/deck.py', wdir='/Users/perhov/Documents')

  File "/Users/perhov/anaconda/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 710, in runfile
    execfile(filename, namespace)

  File "/Users/perhov/anaconda/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 101, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "/Users/perhov/Documents/deck.py", line 47, in <module>
    test_with_replacement()

  File "/Users/perhov/Documents/deck.py", line 40, in test_with_replacement
    card = draw_random(deck)

  File "/Users/perhov/Documents/deck.py", line 23, in draw_random
    k = draw_card(deck, i)

  File "/Users/perhov/Documents/deck.py", line 29, in draw_card
    return deck[x]

IndexError: list index out of range

OOPS! The program drew 3 cards successfully, then crashed. Why did it crash?

In the following, we'll refer to these tools:

1. The editor: Where you edit the source code.
2. The Python console: Displays the program's output and lets you run python commands interactively.
3. The help window / variable explorer / file explorer: Use the buttons below the window to select which one to use.

When the program crashes, we can enter the Python debugger by typing %debug in the Python console. The prompt changes from:

In [2]:

to:

ipdb>

which indicates that the debugger is running, and that the commands we type from now on should be debugger commands (as opposed to Python expressions to be evaluated).

The program has been "rewinded" to the point where it crashed, and the program's state is preserved. We are therefore able to inspect the contents of the variables used in the program, and we can also see where the function that crashed was called from, and which input it was given.

The debugger shows where the program crashed:

> /Users/perhov/Documents/deck.py(29)draw_card()
     27 def draw_card(deck, x):
     28     """Return card no. x from the deck."""
---> 29     return deck[x]
     30 
     31 


ipdb>

Note that this line also has been highlighted in the editor. While we are debugging, the editor will always show you the line you're currently at:

Why did the program crash in draw_card()? The error message:

IndexError: list index out of range

suggests that we may have tried to access a list past its end. The debugger showed us the location of the crash, which is inside this function:

def draw_card(deck, x):
    """Return card no. x from the deck."""
    return deck[x]

We can view the contents of a variable either by running the debugger command p <variable name>, or by using the variable explorer. The p command can also (within certain limits) print Python expressions. Let's see how large the deck was, and which card we were asked to draw:

ipdb> p deck
['2 of Spades', '3 of Spades', '4 of Spades', '5 of Spades', ..., 'King of Clubs', 'Ace of Clubs']

ipdb> p len(deck)
52

ipdb> p x
52

And there it is! An off-by-one error. Unlike MATLAB and Fortran, Python starts counting at 0. Valid indexes into the deck list is between 0 and 51, endpoints inclusive.

The Variable explorer shows the same information:

But why was draw_card() called with 52 as the argument?

We have determined that draw_card() was called with an argument that doesn't make sense, but why did this happen? The value 52 has been calculated somewhere else in our code, and to find out why and where, we must track down where the function was called from.

A brief digression: Now we're starting to see how useful the debugger can be. It is possible to use print() expressions to trace the contents of the variables, but debugging with print() is:

• More work (you may need to sprinkle print() expressions across large parts of your code).
• More time consuming, especially if your program crashes rarely (you need to add print() expressions first, and then reproduce the error).
• Less flexible (you'll only see the contents of the variables you print, and are unable to view all variables in scope).

To jump to the location from which we were called, we'll use the debug command up. To see where that function was called from, just run up again. To navigate down again in the chain of function calls (this chain is also known as the call stack), use the debug command down.

In the debugger window, we'll type:

ipdb> up

and the debugger now jumps to the place where draw_card() was called:

> /Users/perhov/Documents/deck.py(23)draw_random()
     21     n = len(deck)
     22     i = random.randint(0, n)
---> 23     k = draw_card(deck, i)
     24     return k
     25 


ipdb>

At the same time, the editor is now highlighting the line from which we were called (1), and the variable explorer (2) shows that:

• n = 52
• i = 52

which means that:

• len(deck) returned 52.
• random.randint() was called with the arguments (0, 52).
• random.randint() returned 52.

This tells us that the value 52 stems directly from random.randint(), and it's time to bring out the help function (3):

The help window lets us read the documentation to the functions we're using. We want to know how random.randint are used, and what it does, and we'll type its name in the Object field:

The documentation states that the function takes a closed interval, while we assumed that it took a half-open interval (like range() does).

We found that the cause of the bug is that we're using random.randint() incorrectly in the draw_card() function, and we'll change the line:

i = random.randint(0, n)

to:

i = random.randint(0, n-1)

This fixes the bug, and we can exit the debugger by typing q:

ipdb> q

and the prompt in the Python console has now returned to the familiar:

In [3]: