Python testing with doctest and unittest

The problem:

We want to create a function that is able to determine whether a string is a palindrome.

A palindrome is a word that is read the same from front-to-back and from back-to-front.

For instance, the words "noon" and "racecar" are both palindromes.

Algorithm:

Reverse the string.

Compare the reversed string to the original string.

Recipe for designing functions:

  1. Give the function a name
  2. Examples
  3. Type Contract
  4. Header
  5. Description
  6. Body
  7. Test

First, we will see the python module doctest which is good for having an up to date documentation.

But using doctest is not recommended for testing a whole program. This is why we will see the python module unittest which is more relevant.

For more about the cons of using doctest: Narrative tests are lousy tests and Tests are code and doctests are not.

Create the file palindrome.py:

#! /usr/bin/python2.7

def is_palindrome(s):
    """(str) -> bool

    Return True if and only if s is a palindrome.

    >>> is_palindrome('noon')
    True
    >>> is_palindrome('racecar')
    True
    >>> is_palindrome('dented')
    False
    """
    return reverse(s) == s

def reverse(s):
    """(str) -> str

    Return the reversed string.

    >>> reverse('hello')
    'olleh'
    >>> reverse('a')
    'a'
    """
    rev = ''
    # for each character in s, add the character at the beginning of rev.
    for char in s:
        rev = char + rev
    return rev

import doctest
doctest.testmod()

We want to separate the tests from the main program and use the python unittest module.

Create the file test_palindromes.py

#! /usr/bin/python2.7

import unittest
import palindromes


class TestPalindrome(unittest.TestCase):

    def test_is_palindrome_true(self):
        value = palindromes.is_palindrome('racecar')
        self.assertEquals(value, True)

    def test_is_palindrome_false(self):
        value = palindromes.is_palindrome('dedent')
        self.assertEquals(value, False)

    def test_reverse_normal(self):
        value = palindromes.reverse('hello')
        self.assertEquals(value, 'olleh')

    def test_reverse_error(self):
        list_of_bad_value = [
        123,
        None,
        ]
        for bad_value in list_of_bad_value:
            self.assertRaises(
                TypeError,
                palindromes.reverse,
                bad_value
            )

if __name__ == '__main__':
    unittest.main()

Choosing test cases to test

  • Size: For collections, test with an empty collection, a collection with 1 item, the smallest interesting case, and a collection with several items
  • Dichotomies: Vowels/Non Vowels, even/odd, positive/negative, empty/full, and so on.
  • Boundaries: If the function behaves differently for values near a particular threshold, test at that threshold.
  • Order: If the function behaves differently when the values are in different orders, identify and test each of those orders.

Source: https://class.coursera.org/programming2-001

Context Manager in Python

The keyword with

The context manager is going to be used with the keyword with such as:

with MyContextManager as context_var:
    # here the statement is going to evaluate the methods
    # __enter__ and __exit__ of the class MyContextManager.
    # The value returned by __enter__ is going to be assigned
    # to the variable "context_var".

The context manager can be defined as a class

Two essential methods are required for making the class a context manager:

  • __enter__(self)

    Defines what the context manager should do at the beginning of the block created by the with statement. Note that the return value of __enter__ is bound to the target of the with statement, or the name after the as.

  • __exit__(self, exception_type, exception_value, traceback)

    Defines what the context manager should do after its block has been executed (or terminates). It can be used to handle exceptions, perform cleanup, or do something always done immediately after the action in the block. If the block executes successfully, exception_type, exception_value, and traceback will be None. Otherwise, you can choose to handle the exception or let the user handle it; if you want to handle it, make sure __exit__ returns True after all is said and done. If you don't want the exception to be handled by the context manager, just let it happen.

Example

class MyContextManager(object):
    def __enter__(self):
        print '__enter__()'
        return self
    def __exit__(self, exception_type, exception_value, traceback):
        print '__exit__()'

with MyContextManager() as context_var:
    print 'Doing work in the context'

Where the output is:

__enter__()
Doing work in the context
__exit__()

The Context Manager can also be defined as a Generator

Here, we use the contextlib library and the module contextmanager.

The equivalent of the example above would be:

import contextlib

@contextlib.contextmanager
def mycontextmanager():
    print "__enter__"
    try:
        yield
    finally:
        print "__exit__"

with mycontextmanager() as context_var:
    print 'Doing work in the context'

First we use the decorator @contextmanager to indicate to Python that the function will be a context manager.

Then, we do a try/finally (it is not automatic like with __enter__ and __exit__).

The word yield split the code in two parts:

  • everything that is before yield is similar to what we had above in __enter__
  • everything that is after is similar to __exit__

The content (the returned value) of yield is taken in the variable 'context_var' here defined with the key word as.

Grep aliases

In .bashrc

# recursive search in files

function grp {

    GREP_OPTIONS="-rI --color --exclude-dir=\.bzr --exclude-dir=\.git --exclude-dir=\.hg --exclude-dir=\.svn --exclude=tags $GREP_OPTIONS" grep "$@"

}



# recursive search in Python source

function grpy {

  GREP_OPTIONS="--exclude-dir=build --exclude-dir=dist --include=*.py $GREP_OPTIONS" grp "$@"

}

Use:

$ grp WORD .

Or

$ grpy to just search in python files

Then reload bashrc with:

$ . ~/.bashrc

Thank you JB!! (http://tartley.com/)

Monkeypatching in python

A MonkeyPatch is a piece of Python code which extends or modifies other code at runtime (typically at startup). MonkeyPatching would be the practice of writing, or running, a monkeypatch.

A simple example looks like this:

from SomeOtherProduct.SomeModule import SomeClass
def speak(self):
    return "ook ook eee eee eee!"

SomeClass.speak = speak

In this example, if SomeClass did not already have a speak() method, it does now :-) If it had a speak() method before, the new code has replaced the old method definition.

Sources:

Monitor disk usage with python

The module to use in python2 is statvfs imported from os

import os
statvfs = os.statvfs('/')
statvfs.f_frsize * statvfs.f_blocks     # Size of filesystem in bytes
statvfs.f_frsize * statvfs.f_bfree      # Actual number of free bytes
statvfs.f_frsize * statvfs.f_bavail     # Number of free bytes that ordinary users

NB: it is deprecated in python3, the module to use will be fstat os.fstat

Source:

Better usage:

Send email with python

This is a simple snippets for sending an email with a gmail smtp server.

If you want to send emails from localhost, you will have to first enable an smtp server.

So, the simplest is to use gmail smtp server:

import smtplib

# define the method
def sendemail(from_addr, to_addr_list, cc_addr_list,
              subject, message,
              login, password,
              smtpserver='smtp.gmail.com:587'):
    header  = 'From: %s\n' % from_addr
    header += 'To: %s\n' % ','.join(to_addr_list)
    header += 'Cc: %s\n' % ','.join(cc_addr_list)
    header += 'Subject: %s\n\n' % subject
    message = header + message

    server = smtplib.SMTP(smtpserver)
    server.starttls()
    server.login(login,password)
    problems = server.sendmail(from_addr, to_addr_list, message)
    server.quit()

#execute the method
sendemail('me@domain.com', ['you@yourdomain.com'], ['andyou@yourdomain.com'],
          'test subject', 'test message', 'mylogin@gmail.com', 'mypassword')

Source:

Sentry installation

Create a directory called sentry:

# mkdir /opt/sentry && cd /opt/sentry

Create a virtual environment and activate it:

# virtualenv sentry && source /opt/sentry/sentry/bin/activate

Install Sentry:

# easy_install -UZ sentry[postgres]
# easy_install -UZ sentry

Initialize the conifguration

# sentry init /opt/sentry/sentry.conf.py

In the configuration file /opt/sentry/sentry.conf.py, you are going to change:

  • the database configuration such as:
DATABASES = {
  'default': {
                'ENGINE': 'django.db.backends.postgresql_psycopg2',
                'NAME': 'sentry',
                'USER': 'sentry',
                'PASSWORD': 'sentry',
                'HOST': 'localhost',
                'PORT': '',
             }
           }
  • And the url:
SENTRY_URL_PREFIX = 'http://192.168.50.4:9000' # the ip address of the server

Install postgresql

# apt-get install postgresql-9.3

Create a user for sentry:

# sudo su postgres
$ createuser sentry
$ psql template1
# alter role sentry with password 'sentry';

Create the database as postgres:

# createdb -E utf-8 sentry

Create the initial schema using the upgrade command as root:

# sentry --config=/opt/sentry/sentry.conf.py upgrade

If you did not create the user on the first run, you can correct this by doing the following:

# sentry --config=/opt/sentry/sentry.conf.py createsuperuser
# sentry --config=/opt/sentry/sentry.conf.py repair --owner=sentry

You can start Sentry with:

# sentry --config=/opt/sentry/sentry.conf.py start