Test anything (with Perl)

Erwan Lemonnier

Premiepensionmyndigheten


$Revision: 1.3 $

Ever wrote some code?

So...

How do you know it works?

Wrong answer 1

"I never write bugs"

Wrong answer 2

"I have read the code over and over X times"

Wrong answer 3

"It compiles without warnings..."

Wrong answer 4

"It runs without crashing"

Wrong answer 5

"I have a team of testers testing it for me"

Getting closer

"I wrote unitests for every method/function/combination of input argument. I have 100% test coverage and 1 billion beta testers running on every existing operating system and sending me automated feedback. 10000 programmers are reviewing every single release of my code. All tests are automated and run periodically."

The shocking truth

You cannot prove code to be bugfree

Testing software

Testing code is hard (1/4)

Code follows a specification

Testing code is hard (2/4)

Code evolves

Testing code is hard (3/4)

Code has changing dependencies

Testing code is hard (4/4)

Developers evolve

Mastering change

Test must be automated and reproducible.

Software must be fully tested each time:

How?

Writing tests...

...is easier than you think.

let's illustrate with Perl...

Perl!!

TAP

Test Anything Protocol

Can test:

Example

let's test /bin/mv

Example: testing /bin/mv

    $ perl test_mv.t
    1..3
    ok 1 - error if no arguments
    ok 2 - old file removed
    ok 3 - new file created

	

test_mv.t (1/2)

#!/usr/local/bin/perl
use strict;
use warnings;
use Test::More tests => 3;
	

test_mv.t (2/2)

ok(`mv 2>&1` =~ /missing file argument/,
   "error if no arguments");
die if (-e 'foo');
`touch foo`;
`mv foo bar`;
ok(!-e 'foo', "old file removed");
ok(-e 'bar', "new file created");
unlink 'bar';
	

Better tests... (1/2)

Better tests... (2/2)

Code coverage

Regular test runs (1/2)

Regular test runs (2/2)

> prove *.t
test_cp....ok
test_mv....ok
All tests successful.
Files=2, Tests=6,  0 wallclock secs ( 0.10 cusr +  0.16 csys =  0.26 CPU)

prove.sh

# run perl tests in DIRTESTS, save results in DIRLOG
# and send email alarm if tests fail
LOG=$DIRLOG/my_tests_`date +%y%m%d.log` # osx only
cd $DIRTESTS
prove *.t | tee $LOG

if [ `cat $LOG | grep PASS | wc -l` == '0' ]; then
    (   echo "FROM: prove"
	echo "TO: you@somewhere.com"
	echo "SUBJECT: test failed"
	cat $LOG ) | sendmail -t
fi

Regular test runs

Continuous integration

Thinking Agile

More Examples

testing a web server

test a web server (1/3)

test a web server (2/3)

use Test::More tests => 26;
use WWW::Mechanize;

my $m = WWW::Mechanize->new();

$m->get( "www.foo.com/login.html");
ok($m->success, "got login page");
	

test a web server (3/3)

$m->submit_form(
       form_name => 'login',
       fields    => {
           username => 'bob',
           password => 'secret',
           }
       );
       
is($mech->title, "Welcome bob", "logged in");
...
	

More Examples

Test a shared C lib (test_strcat.t)

use strict;
use warnings;
use Test::More tests => 1;

use Inline C =>
    Config =>
        LIBS => '-lm',
        ENABLE => 'AUTOWRAP';
	
Inline->import( C => <<END_HEADERS );
char* strcat(char*,const char*);
END_HEADERS

ok(strcat("foo","bar") eq "foobar");
	

Test a shared C lib

> perl test_strcat.t
1..1
ok 1
	

Advertisment

For more: Perl Testing: A Developer's Notebook (O'Reilly)

The bleeding edge

Some things are almost impossible to test...

Questions?

Thank you!