erwan's code blog

A few days ago, I wrote about some ideas on how to adapt Python decorators to Perl.

Well, the ideas grew and soon enough I couldn't keep them in my head anymore so it became a module: Python::Decorator, now available on CPAN. This module let's you write Python style decorators in Perl, so that the following Perl code for example would compile and do what you expect:

use Python::Decorator;

# memoize incr's results, print debug info upon
# calling incr and check that the first argument
# is an integer:

@memoize
@debug("incr")
@validate("int")
sub incr {
   return $_[0]+1;
}

Of course, Python::Decorator is just a proof of concept. It shows that compile-time function composition (aka macros) can be done in Perl, at least in some restricted way. If such a feature makes it into the Perl core, it will have to have at least a more perlish syntax.

Still, Python::Decorator is fun for at least one more reason: it uses PPI, the Perl parsing module, to analyze and manipulate the source code of Perl programs. Now, that is cool and a lot like macros :)

I have been playing around with Python lately and this evening I stumbled upon this blog entry in which Bruce Eckel gives an excellent introduction of a Python feature called a 'decorator'. Decorators in Python are very similar to LISP macros so, of course, I got interested :)

Simply put, decorators are a way to compose functions at compile time. In the rest of this post, I will assume you know what a Python decorator is, and if you don't, just go and and read the aforementioned blog.

Bruce Eckel gives the following example of a Python decorator that wraps a function into an other function that prints a short message upon entering and exiting the original function. In Python, it looks like this:

class entryExit(object):

      def __init__(self, f):
            self.f = f

      def __call__(self):
           print "Entering", self.f.__name__
           self.f()
           print "Exited", self.f.__name__

@entryExit
def func1():
      print "inside func1()"

@entryExit
def func2():
      print "inside func2()"

func1()
func2()

Now, being a Perl addict to the bone, I couldn't help meditating on how to get a similar feature in Perl. And it's not so hard, really. Wrapping a sub within an other one can be done with a closure and some symbol table hacking. Note that I kind of ignore the issue of identifying the function's name, which is kind of not relevant to the discussion. Here is a similar implementation in Perl:

sub entryExit {
      my ($f,$name) = @_;
      return sub {
            print "Entering $name\n";
            &$f();
            print "Exited $name\n";
      };
}

sub func1 {
      print "inside func1()\n";
}
# we can replace func1 with the closure afterwards
*{__PACKAGE__."::func1"} = entryExit(\&func1,"func1");

# or we could replace "sub func2 {" with the following:
*{__PACKAGE__."::func2"} = entryExit(sub {
      print "inside func2()\n";
},"func2");

func1();
func2();

Clearly, none of those 2 methods is as smooth as the Python syntax. But with a bit of fantasy I am pretty sure we could write a Perl source filter that let us write:

@my_decorator
sub func {
     # do stuff
}

and replace it with:

*{__PACKAGE__."::func"} = my_decorator(sub {
     # do stuff
});

As a matter of fact, I know this can be done since I have done it partially in a prototype of a module that would have been called 'Filter::WrapSub' and can be found within the sub-contract project on sourceforge. This proof-of-concept module is basically a source filter that uses PPI to *understand* the source code of the calling file and identify the beginning and end of subroutine declarations. The big drawback of this method is that it is SLOW. really slow.

But nonetheless, it can be done. Supporting decorator arguments wouldn't be a problem either. The real limitation compared with Python is the absence of a clean object model in Perl which makes it meaningless to implement the equivalent of Python's decorator classes in Perl...

At last! Today is the first day of this year's Nordic Perl Workshop. About 50 attendees, the highest number ever reached in Sweden!

I am pretty sure me and Claes have forgotten tons of things we should have thought of. From tee mugs to ethernet cables, from printed copies of the schedule to coffee brewer and name lists, there is enough place for misses.

But the conference starts today, and I am the first speaker, which is nice in a sense: I won't have to bother about what I'll have to say afterward and be free to listen with full attention to all the others! And talk with fellow developers :)

What an exciting day it's gonna be!
Well, time to shower. No need to shave :)

Lately I have been helping Claes Jakobsson to prepare the 6th edition of the Nordic Perl Workshop, to be held in Stockholm this year, at the end of may: http://conferences.yapceurope.org/npw2008/.

The Nordic Perl Workshop is a kind of lightweight yearly conference gathering Perl developers from north european countries (and further away) to discuss and code Perl.

I am just done with something we could use as a logo: a runic camel!!!



It says 'Nordic Perl' on its back :)

If I say "floating point arithmetic", is your hair rising on your head?

If it is, you probably have experienced the kind of precision issues in arithmetic calculations that I have been troubled with lately.

If it isn't, take a long break and read about floating point and IEEE 754.

As a note to self and anyone interested, I thought I would gather in one post a few techniques that are useful to access the inner representation of numbers in perl.

Let $a be a scalar holding a number:

$a = 1.73696;

In perl, a scalar value (SV) is somewhat of an hybrid beast, and is really up to 3 different things at the same time, baked together within the SV structure (see perldoc perlguts).

An SV can contain an integer value (IV), a native float (NV) or/and a string (PV). Perl converts one representation into one of the others transparantly, depending on the operations performed on that SV.

To see what $a contains, you can use Devel::Peek:

use Devel::Peek;
my $a = 1.73696;
Dump($a);

This will output:

SV = NV(0x8f58810) at 0x8f4993c
   REFCNT = 1
   FLAGS = (PADBUSY,PADMY,NOK,pNOK)
   NV = 1.73696

And you can see that $a is a scalar value (SV) holding only 1 representation, a natural float NV. But what Devel::Peek shows as being the value of this NV is more or less just a printf. You may want to know exactly what this float's sign, exponent and significand are. Data::Float does it for you:

use Data::Float qw(float_parts);
my ($s,$e,$f) = float_parts($a);
print "sign: $s\n";
print "exponent: $e\n";
print "significand: $f\n";

And the result is:

sign: +
exponent: 0
significand: 1.73696

Finally, you may want to know the exact binary representation of this float. Here is how you could do it:

my $bin = unpack("B64",reverse pack("d",$a));
print "sign: ".substr($bin,0,1)."\n";
print "exponent: ".substr($bin,1,11)."\n";
print "significand: ".substr($bin,12)."\n";

Resulting in:

sign: 0
exponent: 01111111111
significand: 1011110010101001011010010001101001110101110011010001

Note that this works for 64bits floats. You may have to adjust this code on platforms where floats are not 64 bits long.

How can you list all the subroutines defined in a Perl package, at runtime?

Trying to answer that question, I googled a bit, digged around on cpan, and stumbled upon a few answers. The first one comes from perlmonks and looks like this:

sub get_function_names_from_package {
     no strict 'refs';
     my $pkg = shift || caller;
     my $symtab = *{ $pkg . "::" }{HASH};
     grep *$_{CODE},
          map $pkg."::$_",
          grep !/^_|^[_A-Z]+$/,
          keys %$symtab;
}

What this code does is to list all the symbols belonging to a given package, filter away those that start with _ (Perl's standard for private subs) or only uppercase (private globals), then those that does not refer to functions. Very perlish and hard to read ;)

Then I found an other variant in a module called Sub::WrapPackage, that boils down to:

sub get_function_names_from_package {
     my $pkg = shift || caller;
     my @subs;
     no strict;
     while(my($name, $v) = each(%{$pkg})) {
          push @subs, $pkg."::".$name if(defined(&{$v}));
     }
     return @subs;
}

Cleaner code. It does not filter out private subs, but that can be easily added.

Then, we have the hardcore way, extracted from the guts of the module Pod::Coverage:

use Devel::Symdump;
use B;

sub get_function_names_from_package {
     my $pkg = shift;
     my @subs;

     eval qq{ require $pkg };

     my $syms = Devel::Symdump->new($pkg);

     for my $sym ( $syms->functions ) {

         # see if said method wasn't just imported from elsewhere
         my $owner;
         {
             no strict 'refs';
             # this works only with perl 5.8 and later
             $owner = *{ B::svref_2object( \&{ $sym } )->GV->object_2svref };
             $owner =~ s/^\*(.*)::.*?$/$1/;
         }

         next if ($owner ne $pkg);

         $sym =~ s/^$pkg\:\://;
         next if ($sym =~ /^_/);

         push @subs, $sym;
     }

     return @subs;
}

Make your choice...

I am currently in Copenhagen, Danmark, just after attending the 5th edition of the Nordic Perl Workshop.

Perl conferences are always very stimulating: it may sound self evident to just say that you get to meet other programmers who share your interest, get geekish with them and enjoy it a lot, but it really is the main point with such conferences! In a world where dynamic languages are still relegated to university studies and small hacking projects, it's such a relief and a joy to meet others who actually understand the mind-blowing power of dynamic syntaxes and love to talk about it. You can actually chat with the guys who wrote the books you learned the language with. Even better, you can talk with the guys who actually created the language! Now, those people have insight! and they are usually more than happy to share it, to everyone else's delight.

But that's not what I thought of writing about.

While listening to a couple of talks about Perl6 yesterday, I came to rethink about an article I read recently: Style is substance by Ken Arnold. The main point in this article was that every software project would win in the long term if programming languages were by design restricted to allow one and only one valid syntax.

For example, instead of allowing all the following syntaxes as valid in a given language:

     if () {
     } else {
     }

     if()
          {
          }
     else
          {
          }

     if () {
     }
     else {
     }

you would just pick up on of them as valid, and hardcode it into the compiler so that only that syntax actually compiles.
Which syntax you would take just does not matter. Seriously.

Think of all the wasted time and programming resources that would suddenly be saved! No more arguing at length about which alternatives should be allowed in a specific project's code standard, or spending time developing proper syntax highlighting in text editors, efficient parsers for compilers, and so on and so forth.

At first, I thought it was a great idea.

Then I sat listening to Jonathan Worthington's talk about the design of classes, roles and constraints in Perl6, and I came to reconsider it all.

Just imagine that Perl's syntax had been closed by design from the beginning. Static. No way to mutate and evolve. What would have happened? Obviously, Perl would have died within years.

The fact is that programming paradigms do evolve. With years passing, new approaches emerge to solve known programming problems, new designs emerge. Think of the way object-orientation emerged, and design patterns, and dynamic programming with things like meta-programming or weak typing. Those new ways of seeing things required new ways of expressing yourself in the code, new syntaxes. How would those new syntaxes have been engineered if the syntax of languages had been all static from the very first release? How would new languages have appeared? Would they have to be engineered to perfection from the very beginning they get released, then break backward compatibility each time someone wants to introduce a new syntax into the language? I am sceptical.

The fact is, much of the durability of Perl comes from its amazing flexibility, the way it lets you redefine the very syntax of the language to try out new syntaxes, new approaches. It's almost an evolutionary process: new features are made possible via language modules and pragmas. Some never catch people's interest and vanish. And some make it into the language's core.

Later, when the core has evolved beyond maintainability, you just refactor the whole language. That would be Perl6 :)

I was just reading about duck typing on wikipedia.

Duck typing is a technique used in object oriented languages in order to identify the type of objects not based on which class they instantiate or inherit from, but based on which methods they implement. Or put in other words, duck typing is about not giving a damn about what an object *is*, and caring only about what an object *does*.

Duck typing brings languages closer to having true dynamic typing, with the flexibility that comes along.

Some languages use duck typing as their inner mechanism for method lookup: they know which methods every object implements and don't even care for the object's type. And with some other languages (Java for example) you can emulate duck typing by using introspection.

Which brings me to Perl...

Perl has introspection: you can call the 'can()' method on any object. So you can do duck typing. But I have difficulties seeing duck typing used on a larger scale in Perl.

The main issue is the lack of standard interfaces in Perl. To use duck typing to manipulate different objects *as if* they were related by way of a class hierarchy, they still need to implement the same interface. If you implement the objects' classes yourself, it will be fine. But if you fetch them from for example CPAN, you won't get any two similar interfaces... That's due to lack of naming conventions for Perl modules. Still, it may be better to write a couple of proxy classes to abstract away the difference between objects instanciating various classes, rather than having a monstruous if-elsif-elsif-elsif... in the middle of it.

Then, there is the issue of code robustness: how can you prevent someone to later on implement an object that quacks like a duck, walks like a duck, but is definitely NOT a duck? And what will happen when this object lands in a method that uses duck typing to manipulate objects? Probably nothing good...

Now I wonder how Perl6 will implement method lookups? Mmmm... nice reading ahead...

I just came across a beautiful piece of code while looking at the Perl module 'mixin' on CPAN. It boils down to:

sub mixer {
     my ($mixin,$mixed_with) = @_;

     require base;

     eval sprintf q{
         package %s;
         base->import($mixed_with);
     }, $mixin;
}

When called, the function mixer will alter dynamically the inheritance of the class $mixin, adding the class $mixed_with to the list of its parent classes.