Whats wrong with my Perl?

So this evening I was making a bit of junk data to time function runs over in Lisp, and for a gag I wrote it in Perl and raced the two of them. Shockingly Perl lost, so the question is, what am I doing wrong with it.

The Lisp

So first heres the lisp

(defun big-path-gen (length)
  "Generate a long list of length sublists, of alternating '(a b c) '(a b c d) form to 
give max-len something to nuke"
  (loop for x from 1 to length 
        collect (if (evenp x)
                    '(a b c)
                  '(a b c d))))

(defun max-len (len queue)
  "Delete sublists from queue that are longer than len"
  (delete-if (lambda (p) (> (length p) len)) queue))

I also wrote a recursive version of big-path-gen, but its performance was horrific (possibly because I sucked and used append instead of nconc because I couldn’t be bothered fighting nconc inside a recursive function).

Results from Lisp
Now keep in mind that this is SBCL 1.0.15 running through SLIME and Emacs. Results were gathered with interactions like this:

(time (length (max-len 3 (big-path-gen 50000))))

Length real time user run time system run time GC run time
50,000 0.004 0.004001 0 0
1,000,000 0.15 0.144009 0 0.036
10,000,000 1.373 1.244078 0.128008 0.612

The Perl

Now I admit this is shoddy and was just knocked out to test a passing interest:


use Benchmark;

sub big_list_gen {
  $length = shift or die "No argn";

  for ( 1 .. $length ) {
    if ($_ % 2 == 0) {
      push(@arr, [qw(a b c)]);
    else {
      push(@arr, [qw(a b c d)]);

  return @arr;

sub max_len {
  $max = shift;
  $arr = shift;

  for($i = 0; $i [$i] }) > $max) {

sub print_arr {
  @arr = @_;
  for($i = 0; $i <= $#foo; $i++) {
    print "DD: $i " . join(',', @{ $foo[$i] }) . "n";

our $amount = shift or die "Can't work out the correct amountn";

$times = 1;
$t = timeit($times, sub {
      @foo = big_list_gen($amount);
      max_len(3, @foo);
print "Took " . timestr($t) . "n";

The Results from Perl

Length Wallclock user run time system run time CPU time
50,000 0 0.09 0.01 0.10
1,000,000 2 1.88 0.17 2.05
10,000,000 22 19.05 2.24 21.29

So the question stands, what was I doing wrong with Perl? Should I be using something other than push() to join the new anonymous elements on? Should I not pass a reference to max_len() ? Would pass by copy be faster? Is there a penalty for list counting like that inside the forced array context @{ } ? Should I deref it to a local var and just check s#var?

Step 2 will be to benchmark the creation vs. creation and the trimming vs. trimming and use DProf on the Perl, but its late so check back tomorrow morning for an edit.

Answers on a postcard please.

7 thoughts on “Whats wrong with my Perl?

  1. captain_carrot

    Where is foo defined within print_arr?

    It’s been too long since I did any lisp. I had to look at the perl to work out what the lisp was doing.

    Is there any particular reason why you’ve used a for loop testing against the array length every time instead of a foreach my $subarr (@{$arr}) or whatever? Cos if there is I should probably go change some of my code for NAC. If you’re going to use a for I would be inclined to save the array length in a variable you can reference every time, instead of doing a new lookup every time you have to check it. But I suspect any difference would be negligible, and I don’t know how perl stores arrays internally so it might make no diff, as it might just be looking at some kind of array header to find the length anyway.

    Why are you putting qw( whatever ) inside []? I believe you should just be able to push(@arr, qw(a b c)).

    I’d actually be inclined to return a reference to the array in big_list_gen and see if that makes a difference. All value passing between functions in perl is scalar, so if you’re passing a big array directly it’s passing a big whack of values one a time. I suspect it might make a fair difference if you pass references around instead, for such a big array.

    1. captain_carrot

      Also as a personal preference I tend to use scalar(@arr) or just use @arr in a scalar context like $i < @arr to get the length of an array, rather than $#. Just remember that $# returns the index of the last element, whereas @ in a scalrar context actually returns the number of elements.

      If i'm reading it right that measn you actually need

    2. mostlyfoo

      As a first quick response foo isn’t defined within print_arr, I was writing print_arr as a checking function but wasn’t paying attention when I yoinked code into it from a quick test, good spot.

    3. mostlyfoo

      The original version of the script has the following runtime on my machine:
      Took 21 wallclock secs (17.91 usr + 2.90 sys = 20.81 CPU) @ 0.05/s (n=1)

      returning @arr from big_path_gen shaves it down to this:
      Took 16 wallclock secs (13.83 usr + 2.03 sys = 15.86 CPU) @ 0.06/s (n=1)

      So its worth a good 5 seconds of time.

  2. captain_carrot

    Don’t mind me. I’ve realised why you’re not using foreach. Foreach steps into any subarrays, instead of returning the actual array. That’s why i’m using it. I have an array or arrays but I want to treat it like 1 flat array. You don’t. Also i’d have to check, but push(@arr, qw(a b c)) might append a b c to the main array instead of creating a sub array.

    My point about passing a reference to the array instead of a fecking huge array itself still stands though.

    1. mostlyfoo

      Yeah the returning of @arr should really be @arr I suspect as otherwise it’ll have to copy all that crap, be interesting to see what a speed boost this gives.

      The real reason I was using for() instead of foreach() was I was rushing at about 10:45 to knock this out just for curiosity and I got mentally confused about the prospect of $#$_ although I could have used the exact same trick as I actually did I think.

      I’m fairly sure that since it was an array of array-references that foreach would have returned an ARRAY ref() for each step so it should have worked, time for some testing possibly.

      1. mostlyfoo

        oh wait, that wasn’t it, it was because I couldn’t workout how to delete from a foreach(), I’ve just recalled as I just hit the same issue.


Leave a Reply