Building Dist::Zilla Distributions with BuildBot

Posted on by

Recently I have been using Buildbot for continuous integration for a number of my open source projects, including some Dist::Zilla based Perl projects (see http://ci.arjones.co.uk/).

Buildbot is very customisable, so I have created some custom build steps for Dist::Zilla based projects. The main benefit is instead of seeing this in your waterfall.

Before Dist::Zilla Steps

You see this.

After Dist::Zilla Steps

OK, so not too ground-breaking, but then it wasn’t a lot of work :-) .

To use it, first download or clone the code and put it somewhere near your buildmaster.

Then, you can use something like the following in your Buildbot configuration file.

from dzil import DzilDependencies, DzilSmoke
 
factory.addStep(DzilDependencies())
factory.addStep(DzilSmoke())

Currently the following steps are implemented:

  • DzilAuthorDependencies
  • DzilDependencies
  • DzilSmoke
  • DzilTest

For more information, see the Github page. If you like, you can also see my configuration files, which I will explain in a future post :-) .

Get all documents from your KinoSearch1 index

Posted on by

I wanted a script to dump the entire contents of my KinoSearch1 index into a flat file. Turns out this was pretty easy by using the KinoSearch1::Index::IndexReader.

The following Gist shows how I was able to do this.

#!/usr/bin/perl
use strict;
use warnings;

use KinoSearch1::Index::IndexReader;

my $r = KinoSearch1::Index::IndexReader->new( invindex => '/path/to/index' );

# get one or more readers
my @readers = ref $r->{sub_readers} eq 'ARRAY' ? @{ $r->{sub_readers} } : $r;

for my $reader (@readers) {
    print "Segment "
      . $reader->get_seg_name . " has "
      . $reader->max_doc
      . " docs\n"; 
    
    # the index is numbered from 0 to max_doc, so just loop
    # through them and get the documents
    for(my $i = 0; $i < $reader->max_doc; $i ++){
        # A KinoSearch1::Document::Doc object
        my $doc = $reader->fetch_doc($i);
        
        # this will depend on how your index has been set up
        my $title = $doc->get_value('title');
        print "$title\n";
    }

}
print "Total documents: " . $r->max_doc . " in " . @readers . " segments\n";
view raw kinosearch1_dump.pl This Gist brought to you by GitHub.

Pretty self-explanatory. Let me know if you have any questions.

Ant Task to generate an XML Sitemap

Posted on by

I use Ant for more than just Java projects, including building my own website. So I have written an Ant task to generate an XML Sitemap and released the first version.

Here is an example of using the task:

<target name="generate_sitemap" description="generates the sitemap">
    <taskdef name="sitemap" classname="uk.co.arjones.ant.task.Sitemap" />
    <sitemap url="http://andrew-jones.com" destdir="${BUILD_DIR}" lastmod="now" gzip="yes">
        <fileset dir="${BUILD_DIR}">
            <include name="**.htm"/>
            <include name="**.html"/>
            <exclude name="google*"/>
        </fileset>
    </sitemap>
</target>

Its pretty self-explanatory. For more information and to download it, go to the project page on GitHub.

Perl, Wildcards and the Windows Command Line

Posted on by

When developing for multiple platforms, its always the little things that catch you out.

Heres one example I found while developing gcal, a Command Line Interface to Google Calendar. On Unix (including Mac OS X shells) wildcard expansion is done by the shell, and then the expanded list of files is passed on to the program being run. On Windows, the shell doesn’t do any expansion, and it’s up to programs to do the expansion[1].

To get around this in Perl, you can do something like this[2].

my @args = ($^O eq 'MSWin32') ? map { glob } @ARGV : @ARGV;

Here we are passing all the arguments to glob, which will do the shell expansions, including wildcards. We only want to do this for Windows, as if you use it on Unix you risk globbing real file names – asterisks and question marks are valid characters in Unix file names.

Related Posts

References

  1. The difference in wildcard expansion between Windows and unix/Macintosh [↩]
  2. How can I handle wildcards on the command line using Perl on Windows? – Stack Overflow [↩]

New Version of gcal Released

Posted on by

I have released version 1.120110 of gcal, a Command Line Interface to Google Calendar. It has the following enhancements:

  • Support wildcards on Windows (i.e. gcal *.ics)
  • Better error message if we can’t find credentials

To upgrade, simply do cpanm App::gcal. For more information, see App::gcal on the CPAN.

Related Posts

WWW::XBoxLive – Perl module to get and parse an XBox Live Gamercard

Posted on by

I have just published WWW::XBoxLive, a new Perl module to get and parse an XBox Live Gamercard (i.e. http://gamercard.xbox.com/en-US/BrazenStraw3.card).

Using it is pretty simple, as follows:

my $xbox_live = WWW::XBoxLive->new();
 
my $gamercard = $xbox_live->get('BrazenStraw3');
 
say $gamercard->name;
say $gamercard->bio;
 
for my $game (@{ $gamercard->recent_games }){
  say $game->title;
  say $game->last_played;
}

For more information, see the documentation on the CPAN, or browse the source on GitHub.

gcal – A Command Line Interface to Google Calendar

Posted on by

A couple of weeks ago I published gcal, a Command Line Interface to Google Calendar.

It’s quite simple. If you have a .ics file of events you want to import into your calendar, run the following:

gcal events.ical

Or, you can create an event using shorthand, like this:

gcal 'tomorrow at noon. Lunch with Bob'

But first, you need to provide your credentials. Add the following to your ~/.netrc file:

machine google.com
login bob
password 1234

Installing

Its written in Perl, so if you already know how to install from CPAN you just need the name, App::gcal.

If not, and you are on a Mac or Linux system, just do the following to install the script:

curl -L http://cpanmin.us | perl - --sudo App::cpanminus
cpanm App::gcal

If on Windows, you first need to install perl. Then something like this should work.

cpanp -i App::gcal

More Information

For more information, see:

Any issues, please let me know.

mp4meta – Apply iTunes-like metadata to an mp4 file

Posted on by

Recently I decided to digitise all my DVD’s and add them to iTunes, so I can view them on my iPad and my (soon to be purchased) Apple TV. Getting the videos off a DVD is easy with Handbrake, but iTunes has no idea what video file I have just imported, and therefore just uses its filename as the title. Not great.

So I decided to write a script that given a file would search the internet and tell iTunes what video I have imported. The result is mp4meta.

Once I have imported the video using Handbrake, I run mp4meta against the file. It then does a search on the internet and gets some metadata for the video, including the title, genre, description and cover image. It then uses AtomicParsley to write the metadata into the video file, in a format that iTunes understands. I can then import it into iTunes and it is able to store and display the video as if it was purchased from the iTunes Store.

Below are some examples of its usage.

# pass as many video as you like
mp4meta film PULP_FICTION.mp4 "The Truman Show.m4v"
 
# if there are more than one film with the same name, we can pass the year to get the correct one
mp4meta film THE-ITALIAN-JOB-2003.m4v

Currently, the script only knows how to search for films. However, TV Series support is on it’s way shortly.

Installing

Firstly, you need to install AtomicParsley. Download from Sourceforge.

Then, if you are on a Mac or Linux system, just do the following to install the script:

curl -L http://cpanmin.us | perl - --sudo App::cpanminus
cpanm App::MP4Meta

If on Windows, you first need to install perl. Then something like this should work.

cpanp -i App::MP4Meta

More Information

For more information, see:

Any issues, please let me know.