Our Blog. We have some things we'd like to share.

I recently had a need to adapt a script that recrawls a site with nutch. One of my design goals was to use the same command line options as the Fetchtool (one of the steps I had to take to recrawl a site).

It became apparent fairly quickly that bash’s built-in ‘getopts’ didn’t support long command line arguments, so I had to fall back on getopt.

Here is the portion of the script that parses the command line arguments:

set -- `getopt -n$0 -u -a --longoptions="depth: adddays: topN:" "h" "$@"` || usage
[ $# -eq 0 ] && usage

while [ $# -gt 0 ]
do
    case "$1" in
       --depth)   depth=$2;shift;;
       --adddays) adddays=$2;shift;;
       --topN)    topN=$2;shift;;
       -h)        usage;;
       --)        shift;break;;
       -*)        usage;;
       *)         break;;            #better be the crawl directory
    esac
    shift
done

Deconstructing this bit by bit:

(more…)

For alot of people, CVS is history; they’ve switched to SVN and are loving life. For those of us continuing the use this venerable tool, I present the following cobbled together bit of utility.

It’s a bash function. and a nice way to keep an eye on what’s happening on a cvs repository. I work with a team of around 7 guys, and I like to keep an eye on what’s changed and who changed it.

It works like this, summarizing what I consider to be the highlights from the cvs history command, and defaults to showing checkins for the current date:

$ cvshist
M 2006-02-22 10:34 EDT jsmith  1.3  InternalOperation.java         APRJ7A/src/com/ppc/oepica/model/internal
M 2006-02-22 10:53 EDT jsmith  1.37 FooBarCopier.java              APRJ7A/src/com/ppc/oepica/model/external
M 2006-02-22 11:26 EDT darrend 1.23 CommonJobBase.java             APRJ7A/src/com/ppc/oepica/model/common

I use the date command, which gives me some flexibility when specifying a date

$ cvshist yesterday
$ cvshist 2 days ago
$ cvshist january 10
$ cvshist 2006-01-11

Here’s the source. I’ve used this on linux and cygwin; you’ll need bash, gnu date, and cvs.

$ declare -f
cvshist ()
{
  export CVSROOT=:pserver:darrend@shlnxtest1:/home/cvs;
  TARGET_DATE=$(date -d "$*" +%Y-%m-%d);
  cvs history -a -z EDT -c -D $TARGET_DATE | sed "s/\s*==.*$//" | grep "$TARGET_DATE"
}