Mission Data Blog
Louisville-based Web Development & Software Engineering

A client site still uses cvs, the ever trusty version control system. After what seemed a run of the mill merge I noticed this:

C lib/jt400_3_3.jar
cvs update: move away lib/jt400_3_3.jar; it is in the way

Very peculiar. That file hadn’t changed on the branch. I googled around and an explanation started to come together.

First, from the always excellent Roedy Green’s Java & Internet Glossary on Mindprod:

CVS is disturbed by the appearance of repository files it did not put there. Your best bet is simply to delete the entire directory containing the offending file by hand, and re checkout or reupdate the directory to build the necessary entries. Then you can add the files safely.

Then, I found this mailing list post:

This means the file that CVS wants to checkout exists on the local machine but CVS never created the file in the past. This it isn’t CVS’s file and it complains rather than blindly overwriting.

The solution everywhere was the same: just blow away the directory and check it out again. Worked for me. Now I need to puzzle out how it happened in the first place…

Our project has entered the stage where we start tuning the data access to improve performance. This project uses Hibernate to access a AS400/iSeries system via the JT400 JDBC driver. My prior experience using the SQL Server
database introduced me to Microsoft’s Query Analyzer, which is a nice tool for identifying where you need to index the database. A bit of searching turned up an IBM tool called Visual Explain that fits the same need. The sysadmin started the profiling tool against my connection, and I ran through a standard usage scenario in our application. Visual Explain showed each query, how long it took, the access strategy, and recommended indexes. This kind of tool is absolutely essential for database performance tuning.

The second important item we learned from this process is the performance difference between a true SQL index on the iSeries and a keyed logical index. From iSeries V4R2 on, SQL indexes have page sizes of 64k. By comparison, a DDS logical will max out at 32k, and will typically be much smaller. The performance is so much better that our customer is converting all all their DDS logical indexes to SQL, even if only their legacy RPG applications use it. This IBM document has a good discussion of this issue.

For my current project we needed to audit the property setters Hibernate was using on our objects to make sure that any logic in them was not overly state dependent. More about this issue in Hibernate is available here. We have fairly rich object models, and a lot of methods, including setters, are never used by Hibernate. We wanted a report of the setters actually used by Hibernate to limit the amount of code we had to examine.

The Hibernate API allows you a lot of access to its configuration object model, and this is ideal for finding out how Hibernate is interacting with your code. I wrote a small class to do this inspection. The method below is run after a Hibernate Configuration object named creatively as “configuration” has been built with mapping files:

public Map findSetters() throws MappingException
{
  Map classToSetters = new HashMap();
  Iterator classMappingIterator =   configuration.getClassMappings();

  while(classMappingIterator.hasNext())
  {
    PersistentClass persistentClass = (PersistentClass)classMappingIterator.next();
    Class mappedClass = persistentClass.getMappedClass();
    Iterator propertyIt = persistentClass.getPropertyIterator();
    List classSetters = new LinkedList();

    classToSetters.put(mappedClass, classSetters);

    while(propertyIt.hasNext())
    {
      Property property = (Property)propertyIt.next();
      Setter setter = property.getSetter(mappedClass);

      classSetters.add(setter.getMethodName());
    }
  }
  return classToSetters;
}

I have uploaded a Java project that contains the full HibernateInspector class, as well as some sample classes and mappings. Un-tar it, and run

ant -Dhibernate.home="path to hibernate 3" 

to build and run the example.

At a customer site, the test and production linux servers for some intranet applications were slowly running out of disk space. The apps themselves were running fine, it was the logfiles generated by the apps that were the issue; logging that came in useful just in case something went wrong, but quickly aged. In one particular case, a catalina.out file was several years old and was 11 gigabytes; 90% of it wasn’t really relevant any longer.

The solution to the problem of wrangling logfiles is probably already installed on your server: it’s logrotate. Chances are logrotate is already configured in your system crontab as a daily task, and chances are it is configured to obey any configuration files found in the /etc/logrotate.d directory. If you find that directory, you are probably good to go.

(more…)

I recently had a need to adapt a script that recrawls a site with nutch. One of my design goals was to use the same command line options as the Fetchtool (one of the steps I had to take to recrawl a site).

It became apparent fairly quickly that bash’s built-in ‘getopts’ didn’t support long command line arguments, so I had to fall back on getopt.

Here is the portion of the script that parses the command line arguments:

set -- `getopt -n$0 -u -a --longoptions="depth: adddays: topN:" "h" "$@"` || usage
[ $# -eq 0 ] && usage

while [ $# -gt 0 ]
do
    case "$1" in
       --depth)   depth=$2;shift;;
       --adddays) adddays=$2;shift;;
       --topN)    topN=$2;shift;;
       -h)        usage;;
       --)        shift;break;;
       -*)        usage;;
       *)         break;;            #better be the crawl directory
    esac
    shift
done

Deconstructing this bit by bit:

(more…)

For alot of people, CVS is history; they’ve switched to SVN and are loving life. For those of us continuing the use this venerable tool, I present the following cobbled together bit of utility.

It’s a bash function. and a nice way to keep an eye on what’s happening on a cvs repository. I work with a team of around 7 guys, and I like to keep an eye on what’s changed and who changed it.

It works like this, summarizing what I consider to be the highlights from the cvs history command, and defaults to showing checkins for the current date:

$ cvshist
M 2006-02-22 10:34 EDT jsmith  1.3  InternalOperation.java         APRJ7A/src/com/ppc/oepica/model/internal
M 2006-02-22 10:53 EDT jsmith  1.37 FooBarCopier.java              APRJ7A/src/com/ppc/oepica/model/external
M 2006-02-22 11:26 EDT darrend 1.23 CommonJobBase.java             APRJ7A/src/com/ppc/oepica/model/common

I use the date command, which gives me some flexibility when specifying a date

$ cvshist yesterday
$ cvshist 2 days ago
$ cvshist january 10
$ cvshist 2006-01-11

Here’s the source. I’ve used this on linux and cygwin; you’ll need bash, gnu date, and cvs.

$ declare -f
cvshist ()
{
  export CVSROOT=:pserver:darrend@shlnxtest1:/home/cvs;
  TARGET_DATE=$(date -d "$*" +%Y-%m-%d);
  cvs history -a -z EDT -c -D $TARGET_DATE | sed "s/\\s*==.*$//" | grep "$TARGET_DATE"
}