Mission Data Blog
Louisville-based Web Development & Software Engineering

At a customer site, the test and production linux servers for some intranet applications were slowly running out of disk space. The apps themselves were running fine, it was the logfiles generated by the apps that were the issue; logging that came in useful just in case something went wrong, but quickly aged. In one particular case, a catalina.out file was several years old and was 11 gigabytes; 90% of it wasn’t really relevant any longer.

The solution to the problem of wrangling logfiles is probably already installed on your server: it’s logrotate. Chances are logrotate is already configured in your system crontab as a daily task, and chances are it is configured to obey any configuration files found in the /etc/logrotate.d directory. If you find that directory, you are probably good to go.

Read the rest of this entry »

iSeries systems and Ruby are separate universes right now; I know of no native Ruby way to connect to an AS400 system. However, with the help of Java we can bridge the gap.

Probably the simplest example is connecting to an iSeries system using JDBC to select from some tables, and with the right libraries it is straightforward. It’s also fun to be doing some nifty quick Ruby but with some preexisting and proven Java libraries.

Also, if you use Java and Ruby, definitely put arton’s RubyJavaBridge in your pocket; you’ll end up using it more than once.

What you’ll need:

• Ruby (well yea)
• Java (ok)
• access to an AS400 system (that’s the whole point here…)
  For testing, I’ve been using a free account available from Holger Scherer Software und Beratung.
• jt400.jar jdbc drivers from JTOpen, a great open-source Java library for AS400 access.
• RubyJavaBridge is the keystone for this.

Installing the software is the hardest part of this exercise, but it’s more tedium than anything. Once you’ve got it all downloaded and running the fun can begin.

Read the rest of this entry »

Threading in java is fairly easy and now with java 1.5 some of the stuff that was harder has become even easier. A few years ago someone pointed me to a site that had some concurrency utils that where the precursor to what are now the concurrent utils in java 1.5. They are very close in functionality and if you can’t use java 1.5 the older version of the utils will work with older versions of java and give you a lot of the same functionality.

I’m going to give a quick thread pooling example using both the new and old concurrency utils. I picked the thread pooling out of both since that seems to be what I end up using the most out of all the new utilities. I may revisit this again at some point to go over the periodic executors or some of the other things I have used but just not as much.

Read the rest of this entry »

Techcrunch has a good review by Frank Gruber of the look and feel of mapping services. I think it is notable that ESRI’s service is not included in the review. I think it is at least as good as the mapquest service. I may have to find time to redo my review of the acuracy of each again and a more technical evaluation of each.

I recently had an opportunity to use ESRI’s ArcSDE again. It is a spatial database interface and in this instance I was using the java api. I wanted to change what used to be a query using a rectangle into a query using a circle. For some reason parts of the java api for ArcSDE require a C library or something. I gave up pretty quickly on trying to make their arc function work since the documentation wasn’t very clear on how it worked. Instead I decided to figure out how to approximate a circle with a polygon and use that instead. Here is the result of that research.

Read the rest of this entry »

A lot of dates that come back from XML based web services are in the ISO 8601 form. I found out recently that it isn’t straight forward to parse such a date using C functions and have the time come out in the correct timezone. It isn’t rocket science but it is a lot more convoluted than higher level languages like Java.

First of lets see an example of the ISO 8601 format:

2006-02-03T16:45:09.000Z

That breaks down into:

<date>T<time><timezone>

Where a timezone of Z is equal to UTC. I was only interested in parsing timestamps in UTC so the following only applies to that timezone.

If you want to know any more about the format check out this page.

The strptime function is a flexible way to turn a string into a struct tm given a specified format. It is like a sscanf for dates. For more information on it check here. I’m not exactly sure of the portability of this function but it seems to be fairly old now so it is probably reasonably portable.

The format used to parse the a ISO 8601 timestamp is: %FT%T%z

Read the rest of this entry »

I’ve been doing a little C programming lately and I have found that if you have a up to date distribution of linux there are a lot of libraries out there that make doing things you do in other languages like java easier.

As I have time I’m going to post some examples of what I have found. The first here is how to base64 encode a chunk of memory using OpenSLL.

Read the rest of this entry »

While creating a small CGI web application in Ruby the question came up about how much slower ActiveRecord is than DBI. Considering that ActiveRecord is initialized each time the CGI runs the performance hit could be significant.

So why not use Rails instead? The server that I am running the application on has limited resources. Rails is reported to need about 20 MB of memory and the node I have is already paging out memory to disk.

To emulate the webserver getting hit repeatedly with connections I created a connect_dbi.rb and and connect_ar.rb that insert the same row into a Tests database table. The ActiveRecord test also has the model file test.rb

The test was run using a PostGreSQL database on Ubuntu 5.10 with Ruby 1.8.4. The table was truncated between tests. I just wanted to get an idea of the performance difference between ActiveRecord and DBI. So this isn’t the strictest performance test but does provide an idea of the magnitude of difference.

Read the rest of this entry »

While developing a webservice based application we ran across some issues using a self signed certificate. After running our wsdl2java ant task we got the following error using Java 1.4:

sun.security.validator.ValidatorException: No trusted certificate found

Using Java 1.5 the error looks like this:

sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested targ

Fair enough. Java is telling us we need to import our self signed certificate into java:

/usr/local/java5/bin/keytool -import -alias mycert \\\\
  -file server.crt -keystore /usr/local/java5/jre/lib/security/cacerts
Enter keystore password:  changeit
... CERTIFICATE DUMP ...
Trust this certificate? [no]:  yes
Certificate was added to keystore

Running our ant task again:

java.io.IOException: HTTPS hostname wrong:  should be <localhost>
  at sun.net.www.protocol.https.HttpsClient.checkURLSpoofing(HttpsClient.java:490)
  at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:415)
...LONG STACK TRACE TRUNCATED....

Read the rest of this entry »

This past Friday I participated in my first presentation, Seeing Ruby: An Introduction to the Ruby Programming Language at the iTRC in Louisville, Kentucky. Chuck Fouts, Steven Yelton, and I did our level best to introduce our audience to one of our favorite development tools. Our audience was fantastic, and had lots of questions. Everyone stayed a good 15 minutes after the official end to finish Steven’s awesome Rails demo.

Honestly, Ruby sells itself though. People tend to sit up and notice blocks and open classes.

We’ll be posting the presentation slides very soon to the permanent page. And as we take the show to other venues we’ll be tweaking and refining the content. Initial feedback is to beef up the handout with a cheatsheet of language features; our whirlwind tour of the language goes by quickly and some sort of reference will help keep everything together.

Our audience expressed great interest in Ruby on Rails and AJAX, even a few requests for some Seeing More Ruby classes. We’re debating on whether “You’ll Be Seeing AJAX” or “You’ll Be Seeing Rails” next. :) Let us know which you’d rather see.

Incidentally, we’re also taking this presentation to a company for a private show, and we’d be willing to come talk with you and your colleagues as well. Just contact us!