Mission Data Blog
Louisville-based Web Development & Software Engineering

A few days ago I made use of a couple Oracle built in functions and it made me happy I didn’t have to write a stored proc or some type of mini-app to do it. I needed to parse a timestamp out of a field that was put there by a java program. The timestamp was just the output of System.currentTimeInMillis() and was concatenated onto some other information.

It took a little digging to find out how to convert a epoch style timestamp but here it is:

select new_time( to_date('01011970', 'ddmmyyyy') + 1/24/60/60 * :currenttimeinmillis/1000, 'GMT', 'EDT' ) from dual

Note that I convert the output from GMT to EDT here.

At a customer site, the test and production linux servers for some intranet applications were slowly running out of disk space. The apps themselves were running fine, it was the logfiles generated by the apps that were the issue; logging that came in useful just in case something went wrong, but quickly aged. In one particular case, a catalina.out file was several years old and was 11 gigabytes; 90% of it wasn’t really relevant any longer.

The solution to the problem of wrangling logfiles is probably already installed on your server: it’s logrotate. Chances are logrotate is already configured in your system crontab as a daily task, and chances are it is configured to obey any configuration files found in the /etc/logrotate.d directory. If you find that directory, you are probably good to go.

(more…)

iSeries systems and Ruby are separate universes right now; I know of no native Ruby way to connect to an AS400 system. However, with the help of Java we can bridge the gap.

Probably the simplest example is connecting to an iSeries system using JDBC to select from some tables, and with the right libraries it is straightforward. It’s also fun to be doing some nifty quick Ruby but with some preexisting and proven Java libraries.

Also, if you use Java and Ruby, definitely put arton’s RubyJavaBridge in your pocket; you’ll end up using it more than once.

What you’ll need:

• Ruby (well yea)
• Java (ok)
• access to an AS400 system (that’s the whole point here…)
  For testing, I’ve been using a free account available from Holger Scherer Software und Beratung.
• jt400.jar jdbc drivers from JTOpen, a great open-source Java library for AS400 access.
• RubyJavaBridge is the keystone for this.

Installing the software is the hardest part of this exercise, but it’s more tedium than anything. Once you’ve got it all downloaded and running the fun can begin.

(more…)

Threading in java is fairly easy and now with java 1.5 some of the stuff that was harder has become even easier. A few years ago someone pointed me to a site that had some concurrency utils that where the precursor to what are now the concurrent utils in java 1.5. They are very close in functionality and if you can’t use java 1.5 the older version of the utils will work with older versions of java and give you a lot of the same functionality.

I’m going to give a quick thread pooling example using both the new and old concurrency utils. I picked the thread pooling out of both since that seems to be what I end up using the most out of all the new utilities. I may revisit this again at some point to go over the periodic executors or some of the other things I have used but just not as much.

(more…)

Techcrunch has a good review by Frank Gruber of the look and feel of mapping services. I think it is notable that ESRI’s service is not included in the review. I think it is at least as good as the mapquest service. I may have to find time to redo my review of the acuracy of each again and a more technical evaluation of each.

I recently had an opportunity to use ESRI’s ArcSDE again. It is a spatial database interface and in this instance I was using the java api. I wanted to change what used to be a query using a rectangle into a query using a circle. For some reason parts of the java api for ArcSDE require a C library or something. I gave up pretty quickly on trying to make their arc function work since the documentation wasn’t very clear on how it worked. Instead I decided to figure out how to approximate a circle with a polygon and use that instead. Here is the result of that research.

(more…)

A lot of dates that come back from XML based web services are in the ISO 8601 form. I found out recently that it isn’t straight forward to parse such a date using C functions and have the time come out in the correct timezone. It isn’t rocket science but it is a lot more convoluted than higher level languages like Java.

First of lets see an example of the ISO 8601 format:

2006-02-03T16:45:09.000Z

That breaks down into:

<date>T<time><timezone>

Where a timezone of Z is equal to UTC. I was only interested in parsing timestamps in UTC so the following only applies to that timezone.

If you want to know any more about the format check out this page.

The strptime function is a flexible way to turn a string into a struct tm given a specified format. It is like a sscanf for dates. For more information on it check here. I’m not exactly sure of the portability of this function but it seems to be fairly old now so it is probably reasonably portable.

The format used to parse the a ISO 8601 timestamp is: %FT%T%z

(more…)

I’ve been doing a little C programming lately and I have found that if you have a up to date distribution of linux there are a lot of libraries out there that make doing things you do in other languages like java easier.

As I have time I’m going to post some examples of what I have found. The first here is how to base64 encode a chunk of memory using OpenSLL.

(more…)

While creating a small CGI web application in Ruby the question came up about how much slower ActiveRecord is than DBI. Considering that ActiveRecord is initialized each time the CGI runs the performance hit could be significant.

So why not use Rails instead? The server that I am running the application on has limited resources. Rails is reported to need about 20 MB of memory and the node I have is already paging out memory to disk.

To emulate the webserver getting hit repeatedly with connections I created a connect_dbi.rb and and connect_ar.rb that insert the same row into a Tests database table. The ActiveRecord test also has the model file test.rb

The test was run using a PostGreSQL database on Ubuntu 5.10 with Ruby 1.8.4. The table was truncated between tests. I just wanted to get an idea of the performance difference between ActiveRecord and DBI. So this isn’t the strictest performance test but does provide an idea of the magnitude of difference.

(more…)

While developing a webservice based application we ran across some issues using a self signed certificate. After running our wsdl2java ant task we got the following error using Java 1.4:

sun.security.validator.ValidatorException: No trusted certificate found

Using Java 1.5 the error looks like this:

sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested targ

Fair enough. Java is telling us we need to import our self signed certificate into java:

/usr/local/java5/bin/keytool -import -alias mycert \\\\
  -file server.crt -keystore /usr/local/java5/jre/lib/security/cacerts
Enter keystore password:  changeit
... CERTIFICATE DUMP ...
Trust this certificate? [no]:  yes
Certificate was added to keystore

Running our ant task again:

java.io.IOException: HTTPS hostname wrong:  should be <localhost>
  at sun.net.www.protocol.https.HttpsClient.checkURLSpoofing(HttpsClient.java:490)
  at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:415)
...LONG STACK TRACE TRUNCATED....

(more…)