Friday, December 26, 2008

Simple Testing

A while ago I've upgraded my whole development AS infrastructure to 10.1.2.3, after the upgrade SSO stopped working. Since SSO (at least mine) tends to be a bit fragile, I sighed with a familiar feeling that this is going to take some time and started browsing through the logs. Once again Oracle proved that their logs suck and tend to display the same error message for totally different errors. So the logs showed an error that usually accompanies a bad keytab file, since I had some similar issues lately I've decided to re-create the keytab file, but it didn't really help. Apparently, all I had to do is to search the Metalink. 

The real issue is that the jdk version this version(actually not only this one, if I'm not mistaken IDM 10.1.4.0.1 is as well) is shipped with (1.4.2_14) has some error that prevents SSO from working, the solution is a simple one - install a higher version (say 1.4.2_19).
Now, these things get me really frustrated. OK, I get it, your error handling is not the thing you take pride in (sure hope not), but it's not the first time (more examples to come in following posts or you can read this post again [last example]) it seems nobody have really tested the final product. Had somebody taken the final product (with jdk 1.4.2_14) about to be published for everyone to download, installed it and tested it for the very basic functionality this error would've been discovered (and hopefully the product wouldn't be published).

That's the part when I start to imagine the following conversation (a special bonus for whoever discovers the meaning behind the aliases):
M: "Hey, there's a new jdk out! We should ship 10.1.2.3 with it because it's the newest" (and new is good, right?)
S: "But we tested it with 1.4.2_x<14!"
M: "Yeah man, but we didn't really touch something that heavily depends on jdk specifics, it's a minor version anyway."
S: "You know what, you're totally right, let's do it! It's not like we've ever shipped any totally-unworking piece of code before."

Really, with the simple applications we have in my company we try to test them thoroughly and even though not always successful try to enforce different rules before deploying, things like a certain period of time in which the application has to work on a test environment without code modifications before moving to production, a clean testing environment for relatively big installations, etc.. 
So how come Oracle manages to ship a totally not working version of a product?

2 comments:

Unknown said...

Great post.
First, I demand the bonus for those aliases, I know what they mean (think I invented that joke...).

Second, and more seriously, for some reason, Oracle is making the same two mistakes all the time:

1. Staying with very old JRE versions. Actually so old, they're slow and crippled. If one is working on upgrading server software from JRE_1.4.1.0 to JRE_1.4.1.1 without considering a greater move (say 1.6), then he is being too conservative, and wasting the QA time. S and M really did waste QA time.
The downside of my suggestion would be longer testing for client applications (which are deployed on the server) as well.

2. I came to realize Oracle did some amazing testing on their product, and it was perfect when it shipped. But they forgot this minor platform, what's its name... hmmm... oh, right, Windows.

Finally, I too believe Oracle should reconsider the efforts and budgets spent on their product quality. Software must be shipped without known bugs. So the least you can do to improve the next version, is to report the bug you just solved. Too bad you won't get paid for it (but having your name on some Oracle patches is nice and sometimes useful).

Unknown said...

Guess you've got yourself a beer mate! Anyone else?