High-End Software

Like I mentioned a few months ago, I'm currently working on an Enterprise Search project. Since then I discovered a few other annoying things about FAST ESP and other high-end search solutions.

I've tested two main types of search solutions:

1. Entry level solutions - by companies that have another core business except search.

2. High-end solutions - by companies whose core business is search.

Obviously, there a lot of products somewhere in between, but that's always true. Actually, FAST has been lately acquired by Microsoft, but for the purpose of this post it is still under the second category.

Being a newcomer to this field of Enterprise Search I was quite innocent and thought that entry level solutions will be simple, basic and easy to use - no disappointment here, but I also expected high-end solutions to be a complete search suite that can do great and cool stuff.

Well, products I've examined can do great and cool stuff, I'll even exaggerate and say that it feels like everything is possible if you know your way around the product, but in no way these products I've examined are a suite. It almost seems (it's actually sounds like a swell business model) that the products are intentionally designed to make you grate your teeth at every step so they can provide their business partners with work for their consultants. It seems like they try to make the product as naked as possible, leaving only the basics of indexing efficiently, providing customizing tools and a few other abilities. As always, a whining is not complete without a few examples. I'll rely mostly on FAST ESP examples, but not only, I just know it the best:

1. I would expect a high-end solution to include some security enforcement. Say I want to index a file system(obviously there are other examples as well) content source, to the entry level products it's obvious that I want to index the ACLs as well, not so much to the high-end stuff. High-end software will require installing an additional module that I'll have to carefully configure. And that leads me straight to the next point...

2. To configure the security model for FAST ESP to enforce file system security I have to follow the documentation which, put in one simple word, sucks.

Fact: surprisingly the FAST guys realized I might want to index Windows based file systems.

What I'd expect to see in the documentation: a simple to-do list containing every step I should do in order to configure the whole thing.

What I actually got: every piece of the puzzle is in a different part of the documentation so before each of the following revelations I had to wonder why nothing works and why nothing is written (or at least not where I expect to find it):

• When the module is initially installed no indexed items are searchable, there are a few ways to work around it and they're written at the end of the documentation.
• There's no built in authentication module (a pity) but at least there are a few ways to work around it as well.
• To index securely you have to modify the processing pipeline (or the whatever the term the product uses), next point here I come.

3. The products I've examined have far too much of encoding related (Hebrew and computers should not co-exist) issues, some of them I've been able to overcome but still, for now I can't index securely paths with Hebrew in FAST ESP.

I have lots of other examples and I'll probably have a lot more as I go along, but I think I made my point. Totally high-end...

Development

I'm not a developer but as the team leader of the sysadmins (which is often referred to as the "technological team") in my department I am interested in the platforms and methods that are used for development, after all the bottom line is that I'm responsible for the stability of the systems and a system is more stable when the correct practices are used.

Let's take for example our Oracle ERP system (which I used to administer so I'm more familiar with it than with other systems). Today we have web applications based on different technologies:

1. PL/SQL cartridge based applications - well, the programmers do know PL/SQL so it's kinda OK, but no new applications are written using this technology and in the next EBS version (R12) it's no longer supported.

2. Java based applications - no one really knows them and they're supposed to be re-written.

3. ApEx based applications - small modifications are still made to them but the knowledge level is pretty low.

4. .NET based applications - that's the technology that all the developers are familiar with and with which the latest applications were developed.

We have all those techniques since each of them fitted a different need at a different time.

One technology we don't currently use is the Oracle Applications Framework which is a pretty strong technology but...

We currently have about 3.5 EBS developers none of which is really specialized in Java (and the Framework is Java based) and the turnover rate in the developers' team is pretty high.

These facts don't really get along well with the multiple development techniques we maintain.

So now it seems like the time to make a strategic decision as to how the EBS development will look in the next few years, I can think of two main approaches:

1. We choose one single technology that we'll use to write all the solutions, obviously it won't meet perfectly all the needs but the programmers will be specialized in this technology and training will be relatively simple.

2. We choose two technologies that will answer two major types of requirements - for example general user applications vs. specialized user applications. The programmers will be less specialized but we'll be able to provide better solutions.

Anything more than that will have too much training and maintenance overhead...