Growing up is the process of discovering that all the great stuff you believe in are just fictions and illusions. Santa isn't real, your daddy doesn't really know everything, guys who write Tetris(Tris) for the iPhone don't really block the possibility of moving the pieces upwards and real-world software isn't really developed and tested like it's described here.
Actually, I've just experienced an example of what I'm talking about in those few first lines of this post. How do you place a link in a post? Very simple, you mark some text, push the link button and enter the desired URL. But then, everything you type after the link is considered part of the link. My way of overcoming this obstacle is to edit the HTML code directly to get outside the link tag, I'll really appreciate simpler solutions. Now, I AM using Blogger and it's pretty good but this little bug is just pathetic.
When I was younger I really thought that these major software products (except Microsoft that always was a subject for mockery) are written and tested to the last bit and that they really do work - otherwise, why would someone use and even pay for them?
But when dealing with the internals of software like a sysadmin frequently does, you sometimes discover code sections that obviously weren't tested and maybe even not thought about for a single second. Just some of the examples I've encountered: code that exits instead of returning an error code, irrelevant localization code that slows down performance tremendously, code with zero error handling and much more.
It's a little sad, like every myth you discover is only a myth.
But in second thought, that's why guys like me have a job.
Saturday, March 29, 2008
Thursday, March 20, 2008
New Technology
Usually my tasks as a sysadmin are pretty standard: upgrade the database, fix the bug, deal with the backup issues and so on. While I might have not upgraded any production database yet, I still pretty much know what I'm dealing with.
But every once in a while I'm required to deal with something (almost) totally new. For the last couple of weeks my team leader and me have been working on implementing EUS - or more simply: database user authentication via OID using Kerberos. While we both pretty much know our OID, we haven't implemented this specific solution before (at least not one that worked :-)).
I really like those new challenges, I learn something new each day, but this kind of stuff makes you feel like you can add a new bullet in your CV. Like my team leader mentioned, it makes you want to work only on this new and shiny "toy" until you finally get it up and running and forget about the more mundane tasks. Unfortunately, those mundane tasks are often more pressing and so we end up working between other tasks and making little (if in a steady pace) progress each time. Well, life's tough. At least we ARE making progress...
But every once in a while I'm required to deal with something (almost) totally new. For the last couple of weeks my team leader and me have been working on implementing EUS - or more simply: database user authentication via OID using Kerberos. While we both pretty much know our OID, we haven't implemented this specific solution before (at least not one that worked :-)).
I really like those new challenges, I learn something new each day, but this kind of stuff makes you feel like you can add a new bullet in your CV. Like my team leader mentioned, it makes you want to work only on this new and shiny "toy" until you finally get it up and running and forget about the more mundane tasks. Unfortunately, those mundane tasks are often more pressing and so we end up working between other tasks and making little (if in a steady pace) progress each time. Well, life's tough. At least we ARE making progress...
Saturday, March 15, 2008
Overtraining
Metalink is one of the most useful tools for an Apps' DBA, you can search it for articles or upgrade notes, download patches and search for bug resolution instructions.
The first time you search an error message and get many results, you go over nearly every note, you don't really know to distinguish the relevant from the irrelevant and since you're probably new to the job you're afraid to miss the desired solution. Over time you develop a sense for what is relevant, even if the note doesn't mention the exact error message, you sometimes know that the issue is the same. On the other hand, just like with machine learning, you might get a bit too confident in your ability to distinguish relevant from irrelevant and miss valuable notes.
That's what happened to me this week. I was looking for a solution to a problem with an engineering form, I found a note that mentioned a patch that was supposed to install a new version of the form, but applying the patch didn't upgrade the form. So after doing a bit more searching I've asked from one of the analysts to log an SR (I wouldn't really know how to describe an engineering issue). Luckily, he was wise enough to search Metalink himself first, and indeed he've stumbled upon a similar issue in which it was suggested to make a full installation of the engineering application. Now that I think about it, it's trivial and probably the first thing I should've thought about - a form might not get installed if the application is only in shared installation mode.
Anyway, this fresh perspective have saved logging an SR and probably a couple of days work.
The first time you search an error message and get many results, you go over nearly every note, you don't really know to distinguish the relevant from the irrelevant and since you're probably new to the job you're afraid to miss the desired solution. Over time you develop a sense for what is relevant, even if the note doesn't mention the exact error message, you sometimes know that the issue is the same. On the other hand, just like with machine learning, you might get a bit too confident in your ability to distinguish relevant from irrelevant and miss valuable notes.
That's what happened to me this week. I was looking for a solution to a problem with an engineering form, I found a note that mentioned a patch that was supposed to install a new version of the form, but applying the patch didn't upgrade the form. So after doing a bit more searching I've asked from one of the analysts to log an SR (I wouldn't really know how to describe an engineering issue). Luckily, he was wise enough to search Metalink himself first, and indeed he've stumbled upon a similar issue in which it was suggested to make a full installation of the engineering application. Now that I think about it, it's trivial and probably the first thing I should've thought about - a form might not get installed if the application is only in shared installation mode.
Anyway, this fresh perspective have saved logging an SR and probably a couple of days work.
Saturday, March 8, 2008
Less is More
This week I had to bring back to life an environment I'd kept "frozen" for more than a year because of a revived project, kinda like Demolition Man. I've cleared a server to host this environment and thought I had a creative idea:
Nowadays I use NetApp storage (through iScsi) for this server, mainly because of the snapshot option, but this server has also a local disk that can accommodate the database. So I've decided to use the local disk - I get to keep the environment I had on this server before, the robocopy is much faster and I won't need snapshots for this environment anyway since it's for reference purposes only, not for development.
Do you know this guy Murphy?
A few days after the environment was up I've discovered that a couple of ApEx applications that were developed on this environment weren't kept anywhere else, so I had to export them and then import to the environment where the rest of project will be developed. Yes, I know, someone has real bad development practices, but, as my boss told me, part of my job is to clean the mess others make. So I've tried to open ApEx - nothing, a blank page. Now, you see, when those ApEx applications were developed ApEx was relatively new, actually, this old environment had an ApEx 2.0 (when it was still referred to as HTMLDB) schema as well as an ApEx 2.2 schema, and the only version to ever reach production was 2.2, so at first I thought there might be some misconfiguration of the iAS server or something like this. After exhausting the few ideas I had regarding misconfiguration I've tried to export the applications using APIs and got a strange error message about a missing datafile.
Well, you learn new things each day, it turns out that even if you can see the tablespace's definition and see the package specs for packages in it, it doesn't necessarily mean that the datafile for the tablespace is present.
It turns out that when I recreated the control file for the database I didn't have the datafile for the ApEx tablespace in the template script (remember it was new at the time). I'd expect the database to shout at me something like "man! you're missing a datafile" instead of just generating some nonexistent path for the missing datafile and going around acting like everything's ok.
God I wish I'd used the NetApp option.
I had two options:
1. Try to re-integrate the additional datafile into the database.
2. Restore the database again.
Now I get to the meaning of the title of this post. Re-integrating the datafile, if successful, will save precious copy time and giving up on a challenge is against my nature (and against this of another DBA from my team who tried to help me). But, during copy time I can do other things and this re-integration thing didn't look like it has high odds, and so we would probably end up with the second option anyway.
So, what I'm trying to say here is that being a good sysadmin doesn't always mean you have to solve any single problem, sometimes you just have to pick the more efficient course of action even if it is the more trivial one, even if it means you leave a problem unsolved. In life you have to pick your fights wisely.
Nowadays I use NetApp storage (through iScsi) for this server, mainly because of the snapshot option, but this server has also a local disk that can accommodate the database. So I've decided to use the local disk - I get to keep the environment I had on this server before, the robocopy is much faster and I won't need snapshots for this environment anyway since it's for reference purposes only, not for development.
Do you know this guy Murphy?
A few days after the environment was up I've discovered that a couple of ApEx applications that were developed on this environment weren't kept anywhere else, so I had to export them and then import to the environment where the rest of project will be developed. Yes, I know, someone has real bad development practices, but, as my boss told me, part of my job is to clean the mess others make. So I've tried to open ApEx - nothing, a blank page. Now, you see, when those ApEx applications were developed ApEx was relatively new, actually, this old environment had an ApEx 2.0 (when it was still referred to as HTMLDB) schema as well as an ApEx 2.2 schema, and the only version to ever reach production was 2.2, so at first I thought there might be some misconfiguration of the iAS server or something like this. After exhausting the few ideas I had regarding misconfiguration I've tried to export the applications using APIs and got a strange error message about a missing datafile.
Well, you learn new things each day, it turns out that even if you can see the tablespace's definition and see the package specs for packages in it, it doesn't necessarily mean that the datafile for the tablespace is present.
It turns out that when I recreated the control file for the database I didn't have the datafile for the ApEx tablespace in the template script (remember it was new at the time). I'd expect the database to shout at me something like "man! you're missing a datafile" instead of just generating some nonexistent path for the missing datafile and going around acting like everything's ok.
God I wish I'd used the NetApp option.
I had two options:
1. Try to re-integrate the additional datafile into the database.
2. Restore the database again.
Now I get to the meaning of the title of this post. Re-integrating the datafile, if successful, will save precious copy time and giving up on a challenge is against my nature (and against this of another DBA from my team who tried to help me). But, during copy time I can do other things and this re-integration thing didn't look like it has high odds, and so we would probably end up with the second option anyway.
So, what I'm trying to say here is that being a good sysadmin doesn't always mean you have to solve any single problem, sometimes you just have to pick the more efficient course of action even if it is the more trivial one, even if it means you leave a problem unsolved. In life you have to pick your fights wisely.
Tuesday, February 26, 2008
My Good Deed
Those of you who have keen eyes have noticed that I've added a new link in my favorite blogs section. I've stumbled upon Aviad's blog this week when I've encountered a problem while upgrading my EBS database to 10g. Actually that's the second time I encounter this blog, the first time was when Steven Chan mentioned it in his "Notable Apps Blogs" list. This first time I was just proud to see an Israeli in Steven's list and didn't linger too much in the blog (although it did strike me as quite an impressive blog).
The second time, even after I've discovered the specific problem I had on my hands wasn't mentioned in the blog (although many others were), I did some more thorough browsing. The guy really does excellent job, his blog might have saved me a lot of time if I was aware of it before.
Anyway, this blog inspired me to stop complaining for at least one post and share some useful information, the issue I had looks like a good place to start.
As mentioned, I'm in the process of upgrading my EBS(11i) database from a 9.2.0.5 database to a 10.2.0.3 database. During the upgrade phase (after the database software is already installed) I've encountered an "ORA-12631: Username retrieval failed" error message. The problem, apparently, was that I didn't have a listener up and running. Having a listener up might seem very basic to some of you, but I was working with my own upgrade notes that were compiled based two successful upgrades. More than that, DBUA starts the database on its own and I'm pretty sure it did the same thing with the listener on my previous attempts. I don't really know what was wrong this time but I did get back to the original notes by Oracle to see if I might have misinterpreted something.
There's definitely some confusion here: Note 362203.1 (EBS upgrade note) orders the stopping of any existent listener while the database upgrade document (not EBS related) mentions the need to create a new listener using NetCA if I don't already have a listener. So, does the upgrade manual mean I should have some listener up? And if so, why does the EBS note tells me to stop all listeners?
Again, if the stopped listener was indeed the issue, I'm sure there was some more sophisticated way to phrase the error message.
Well, had some bitterness in my post after all.
At least this might be useful to someone.
The second time, even after I've discovered the specific problem I had on my hands wasn't mentioned in the blog (although many others were), I did some more thorough browsing. The guy really does excellent job, his blog might have saved me a lot of time if I was aware of it before.
Anyway, this blog inspired me to stop complaining for at least one post and share some useful information, the issue I had looks like a good place to start.
As mentioned, I'm in the process of upgrading my EBS(11i) database from a 9.2.0.5 database to a 10.2.0.3 database. During the upgrade phase (after the database software is already installed) I've encountered an "ORA-12631: Username retrieval failed" error message. The problem, apparently, was that I didn't have a listener up and running. Having a listener up might seem very basic to some of you, but I was working with my own upgrade notes that were compiled based two successful upgrades. More than that, DBUA starts the database on its own and I'm pretty sure it did the same thing with the listener on my previous attempts. I don't really know what was wrong this time but I did get back to the original notes by Oracle to see if I might have misinterpreted something.
There's definitely some confusion here: Note 362203.1 (EBS upgrade note) orders the stopping of any existent listener while the database upgrade document (not EBS related) mentions the need to create a new listener using NetCA if I don't already have a listener. So, does the upgrade manual mean I should have some listener up? And if so, why does the EBS note tells me to stop all listeners?
Again, if the stopped listener was indeed the issue, I'm sure there was some more sophisticated way to phrase the error message.
Well, had some bitterness in my post after all.
At least this might be useful to someone.
Saturday, February 16, 2008
Power To The Numbers
Our IT department (which I'm part of) has an internal Sharepoint based Wiki, our aspiration is to make it, as Wikipedia is, the place you start from when looking for information. It's supposed to include all kinds of information: general technological issues like how to install a VM, more specific issues like how to change the infrastructure for an IAS and even not work related topics like take aways to stay away from.
We're not at all there yet, and thinking about it I'm not sure we'll ever really get there. I can think of a several reasons:
1. A Wiki page is not really that good a replacement for a real document, a Wiki page can contain summarized information and maybe a link to the original document, but unless you have trouble finding the document, you don't really have a reason to look in the Wiki in the first place.
When looking for some arbitrary subject on the internet it's not always trivial to find a good document about it, so you use Wikipedia. In an IT department when looking for a documented issue it's simpler to just look for the document, since whoever documented the issue have probably written a document even if he wrote about it in the Wiki as well, and as mentioned, documents are better.
2. Specific issues like the mentioned example are too specific since they often address only one person, the one who wrote them down. Sure, people can forget and get replaced but still a document will do the job just as well.
3. The main reason as I see it.
A Wiki exemplifies the egg or chicken paradox. For people to use Wiki you obviously have to have some content, but for people to add content they usually have to consider the Wiki a source of information people use, not much point in writing something if nobody reads it.
Well, Wikipedia DOES work.
But let's say you have a blank new Wikipedia. Right, according to my assumption no one will
write anything, but after all, IT IS the internet. Even if one of every one thousand users writes something you get plenty of data, so now a higher percentile of the population is willing to donate their knowledge and so you get a plague-like effect where people use Wikipedia, find what they were looking for and add their own insights.
Now, lets get back to our department of about 30 people. Sure, we did write some pages at the beginning and even had a "Wiki week" in which we tried to add as much new content as possible, but as long as it's easier to ask the guy over the table or just browse for the right document, the Wiki usage seems pretty artificial. I've written some pages in our Wiki but I can't think of a situation where I'll actually think about looking there for answers.
That's why I'm trying to make the trainee to document everything he learns in the Wiki, this actually achieves some goals, first, as a newbie he has a better sense for what should be documented, second that's a good way to create a sort of a "training Wiki" and third it makes him go over what he have learned - everybody's in gain.
On the optimistic side, maybe it'll just take time, maybe in several years our Wiki WILL be considered a central source for information.
We'll just have to wait and see.
We're not at all there yet, and thinking about it I'm not sure we'll ever really get there. I can think of a several reasons:
1. A Wiki page is not really that good a replacement for a real document, a Wiki page can contain summarized information and maybe a link to the original document, but unless you have trouble finding the document, you don't really have a reason to look in the Wiki in the first place.
When looking for some arbitrary subject on the internet it's not always trivial to find a good document about it, so you use Wikipedia. In an IT department when looking for a documented issue it's simpler to just look for the document, since whoever documented the issue have probably written a document even if he wrote about it in the Wiki as well, and as mentioned, documents are better.
2. Specific issues like the mentioned example are too specific since they often address only one person, the one who wrote them down. Sure, people can forget and get replaced but still a document will do the job just as well.
3. The main reason as I see it.
A Wiki exemplifies the egg or chicken paradox. For people to use Wiki you obviously have to have some content, but for people to add content they usually have to consider the Wiki a source of information people use, not much point in writing something if nobody reads it.
Well, Wikipedia DOES work.
But let's say you have a blank new Wikipedia. Right, according to my assumption no one will
write anything, but after all, IT IS the internet. Even if one of every one thousand users writes something you get plenty of data, so now a higher percentile of the population is willing to donate their knowledge and so you get a plague-like effect where people use Wikipedia, find what they were looking for and add their own insights.
Now, lets get back to our department of about 30 people. Sure, we did write some pages at the beginning and even had a "Wiki week" in which we tried to add as much new content as possible, but as long as it's easier to ask the guy over the table or just browse for the right document, the Wiki usage seems pretty artificial. I've written some pages in our Wiki but I can't think of a situation where I'll actually think about looking there for answers.
That's why I'm trying to make the trainee to document everything he learns in the Wiki, this actually achieves some goals, first, as a newbie he has a better sense for what should be documented, second that's a good way to create a sort of a "training Wiki" and third it makes him go over what he have learned - everybody's in gain.
On the optimistic side, maybe it'll just take time, maybe in several years our Wiki WILL be considered a central source for information.
We'll just have to wait and see.
Friday, February 8, 2008
Syntax Confusion
There are many programming languages, each has it's strengths and weaknesses. And it's own syntax. I don't really understand what is it good for. Sure, each language has it's unique features, a strongly-typed language will have different syntax from a weakly-typed language, but I'm not talking about this "advanced" stuff, I'm talking about the very basics. And no better way to show what I mean than an example:
You already know I'm not a programmer, but I write PL/SQL code for the application, cmd and Perl scripts for administration purposes and this last semester I took Computer Vision and Image Processing courses and had a lot of assignments to implement in Matlab. Four languages, four different ways to write a comment. At some point I got confused with all the '%','#','--' tags (REM stands out more clearly), is this really necessary? Why should there be three(!) different ways to write a simple "else if" statement? Ok, let's say "elseif" somehow simplifies parsing, but "elsif"?! That's really ridiculous, it just looks like someone was trying to prove his creativity.
Why won't some standardization organization decide on how the basic syntax (comments, loops, if-then-else statements) should look like, and make - I don't really know how those things work - the vendors to have the next versions of their compilers to support the standard (unfortunately, backward compatibility is inevitable here)?
I don't really know if my idea is in any way feasible but this syntax confusion can get very... well, confusing.
You already know I'm not a programmer, but I write PL/SQL code for the application, cmd and Perl scripts for administration purposes and this last semester I took Computer Vision and Image Processing courses and had a lot of assignments to implement in Matlab. Four languages, four different ways to write a comment. At some point I got confused with all the '%','#','--' tags (REM stands out more clearly), is this really necessary? Why should there be three(!) different ways to write a simple "else if" statement? Ok, let's say "elseif" somehow simplifies parsing, but "elsif"?! That's really ridiculous, it just looks like someone was trying to prove his creativity.
Why won't some standardization organization decide on how the basic syntax (comments, loops, if-then-else statements) should look like, and make - I don't really know how those things work - the vendors to have the next versions of their compilers to support the standard (unfortunately, backward compatibility is inevitable here)?
I don't really know if my idea is in any way feasible but this syntax confusion can get very... well, confusing.
Subscribe to:
Posts (Atom)