Security Issues

A while ago I was phoned by the Apps DBA when I was away from the office and he told me the production is experiencing an issue that includes concurrents failing with an error message regarding access problems to .tmp files. It's not the first time I encounter this kind of problems and it's needles to say that it was never the stated issue with file permissions or disk space. So I came to the office to help the DBA to solve this issue (nothing trivial worked).

Solving the problem was odd, I had a good guess regarding the solution based on the previous cases, but it's really strange it did work since if what I did fixed a real problem I have no idea what triggered it and how the issue didn't manifest itself before, since nothing I know of changed (definitely not the problematic database parameter). Concurrents just started crashing with no apparent reason.

But that's not the main issue here (unless you're stuck with it and then you better know the solution or else you'll have some "fun" trying to solve the issue).

The really alarming thing about this case is that when concurrents crashed some of them had the apps password  (in cleartext) in their log files - the same log files every user can see for the concurrents they submit. 

Really, I couldn't believe my eyes, it seemed like the format usually present in those damn .tmp files, but I really don't understand what kind of a leak can cause the apps password to be dumped into the log file, to tell the truth it almost looks like an intended (buffer overflow?) attack.

The implications are very unnerving, what if we didn't happen to check a problematic log (not all of them had the issue) or what if we had just overlooked it? The meaning would be that everyone would be able to see the password until those logs were purged and we wouldn't have a clue. How can we be sure it won't happen again and who guarantees we will notice it the next time?