INTEGRATOR'S BLUES

by Wayne M. Krakau - Chicago Computer Guide, October, 2002
The subtitle of this column could easily be "Mothers, don't let your kids grow up to be systems integrators." To paraphrase an old cliché, "Sometimes you get the computer bear and sometimes the computer bear gets you." I still have bite (byte?) marks from this misadventure which should have taken a portion of one weekend but actually took four full weekends spread out over several months, and included my single longest tech support call ever (eleven and a half continuous hours).

The system administrator at a client of mine and I were going kill a "fun" weekend migrating a NetWare 5.0 server over to a new NetWare 6 computer. I had taken the original "Breakout" NetWare 6 class for instructors, but she had only a quickie sample NetWare 6 seminar. I knew things weren't going to go well when I found out that she hadn't done any of the preparation work that I had suggested.

It turns out that during a tech support call to Novell earlier that week, the technician contradicted all of my advice and declared my suggestions superfluous. Given a choice of applying potentially disruptive general patches and eDirectory (formerly called NDS or Novell Directory Services) updates to a production server, and just leaving things be, she made the obvious choice. I couldn't blame her for taking a Novell technician's word over mine. I could feel bear's hot breath on the back of my neck as he approached.

We ran Novell's NWDEPLOY program to prepare the old system for the migration. It automatically updated the eDirectory; something that I had been warned was better done manually. We then ran the Migration Wizard to actually move data and properties from the old server to the new server. The data transfer went well, though it took many hours.

The movement of properties, which included the eDirectory, seemed to go well, but left us unable to run any tools to access, alter, or even check the validity of the eDirectory. Every program we ran, such as DSREPAIR (the Directory Services repair program) would fail and give us a mismatching version error. This error was quite similar to the old DOS error that you would get from running say, the DOS 5.0 CHKDSK command on a DOS 6.0 system.

After a couple of hours of experimentation, we gave up and called Novell, paying for a tech support incident. The first thing they asked was the eDirectory version on the new server. When we told them, they didn't believe us! They said there was no such version and talked to us like we were either completely inexperienced or perhaps just a few transistors short of a CPU. This argument went on for quite some time until one of their technicians linked up and remote-controlled the new server. That finally convinced them that we had a nonstandard, never-released-to-the-public (at least in theory) version of eDirectory.

As far as we could tell, though based totally on conjecture, some developer had accidentally left in a line of code that stated, in essence, "If the following conditions are true, then this must be my special test server, and it should be loaded with this special, experimental version of eDirectory instead of the standard version." Somehow we met those conditions, though, from subsequent inquiries on my part, we may have been the only ones to ever run into this since-fixed bug.

After spending several hours trying to get the new server into a usable state we finally had to give up since it was already Monday morning and employees would need a live server within the hour. The technician assisted us in reviving the old NetWare 5 server and apologized for the problem. Chomp! That bear took quite a piece out of me.

A few weeks later, we tried again. This time we were at the client's new location where the air conditioning was not yet installed in the computer room, so conditions were quite miserable. This time we decided to get creative (always a dangerous idea) and back up some of the auxiliary volumes onto tape so that the migration would go faster. We planned to restore those volumes while we did the post-migration installation of updated applications and the reconfiguration of the many printers this client used.

This time the deployment and migration went so well that we didn't have to call tech support at all. We were so proud of ourselves. We cranked up Veritas Backup Exec, inserted the appropriate tape, and watched while the server promptly froze. The bear already had one paw on my shoulder.

After reconfiguring the system every way we could, including even swapping out parts where possible, we finally had to give up again. We had hours of updating, reconfiguration, and subsequent testing to do and it was already late Sunday night. We couldn't risk not having a viable server in the morning.

The administrator was convinced that I had sold her a defective server, and I was worried about another mysterious NetWare bug. It wasn't until several weeks and many hours of research later that I found a brand-new message in the forums on the Veritas Web site that indicated that there was a bug in Backup Exec which causes a complete server lock-up if you try to back up information from a NetWare 5 server using traditional volumes to a NetWare 6 server using the newer NSS (Novell Storage Services) volumes, a bug which still exists. Chomp! There went another hunk of my backside. Boy, that bear is hungry.

Next month I'll cover yet another lost battle with the bear as well as our final victory over him. Meanwhile, could somebody please call a Medic?

©2002, Wayne M. Krakau