As mainframes have been the enterprise back-office foundation for decades, their work environment has changed. Vendors have consolidated, come and gone, introduced new products and sunsetted others. Where once staffers were able to specialize—perhaps being responsible for a few IBM subsystems or ISV products—people now typically cover broader areas. And as overall and component reliability has improved, you may occasionally face issues in areas in which you’ve not previously needed to diagnose or report problems. A positive mindset and a few simple practices will usually resolve what’s wrong.
Problem databases and your favorite search engine are your friends. A variation on RTFM (though that’s important, too, confirming that something really is wrong!) is researching problems before reporting them:
• Search vendor databases.
• Use your favorite search engine.
• Consult relevant mailing lists—IBM-MAIN, IBMVM, VSE-L, LINUX-390, specialized lists for CICS, Assembler, etc.
• If timing allows, query local or national user group sessions or forums, etc.
It’s efficient to perform quick initial research yourself because you know your environment and understand what’s going on. Best of all, this might let you rebut what you hate to hear: “You’re the only site reporting the problem.” But don’t bog down doing this; vendor support staffers usually know their products well enough to quickly search and categorize problems as known or new. For known problems you can learn status and projected resolution timing, and be added to an “interested parties” list. When reporting a problem, however, keep in mind the following:
Be professional. Treat it like a sales call. Establish your competence and desire to collaborate, communicate urgency of problem and its potential impact, guide conversation towards your desired resolution and avoid wasted interactions by being told to gather more information and call back.
Be prepared. Learn vendors’ problem-reporting facilities before you need them and register if required; don’t wait for a severe problem as motivation. Collect as much information as possible before asking for help. As a bonus, gathering problem details and considering relevant factors can often allow solving problems oneself! Preserve documentation that might be useful so you needn’t resort to archives/backups or have to recreate problems.
Be specific. You won’t get useful answers and solutions without asking the right questions and describing problems at hand—just as with your doctor or auto mechanic. Recreating the problem may let you focus on otherwise missed details. Report problems as separate incidents to avoid confusion and losing track of anything.
Provide technical details. Describe problem searches done and your environment and confess if you’re behind on maintenance. Avoid misdirection by distinguishing facts from guesses and interpretations. For batch-job problems, include a full job log rather than just one error message and perhaps console messages for an interval before the problem occurred. Don’t omit details—it’s better to have too much information than to miss a crucial factor. Of course, mention whether it’s a first-time (or repeatable!) problem and, if the latter, whether and how you handled it. Think carefully about what (hardware, software, settings, network, anything) changed recently. It’s awkward remembering a relevant change after you assure the vendor that “nothing changed.” If you’re submitting online, reread your query, imagining you know nothing about the problem except what’s written. Add what’s missing that someone else needs to know. Providing a simple problem recreation scenario can move your issue ahead of those requiring intense research (and often leads to discovering a solution on your own!). Offering to run traces and collect other reasonable data will make you the vendor’s friend. And as more mainframe tools and products acquire graphical interfaces, be familiar with screen capture/print facilities for simple evidence gathering. In worst-case situations, use a phone camera for recording.
Beware misleading problem counts. Searching or talking to support might reveal a product’s problem count. But raw numbers don’t translate to product quality. A low count can indicate a solid and mature product or something not being used much. And a high count can reveal severely flawed software or new code being used enthusiastically. Years ago, a high APAR rate for z/VM’s newly introduced XEDIT editor made IBM wonder whether releasing it had been a mistake. SHARE members responded that people only report errors in software if they care enough to go to the trouble (and are actually using it). XEDIT, of course, was one of the most valuable additions to z/VM. A counter-example was another z/VM product with low APAR rate because it was rarely used.
Set severity. Be clear about your overall goal; don’t focus on an intermediate step you think is necessary—you might distract someone into addressing how you’re trying to do something, rather than what you actually want to do. Understand that “doesn’t do what I want” doesn’t mean “broken.” Remember that vendors deal differently with problems, suggestions and questions. Recognize that vendor resource limitations (even IBM’s) might prevent fixing some genuine problems. So set appropriate priorities—everything can’t be equally important. Be reasonable; don’t complain for the sake of complaining. And unless they’re equally urgent, don’t submit too many problems at once.
Debug collaboratively. Get a unique problem-tracking number, ticket/incident ID—whatever it’s called—to designate each matter. Understand vendor follow-up procedures; note how you prefer to communicate—email, phone, instant message. Respond quickly to requests for information, lest you lose a support person’s attention, and submit information in the requested format/manner. Develop procedures and tools to sanitize sensitive installation information so you can comply with requests and needn’t delay responding. Get to know support staff, build relationships, request and cherish individual support staffers’ contact information (but only use it in emergencies). As you work with an individual or a group, collaborate—don’t try to overpower them. Be respectful, not belligerent, and don’t make them feel stupid even if you feel they are. They might have missed a detail or you might not have conveyed the whole story. Don’t try to “pull rank” based on age or experience or professional credentials. Mention past interactions with the company, emphasizing positive outcomes. Avoid threats, and never ask, “Do you know who I am?” Don’t offer a choice between an easy way and a hard way for problem resolution. Support staffs are often constrained by company policy or—worse—scripts to read. When I thought I was being “scripted,” I’ve had some success by asking whether, off the record as one IT professional to another, we can reshape the conversation.
Track problems. Log all vendor interactions, whether telephone, email, online, FTP, etc. On a sheet of paper (or in one disk file/directory), note vendor contact information (phone or email), problem number and problem description. Add time-stamped text describing what you’re told (e.g., fix to try, request for more documentation). Doing this has two benefits: It lets someone else pick up the problem trail if you’re not available; and if conflicts arise over the support process, it’s ammunition to use with unresponsive vendors or questioning bosses. Include this identification with all communications and information submitted.
Follow up. Depending on problem/severity you might expect substantive response in a few hours, days, or (hopefully not) weeks. Understand vendor policies for handling problems of various severities and use your tracking information to set a next expected interaction; when it’s overdue, nudge the vendor. If it’s severely overdue—or you’re getting unresponsive responses—escalate. That can involve contacting a higher support level, product manager, sales personnel or company executive. At some point it’s worth involving someone senior in your organization. Especially for this, detailed problem tracking information can be essential and convincing.
Finally, Eric Raymond—an influential leader of the open source movement—posts a longish tips document
, though it’s a little too harsh in places for my taste. But comments on how to ask questions and where to seek help are interesting and helpful.
Gabe Goldberg has developed, worked with and written about technology for decades. He can be contacted at email@example.com.