EnterpriseOne IBM iSeries (AS400) Power Outage Fix
POWER OUTAGE IN THE DATA CENTER!
WHAT!?! That’s not supposed to happen.
Don’t I know it. Don’t ask.
After this unfortunate incident, EnterpirseOne came back up great. Nothing in the logs indicating an issue and no issues reported by users.
That is until several users logged in and the system came to a screeching halt. Grids wouldn’t populate. UBE jobs stayed in a “Waiting” status. And eventually, the following error was found in the enterprise server logs:
IPC2100017 – createIPCMsgq (name Net32Q) failed, errno=3457: File exists..
It was a very frustrating day, but thankfully we were able to resolve it using Oracle Support Doc 659250.1 :
Cause
Invalid files exist in system library.
UBEs on the iSeries process in USRQs (User Queues).? When the UBE is done processing, the USRQ should be deleted by EnterpriseOne. On occasion, USRQs do not get deleted, leaving orphaned USRQs in the system.? If another UBE is submitted to the server, it may try to use the orphaned USRQ causing the error message above.? To resolve this issue, the orphaned USRQ(s) need to be deleted.
Solution
Steps to delete a USRQ:
- ENDNET to end oneworld services using AS400 user profile ONEWORLD
- CLRIPC
- Issue command:? WRKOBJ E910SYS/Q0000*? –? where E910SYS is your E1 system library name
- If the object type is *USRQ you can delete these orphaned user queues.
NOTE: If there are no *USRQs, this is not the problem and further investigation is warranted.- CLRIPC
- STRNET to start JDE services
- Start HTML server
Nice tip! Thanks.
Is there any issue with adding this into our regular stop/start scripts?
Thanks, David. I don’t think there would be any issue with that. However, I’m definitely not an iSeries guru so I would make sure with your friendly neighborhood IBM iSeries Pro.
OMG Thank you so much for the help, what a life saver!
System backup and we are ready for the day well before users log in.
Glad this could help, Francine! Crazy that it happens enough that they have to have a process documented for it.