Fixing EMC VNX 5×00 naviagent not starting failover
Fixing EMC VNX 5300 naviagent not starting on failover control station when the failover actually occured.
** from emc eservice doc
The following is a Primus(R) eServer solution: ID: emc262797 Domain: EMC1 Solution Class: 3.X Compatibility Goal How to correct Naviagent errors when Control Station is failed over to CS1 Fact Product: Celerra Fact Product: VNX Series Fact Product: VNX Unified/File Fact Product: VNX VG8 Fact Product: VNX VG2 Fact EMC SW: NAS Code 6.x Fact EMC SW: NAS Code 7.x Fact EMC SW: NAS Code 5.6.50 and later Symptom Naviagent messages unexpectedly exited logged in /var/log/messages every five minutes for Navisphere agent "cannot create messages directories" Symptom Naviagent errors occurs when Control Station is failed over to CS1. Symptom Call home message: Slot 1 becomes primary because of timeout. Symptom /var/log/messages CS_PLATFORM:NaviEventMonitor:CRITICAL:4:::::CLARiiON event number 0x0 to /etc/Unisphere/backup/navimon.log. ExtCode1=107373568<http://remedytools.rtp.lab.emc.com/nsgview.pl?entry=107373568>,; ExtCode2=log. Symptom /var/log/messages show the following errors: Feb 11 11:22:19 emcnas_i0 EMCServer: nas_mcd: daemon "Apache daemon" unexpectedly exited (status = 0) ifexit=1 exitstatus=0 ifsignal=0 termsig=0 ifstop=0 stopsig=0 ifdump=0 Feb 11 11:22:19 emcnas_i0 EMCServer: nas_mcd: daemon "Naviagent" unexpectedly exited (status = 11) ifexit=0 exitstatus=0 ifsignal=1 termsig=11 ifstop=0 stopsig=0 ifdump=0 Feb 11 11:22:19 emcnas_i0 login(pam_unix)[27969]: session closed for user nasadmin Feb 11 11:22:22 emcnas_i0 Navisphere Agent[31217]: Error: Cannot create messages directory /etc/Unisphere/messages/ Feb 11 11:22:22 emcnas_i0 Navisphere Agent[31217]: Error: Cannot create event log backup directory /etc/Unisphere/backup/ Feb 11 11:22:22 emcnas_i0 Navisphere Agent[31217]: Open application log failed Error re-occurs after five minutes: Feb 11 17:27:34 emcnas_i0 Navisphere Agent[1287]: Error: Cannot create messages directory /etc/Unisphere/messages/ Feb 11 17:27:34 emcnas_i0 Navisphere Agent[1287]: Error: Cannot create event log backup directory /etc/Unisphere/backup/ Feb 11 17:27:34 emcnas_i0 Navisphere Agent[1287]: Open application log failed Cause Installations at these code levels don't create the symbolic link for the Naviagent service to start properly on CS1. The symbolic links created from installation point at /etc/Navisphere instead of /etc/Unisphere on CS1, which is what the naviagent service is looking for on start-up. Fix Create the symbolic links and directories while the Control Station is failed over to CS1: 1. ln -s /nas/etc/Navisphere/ /etc/Unisphere/ 2. mkdir /etc/Unisphere/messages 3. mkdir /etc/Unisphere/backup Monitor for naviagent service to start on next 5-minute interval: $ ps -ef | grep -i navia root 9198 2831 0 Feb11 ? 00:00:02 /nas/opt/Navisphere/bin/naviagent -d -f /nas/etc/Navisphere/agent.config -r /nas/log Note CS1 has been re-installed and problem persists.
