Mir ist ein interessanter Fehler untergekommen und zwar hatten alle VMs auf einem Host regelmäßige Ping Aussetzer. Alle VMs waren gleichzeitig für ca. 5 - 10 Sekunden nicht erreichbar. Der Zeitintervall zwischen den Aussetzern variierte. Ursache für das Phänomen war das Entfernen von einer LUN. Das System hat diesen Vorgang nicht sauber mitbekommen, trotz eines Rescans des Systems. Die Ping Aussetzer erfolgten zu dem Zeitpunkt, zu dem die Last der Anfrage an den Storage das System stark belastet hat, was aber weder zu einer hohen CPU-Last noch zu einer hohen Memory-Last geführt hat. Anbei Auszüge aus dem Log:
Jul 20 08:01:27 ESX1 vmkernel: 201:14:37:23.913 cpu5:4223)WARNING: NMP: nmp_DeviceAttemptFailover: Logical device "naa.6000d7740000c67552cc7dfcb758fff2": awaiting fast path state update...
Jul 20 08:01:28 ESX1 vmkernel: 201:14:37:24.915 cpu6:20894)WARNING: vmw_psp_rr: psp_rrSelectPathToActivate: Could not select path for device "naa.6000d7740000c67552cc7dfcb758fff2".
Jul 20 08:01:28 ESX1 vmkernel: 201:14:37:24.915 cpu0:4223)WARNING: NMP: nmp_DeviceAttemptFailover: Retry world failover device "naa.6000d7740000c67552cc7dfcb758fff2" - issuing command 0x410007075f00
Jul 20 08:01:28 ESX1 vmkernel: 201:14:37:24.915 cpu0:4223)WARNING: vmw_psp_rr: psp_rrSelectPath: Could not select path for device "naa.6000d7740000c67552cc7dfcb758fff2".
Jul 20 08:01:28 ESX1 vmkernel: 201:14:37:24.915 cpu0:4223)WARNING: NMP: nmp_DeviceAttemptFailover: Retry world failover device "naa.6000d7740000c67552cc7dfcb758fff2" - failed to issue command due to Not fo
und (APD), try again...
Jul 20 08:01:28 ESX1 vmkernel: 201:14:37:24.915 cpu0:4223)WARNING: NMP: nmp_DeviceAttemptFailover: Logical device "naa.6000d7740000c67552cc7dfcb758fff2": awaiting fast path state update...
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:25.914 cpu5:21232)WARNING: vmw_psp_rr: psp_rrSelectPathToActivate: Could not select path for device "naa.6000d7740000c67552cc7dfcb758fff2".
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:25.914 cpu7:4223)WARNING: NMP: nmp_DeviceAttemptFailover: Retry world failover device "naa.6000d7740000c67552cc7dfcb758fff2" - issuing command 0x410007075f00
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:25.914 cpu7:4223)WARNING: vmw_psp_rr: psp_rrSelectPath: Could not select path for device "naa.6000d7740000c67552cc7dfcb758fff2".
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:25.914 cpu7:4223)WARNING: NMP: nmp_DeviceAttemptFailover: Retry world failover device "naa.6000d7740000c67552cc7dfcb758fff2" - failed to issue command due to Not fo
und (APD), try again...
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:25.914 cpu7:4223)WARNING: NMP: nmp_DeviceAttemptFailover: Logical device "naa.6000d7740000c67552cc7dfcb758fff2": awaiting fast path state update...
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.100 cpu4:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.100 cpu4:4205)WARNING: NMP: nmp_DeviceStartLoop: NMP Device "naa.6000d7740000c67552cc7dfcb758fff2" is blocked. Not starting I/O from device.
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.100 cpu1:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.101 cpu7:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.101 cpu4:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 last message repeated 2 times
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.101 cpu1:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.101 cpu4:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 last message repeated 3 times
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.101 cpu6:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.101 cpu4:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.101 cpu7:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.101 cpu4:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.101 cpu7:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 last message repeated 2 times
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.101 cpu6:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 last message repeated 2 times
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.101 cpu4:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.101 cpu4:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 vmkernel: 201:14:37:26.101 cpu5:4205)ScsiDeviceIO: 747: Command 0x25 to device "naa.6000d7740000c67552cc7dfcb758fff2" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x5 0x25 0x0.
Jul 20 08:01:29 ESX1 last message repeated 2 times
Comments
Usache und Lösung
http://kb.vmware.com/kb/1016626