Sharing this blog post about the recent crashes that we have faced with IIS, Sharepoint farms with multiple clients.
Event ID 1000 was observed on all the servers.
w3wp crash, app pool was getting stopped, websites becoming unavailable.
This had caused Critical issues and we immediately contacted Microsoft and provided the crash dump which they analyzed and reason for this behavior was found.
Heap corruption (0xC0000374) was observed in Crash dump. This is a known-issue caused by SCOM agent.
More details: https://docs.microsoft.com/en-us/sharepoint/support/administration/system-center-2016-operations-manager-apm-agent-causing-heap-corruption
System Center 2016 Operations Manager APM Agent causing heap corruption in SharePoint
Uninstalled SCOM agent from servers and rebooted them (This resolved the issue)
We had to reinstall the SCOM agent with the switch mentioned in the article msiexec /i MonAgent.msi NOAPM=1
Note: Article mentions System Center 2016 Operations Manager but we have faced it with 2012 version as well and switch we used was below:
msiexec /i MomAgent.msi NOAPM=1 (instead of MonAgent it is MomAgent in 2012 version)
Microsoft Engineer confirmed the bug existence in 2012 as well.
If you are facing the similar crashes then issue may be because of the SCOM agent bug as without the above switch it installs its on binaries of .net that causes a conflict with existing .net version on the OS.
I hope this small post will save you hours of work...
PowerShell Cheat Book