Fix for SharePoint 2007 Deployment Issue - Solutions in Permanent state of "Deploying", Application Server Administration Service Timer Job in permanent State of 'Initialized'

If you encounter the following issues when attempting to deploy a SharePoint solution (wsp):

1. It is permanently/constantly in a status of "Deploying" (as seen in Central Administration - Operations - Solution Management)
2. You cannot cancel the Deployment (no matter how many times you click on 'Cancel')
3. The SharePoint Timer job definitions are permanently in a status of "Initialized" (as seen in Central Administration - Operations - Timer Job Status).
4. Your SharePoint Log files are being flooded with the following 5uuf error and growing extremely large (one of ours got to 3GB!):
   
   05/21/2010 11:41:44.87 OWSTIMER.EXE (0x08E0) 0x08E8 Windows SharePoint Services Timer 5uuf Monitorable The previous instance of the timer job 'Config Refresh', id '{604B2E6E-5850-4C95-8015-D49A61449456}' for service '{681C12E2-4C2C-4BB5-9C9C-BCCF5B4FF5BE}' is still running, so the current instance will be skipped. Consider increasing the interval between jobs.

This is due to an invalid configuration cache on one or or all of the servers. What was happening in my situation was that:

1. The SharePoint Configuration databasse went down.
2. The XML configuration files on SERVER02 were updated by SharePoint but not on SERVER01
3. In fact, there were 500 XML files on one server and 520 on the other whereas they should always be in sync.

However, is is not sufficient to JUST do it on the central admin server. You have to follow this blog entry (http://blogs.msdn.com/josrod/archive/2007/12/12/clear-the-sharepoint-configuration-cache-for-timer-job-and-psconfig-errors.aspx) or (http://support.microsoft.com/kb/939308)  and perform a similar process (as desribed above on each server.)

You must do the following:

1. Stop the OWSTIMER service on the Index Server and then ALL of the MOSS servers in the farm (for me it was the 2 servers mentioned above e.g. SERVER01 and SERVER02. Just use the following at a command prompt to do this on EVERY WFE and INDEX SERVER
   net stop "Windows SharePoint Services Timer"
2. [DO FOR ALL SERVERS BEFORE RESTARTING THE TIMER SERVICE]
   Go to C:\Documents and Settings\All Users\Application Data\Microsoft\SharePoint\Config\
   Move (not just copy) all the xml config files (don't delete the config file or the folder itself, just all the xml files) to another location (e.g. a "zz" folder") in a temporary directory as a backup.
3. [DO FOR ALL SERVERS BEFORE RESTARTING THE TIMER SERVICE]
   Open the cache.ini with Notepad and reset the number to 1. Save and close the file.
4. Once all XML files are removed and the cache.ini files reset to 1 for ALL SERVERS, run
   net stop "Windows SharePoint Services Timer"on the Index Server first.
5. Once all the Xml files are generated, run
   net stop "Windows SharePoint Services Timer" on Query Servers and Web Front End Servers in turn.

DDK


Additional Blog Note - how to remove unwanted solutions that are stuck in a "Deploying"State (without fixing the underlying Config problems as detailed above)

1. Reboot Server (in line with my "If in doubt, restart it!" motto). This in effect will restart all the services anyway.
2. Use the following command to get a list of all solutions that are being deployed.
   stsadm -o enumdeployments
3. Use the JobId GUID that comes back from this e.g. Deployment JobId="e99b7304-cfc0-419a-a3f2-18ca5193c838"
4. Cancel the "stuck" deployment (in "Deploying" status" with the following command
   stsadm -o canceldeployment -id e99b7304-cfc0-419a-a3f2-18ca5193c838
5. Delete the stuck solution once and for all with
   stsdm -o deletesolution -name mysolution.wsp -override
6. Redeploy your solution.