TFS 2008 Build Service will not start, with exception: "The underlying connection was closed: An unexpected error occurred on a receive. (type WebException)"

Yesterday, the build agent on the TFS 2008 of our client suddently stopped working after a reboot. I was assured that No settings had changed on the server, so it was a bewildering problem. Several errors started to occur in the event logs such as "Detailed Message: TF224002: An unexpected error has occurred. Exception Message: The underlying connection was closed: An unexpected error occurred on a receive. (type WebException)".
As a symptom of the problem, the TFS 2008 Build Agents would never initialize and would constantly go to "Unreachable" status on the server.

The full exception in the event log was as follows:

TF53010: The following error has occurred in a Team Foundation component or extension:

Date (UTC): 23/02/2011 11:06:06 PM
Machine: TFS1-MYSERVER
Application Domain: TFSBuildService.exe
Assembly: TfsBuildService, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; v2.0.50727
Process Details:
  Process Name: TFSBuildService
  Process Id: 6256
  Thread Id: 5132
  Account name: MYDOMAIN\TFSService

Detailed Message: TF224002: An unexpected error has occurred.
Exception Message: The underlying connection was closed: An unexpected error occurred on a receive. (type WebException)

Exception Stack Trace:    at System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(WebRequest request)
   at System.Web.Services.Protocols.HttpWebClientProtocol.GetWebResponse(WebRequest request)
   at Microsoft.TeamFoundation.Client.TeamFoundationSoapProxy.GetWebResponse(WebRequest request)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at Microsoft.TeamFoundation.Proxy.BisRegistrationServiceProxyWsdl.GetRegistrationEntries(String toolId)
   at Microsoft.TeamFoundation.Proxy.BisRegistrationProxy.GetRegistrationEntries(String toolId)
   at Microsoft.TeamFoundation.Proxy.BisRegistrationService.RefreshMemoryCache()
   at Microsoft.TeamFoundation.Proxy.BisRegistrationService.RefreshCachesIfNeeded(Boolean direct)
   at Microsoft.TeamFoundation.Proxy.BisRegistrationService.GetRegistrationEntries(String toolId)
   at Microsoft.TeamFoundation.Build.Client.BuildServer.Microsoft.TeamFoundation.Client.ITeamFoundationServerObject.Initialize(TeamFoundationServer tfs)
   at Microsoft.TeamFoundation.Client.TeamFoundationServer.CreateITFSObjectInstance(Assembly assembly, String fullName)
   at Microsoft.TeamFoundation.Client.TeamFoundationServer.GetService(Type serviceType)
   at Microsoft.TeamFoundation.Build.Agent.AgentService.InitializeTeamFoundationServer(String callingAT)
   at Microsoft.TeamFoundation.Build.Agent.AgentService.GetBuildInProgress(String tfsUrl, String teamProject)
   at SyncInvokeGetBuildInProgress(Object , Object[] , Object[] )
   at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
   at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)

Inner Exception Details:

Exception Message: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. (type IOException)

Exception Stack Trace:    at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
Inner Exception Details:
Exception Message: An existing connection was forcibly closed by the remote host (type SocketException)
Exception Stack Trace:    at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)

The problem in our case was that we had (apparently) new problems with our proxy configuration in Internet Explorer (IE). Fiddler had been installed on this machine several months prior to me arriving at the client and it stopped working. I removed all references to Fiddler (port 8888 by default) in the proxy configuration and the build started to work correctly.

So this is an important reminder that - the Build Agent in TFS 2008 requires local HTTP access to resources. It uses IE proxy settings during every build run. If these are not correct, TFS will cause you absolute grief and fall over left,right and centre.

Now to the Fiddler problem...

DDK