Error: Failed to compile both on .NET 4.5 and on .NET 4.0. 4.5 Exception: System.IO.IOException: The file exists.

Today I had a weird issue with a Microsoft IIS web-server. The latest .NET Framework was correctly installed and registered on the server, but when I put my .NET code in the web directory and browsed to it from a remote browser I got the standard .NET error page. I would get the error regardless of whether I was trying to access a page served by .NET or a basic HTML page hosted on the server. I confirmed that IIS was working correctly by removing all files (including the web.config file) from the web-directory and putting in a basic test HTML page on its own. The test page rendered correctly from a remote browser.

Having put the .NET code back in, I browsed to a .NET URL locally on the web-server in order to get a full view of the error message. The rather strange error was as follows:

File Exists Error

In order to help people find this page in search engines, the full text was as follows:

Server Error in '/' Application.

Failed to compile both on .NET 4.5 and on .NET 4.0. 4.5 Exception: System.IO.IOException: The file exists.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Path.InternalGetTempFileName(Boolean checkHost)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileInternally(String fileName, List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String outputFolder, String compilerVersion, Int32 tryCount)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileAssembly(List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String assemblyFileName, String outputFolder). 4.0 Exception: System.IO.IOException: The file exists.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Path.InternalGetTempFileName(Boolean checkHost)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileInternally(String fileName, List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String outputFolder, String compilerVersion, Int32 tryCount)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileAssembly(List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String assemblyFileName, String outputFolder) 
  Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. 

 Exception Details: System.InvalidOperationException: Failed to compile both on .NET 4.5 and on .NET 4.0. 4.5 Exception: System.IO.IOException: The file exists.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Path.InternalGetTempFileName(Boolean checkHost)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileInternally(String fileName, List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String outputFolder, String compilerVersion, Int32 tryCount)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileAssembly(List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String assemblyFileName, String outputFolder). 4.0 Exception: System.IO.IOException: The file exists.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Path.InternalGetTempFileName(Boolean checkHost)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileInternally(String fileName, List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String outputFolder, String compilerVersion, Int32 tryCount)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileAssembly(List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String assemblyFileName, String outputFolder)

Source Error: 


 An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.  

Stack Trace: 



[InvalidOperationException: Failed to compile both on .NET 4.5 and on .NET 4.0. 4.5 Exception: System.IO.IOException: The file exists.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Path.InternalGetTempFileName(Boolean checkHost)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileInternally(String fileName, List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String outputFolder, String compilerVersion, Int32 tryCount)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileAssembly(List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String assemblyFileName, String outputFolder). 4.0 Exception: System.IO.IOException: The file exists.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Path.InternalGetTempFileName(Boolean checkHost)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileInternally(String fileName, List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String outputFolder, String compilerVersion, Int32 tryCount)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileAssembly(List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String assemblyFileName, String outputFolder)]
   HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileAssembly(List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String assemblyFileName, String outputFolder) +712

[InvalidOperationException: System.InvalidOperationException: Failed to compile both on .NET 4.5 and on .NET 4.0. 4.5 Exception: System.IO.IOException: The file exists.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Path.InternalGetTempFileName(Boolean checkHost)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileInternally(String fileName, List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String outputFolder, String compilerVersion, Int32 tryCount)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileAssembly(List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String assemblyFileName, String outputFolder). 4.0 Exception: System.IO.IOException: The file exists.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Path.InternalGetTempFileName(Boolean checkHost)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileInternally(String fileName, List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String outputFolder, String compilerVersion, Int32 tryCount)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileAssembly(List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String assemblyFileName, String outputFolder)
   at HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileAssembly(List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String assemblyFileName, String outputFolder)
ImageRuntimeVersion: v4.0.30319]
   HibernatingRhinos.Profiler.Appender.Util.GenerateAssembly.CompileAssembly(List`1 sources, HashSet`1 assemblies, HashSet`1 defineSymbols, String assemblyFileName, String outputFolder) +1042
   HibernatingRhinos.Profiler.Appender.NHibernate.NHibernateProfiler.RegisterAppenderUsingNHibernateLogger() +314
   HibernatingRhinos.Profiler.Appender.NHibernate.NHibernateProfiler.Initialize(NHibernateAppenderConfiguration configuration) +53
   XXXXX.App_Start.NHibernateProfilerBootstrapper.PreStart() in c:\TeamCity\buildAgent\work\XXXXX\App_Start\NHibernateProfilerBootstrapper.cs:18

[TargetInvocationException: Exception has been thrown by the target of an invocation.]
   System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) +0
   System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) +260
   System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) +146
   System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters) +34
   WebActivator.BaseActivationMethodAttribute.InvokeMethod() +288
   WebActivator.ActivationManager.RunActivationMethods() +698
   WebActivator.ActivationManager.RunPreStartMethods() +46
   WebActivator.ActivationManager.Run() +68

[InvalidOperationException: The pre-application start initialization method Run on type WebActivator.ActivationManager threw an exception with the following error message: Exception has been thrown by the target of an invocation..]
   System.Web.Compilation.BuildManager.InvokePreStartInitMethodsCore(ICollection`1 methods, Func`1 setHostingEnvironmentCultures) +874
   System.Web.Compilation.BuildManager.InvokePreStartInitMethods(ICollection`1 methods) +169
   System.Web.Compilation.BuildManager.CallPreStartInitMethods(String preStartInitListPath, Boolean& isRefAssemblyLoaded) +166
   System.Web.Compilation.BuildManager.ExecutePreAppStart() +176
   System.Web.Hosting.HostingEnvironment.Initialize(ApplicationManager appManager, IApplicationHost appHost, IConfigMapPathFactory configMapPathFactory, HostingEnvironmentParameters hostingParameters, PolicyLevel policyLevel, Exception appDomainCreationException) +882

[HttpException (0x80004005): The pre-application start initialization method Run on type WebActivator.ActivationManager threw an exception with the following error message: Exception has been thrown by the target of an invocation..]
   System.Web.HttpRuntime.FirstRequestInit(HttpContext context) +579
   System.Web.HttpRuntime.EnsureFirstRequestInit(HttpContext context) +112
   System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context) +716

  


Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.6.1055.0  

I found this article on stack overflow which helped me to solve the problem. The answer was simply to clear down the %TEMP% folder, which for me was C:\windows\temp. Apparently, if there are more than 65,535 files in there .NET is not able to serve web-pages through IIS. There were hundreds of thousands of files in the folder so I couldn’t even browse to it using Windows Explorer. It ended up being a command prompt job:

File Exists Error Command

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Windows\system32>cd c:\Windows\Temp

c:\Windows\Temp>del c:\Windows\Temp\*.*
c:\Windows\Temp\*.*, Are you sure (Y/N)? y

That fixed the problem.

Using Visual Studio to run web-sites locally using real domains

Visual Studio is great for developing and testing web-sites, but by default it runs them under a localhost address using a randomly generated port number, as given in the property page for the web-site:

Visual Studio Real Domains 01

This is fine, but sometimes it’s useful to have sites run at a location that looks more like a real internet URL, especially if you plan on including functionality that manipulates the URL for redirects etc… I’ve also encountered issues creating cookies using the default http://localhost:xxxx address.

Fortunately, there’s a hosts file trick you can use to make Visual Studio run sites under more realistic URLs. First point 127.0.0.1 at the URL you want to use in your hosts file, for example:

127.0.0.1 www.testing.com

Note though that if the URL you are using points to a real web-site on the internet, you won’t be able to access that site again until you remove the entry from your host file.

Then edit the property pages so that you’re using the Visual Studio Development Server, a specific port (rather than an auto-assigned port) and a custom start URL containing your chosen port number and test URL, for example:

http://www.testing.com:1455

Your property pages should look like this:

Visual Studio Real Domains 02

Then, when you run your web-site using Visual Studio, it will use http://www.testing.com:1455 as the base URL. All paths relative to that should work as expected:

Visual Studio Real Domains 03

NHibernate and log4net

Here’s a quick trick for people using NHibernate and log4net within their .NET applications. (…and if you’re not using NHibernate or log4net, you should seriously think about it!)

Add the following to your log4net configuration to find out what’s going on inside NHibernate:

  <logger name="NHibernate">
    <level value="DEBUG" />
  </logger>
  <logger name="NHibernate.SQL">
    <level value="DEBUG" />
  </logger>

As those kind folks at NHibernate have already set up the loggers behind the scenes, everything just works. The SQL logger is especially useful as it’ll show you explicitly which SQL statements are being generated by NHibernate. You can adjust the levels of NHibernate logging independently of your normal application logging, which is useful as the NHibernate logs tend to be pretty verbose.

The full configuration will look something like this:

<log4net>
  <appender name="TraceAppender" type="log4net.Appender.TraceAppender" >
    <layout type="log4net.Layout.PatternLayout">
      <param name="ConversionPattern" value="%d %-5p- %m%n" />
    </layout>
  </appender>
  <appender name="RollingFileAppender" type="log4net.Appender.RollingFileAppender">
    <file value="Application.log"/>
    <appendToFile value="true"/>
    <rollingStyle value="Date"/>
    <datePattern value="yyyy-MM-dd'.log'"/>
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline"/>
    </layout>
  </appender>
  <root>
    <level value="DEBUG"/>
    <appender-ref ref="TraceAppender"/>
    <appender-ref ref="RollingFileAppender"/>
  </root>
  <logger name="NHibernate">
    <level value="ERROR" />
  </logger>
  <logger name="NHibernate.SQL">
    <level value="ERROR" />
  </logger>
</log4net>

Basic Log4Net step-up

The following post explains how to get Log4Net working in a .NET console application.

The first step is create your console application and add references to Log4Net. The easiest way to add the correct references is via NuGet, where it is simply listed as log4net.

You’ll need to add a configuration file to tell Log4Net how you want it to log. A basic example would be:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  
  <configSections>
    <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler,log4net" />
  </configSections>

  <log4net>
    <appender name="RollingFile" type="log4net.Appender.RollingFileAppender">
      <file value="Test.log" />
      <appendToFile value="true" />
      <maximumFileSize value="5000KB" />
      <maxSizeRollBackups value="50" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%utcdate{yyyy-MM-dd HH:mm:ss.fffffff} - %level - %logger - %message%newline" />
      </layout>
    </appender>
    <root>
      <level value="DEBUG" />
      <appender-ref ref="RollingFile" />
    </root>
  </log4net>
  
</configuration>

This example simply adds a rolling log file. It’s called rolling because it starts a new log file at triggered intervals, allowing you to delete historical log files when you’ve finished with them. In this case, a new log file is started each time the current log file gets to 5000KB. It’s worth taking a look at the Log4Net documentation to find out about the different types of logging available. It is also possible to write your own appender if you want to do something not already built it to Log4Net.

In order to tell Log4Net to pick your configuration, you need to add the following to your application:

[assembly: XmlConfigurator]

The most obvious place for this is the AssemblyInfo.cs file.

This tells Log4Net to look in the default configuration file for the Log4Net configuration settings, but you can also tell Log4Net to look in another custom configuration file using the overloads available to the XmlConfigurator, although I prefer to keep all configuration in a single file.

Then it’s just a case of using the logger:

using System;
using log4net;

namespace Test
{
    public static class Program
    {
        public static void Main()
        {
            var log = LogManager.GetLogger(typeof(Program));
            log.Debug("Test console application started");

            Console.ReadLine();
        }
    }
}

Note that the LogManager returns an object implementing the ILog interface.

If you’re using dependency injection, you should wire up the dependency injector to return your ILog instance. The following shows how to do this if you’re using Castle Windsor:

using Castle.MicroKernel.Registration;
using Castle.Windsor;
using Castle.Windsor.Installer;
using log4net;

namespace Test
{
    public static class Injector
    {
        private static readonly object InstanceLock = new object();

        private static IWindsorContainer instance;

        public static IWindsorContainer Instance
        {
            get
            {
                lock (InstanceLock)
                {
                    return instance ?? (instance = GetInjector());
                }
            }
        }

        private static IWindsorContainer GetInjector()
        {
            var container = new WindsorContainer();

            container.Install(FromAssembly.This());

            RegisterInjector(container);
            RegisterLog(container);

            return container;
        }

        private static void RegisterInjector(WindsorContainer container)
        {
            container.Register(
                Component.For<IWindsorContainer>()
                    .Instance(container));
        }

        private static void RegisterLog(WindsorContainer container)
        {
            container.Register(
                Component.For<ILog>()
                    .Instance(LogManager.GetLogger(typeof(Injector)))
                    .LifestyleSingleton());
        }
    }
}

The following statement will return an ILog instance as expected:

var log = Injector.Instance.Resolve();

There you have it!

Changing the default naming convention for foriegn keys when using Fluent NHibernate

By default, NHibernate will assume that foreign keys should be named using the convention ReferencedTableName_id (e.g. Person_id). I personally don’t like this conversion and would prefer ReferencedTableNameId (e.g. PersonId).

Fortunately, this is pretty easy to achieve by extending the ForeignKeyConvention class in Fluent NHibernate:

using System;
using FluentNHibernate;
using FluentNHibernate.Conventions;

namespace NHibernate.Helpers.Conventions
{
    public class CustomForeignKeyConvention : ForeignKeyConvention
    {
        protected override string GetKeyName(Member property, System.Type type)
        {
            if (property != null)
            {
                return property.Name + "Id";
            }

            if (type != null)
            {
                return type.Name + "Id";
            }

            // If both are null, we throw an exception:
            throw new ArgumentNullException("property", "The property and type parameters cannot both be null");
        }
    }
}

This needs to be wired into your SessionFactory creation code as follows:

sessionFactory = Fluently.Configure()
    .Database(DatabaseConfiguration)
    .Mappings(x => x.FluentMappings.AddFromAssembly(MappingsAssembly))
    .Mappings(x => x.FluentMappings.Conventions.Add(new CustomForeignKeyConvention()))
    .ExposeConfiguration(x => new SchemaUpdate(x).Execute(false, true))
    .BuildSessionFactory();

Note that the example above will also update the database schema.

Once you’ve done that, all foreign keys will use the desired naming convention.

Testing Windows Services with dependency injection

Last year I wrote a post on Testing Windows Services. I pointed out that recompiling and installing your Windows Service following each change is a really inefficient way to develop and test a service. Instead, I suggest that you can run and debug your service in Visual Studio simply by wrapping the service up in a test console application.

I said that I would revisit the post for services using dependency injection, so here goes. Note that the code in this article follows on directly from the code in the previous article, so you’ll need to read the previous article in order for this to make sense!

First, a quick discussion as to how you could add dependency injection to a Windows Service:

I believe the easiest way to use dependency injection in a .NET Windows Service application is to use the dependency injection engine to inject required services into the ServiceBase.Run() method in the Program class for the service:

using System.ServiceProcess;

namespace MyService
{
    public static class Program
    {
        public static void Main()
        {
            var servicesToRun = new[] { (ServiceBase)Injector.Instance.Resolve() };
            ServiceBase.Run(servicesToRun);
        }
    }
}

In order to do this, you simply need to create an empty interface for your service, which the service implements:

namespace MyService
{
    public interface IService
    {
        // No implementation
    }
}
using System.IO;
using System.ServiceProcess;

namespace MyService
{
    public partial class Service : ServiceBase, IService
    {
        public Service()
        {
            InitializeComponent();
        }

        // Service implementation...
    }
}

My original article gave an example of a service which creates and deletes a file when it starts and stops. To complete the example, let’s extract the file interaction code out as a dependency. Here’s the interface:

namespace MyService
{
    public interface IFileCreator
    {
        void Create();
        void Delete();
    }
}

…and here’s the implementation:

using System.IO;

namespace MyService
{
    public class FileCreator : IFileCreator
    {
        public void Create()
        {
            using (var stream = new FileStream("Running", FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite))
            {
                using (var writer = new StreamWriter(stream))
                {
                    writer.WriteLine("Running");
                    writer.Flush();
                }
            }
        }

        public void Delete()
        {
            var fileInfo = new FileInfo("Running");
            if (fileInfo.Exists)
            {
                fileInfo.Delete();
            }
        }
    }
}

The injector class for the service would then look something like this:

using Castle.MicroKernel.Registration;
using Castle.Windsor;
using Castle.Windsor.Installer;

namespace MyService
{
    public static class Injector
    {
        private static readonly object InstanceLock = new object();

        private static IWindsorContainer instance;

        public static IWindsorContainer Instance
        {
            get
            {
                lock (InstanceLock)
                {
                    return instance ?? (instance = GetInjector());
                }
            }
        }

        private static IWindsorContainer GetInjector()
        {
            var container = new WindsorContainer();

            container.Install(FromAssembly.This());

            RegisterInjector(container);
            RegisterService(container);
            RegisterFileCreator(container);

            return container;
        }

        private static void RegisterInjector(WindsorContainer container)
        {
            container.Register(
                Component.For()
                .Instance(container));
        }

        private static void RegisterService(WindsorContainer container)
        {
            container.Register(
                Component.For()
                    .ImplementedBy(typeof(Service)));
        }

        private static void RegisterFileCreator(WindsorContainer container)
        {
            container.Register(
                Component.For()
                    .ImplementedBy(typeof(FileCreator)));
        }
    }
}

Note that I’ve wired up both the service (using IService) and my new FileCreator class.

The ServiceWrapper class would be exactly as in the previous example. The only change to the wrapper console application would be to change the Program, as follows:

using System;
using MyService;

namespace MyServiceWrapper
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var serviceWrapper = new ServiceWrapper();

            // Wire up service dependencies here:
            serviceWrapper.FileCreator = Injector.Instance.Resolve();

            try
            {
                serviceWrapper.TestStart();
                Console.ReadLine();
                serviceWrapper.TestStop();
            }
            catch (Exception exception)
            {
                Console.WriteLine("Error: " + exception.Message);
                Console.ReadLine();
            }
        }
    }
}

Unfortunately, in this class I’ve had to wire up all the dependencies in the ServiceWrapper manually as I haven’t found a way to do this automatically. However, this one small pain point is a significant improvement compared to compiling and installing your service each time you make a change!

Removing duplicates with LINQ in C#

Here are a couple of tricks to remove duplication using LINQ.

Firstly, for simple lists:

var numbersWithDuplicates = new[] { "One", "Two", "Two", "Three", "Four" };
var numbers = numbersWithDuplicates.GroupBy(x => x).Select(group => group.First());

…and secondly for more complex objects:

public class Person
{
    public int Id { get; set; }
    public string Name { get; set; }
    public int Age { get; set; }
}

…you can de-duplicate using any property you like:

var peopleWithDuplicates = new[]
{
    new Person { Id = 2, Name = "Alice", Age = 29 },
    new Person { Id = 1, Name = "Bob", Age = 30 },
    new Person { Id = 3, Name = "Claire", Age = 31 },
    new Person { Id = 4, Name = "Bob", Age = 50 },
};
var people = peopleWithDuplicates.GroupBy(x => x.Name).Select(group => group.First());

Prevent malicious image uploads

If you allow users to upload images to your site, you may be opening up your site to malicious use. This is because particularly cunning hackers can hide commands to do just about anything in the data that makes up the image.

For a great example of an infected image file, see the following page:

http://www.eicar.org/86-0-Intended-use.html

Note that you’ll need to disable your anti-virus if you want to create this file on your computer.

If you allow users to upload this file, it will produce some interesting results!

To prevent the vulnerability (at least when using C#) you simply need to try and load the image data into an Image object. This will throw an exception if the image contains anything nasty. Here is an example:

var image = Image.FromStream(imageDataStream);
using (var memoryStream = new MemoryStream())
{
    image.Save(memoryStream, ImageFormat.Png);
}

Note that you’ll need to load the uploaded data into a stream (imageDataStream in the example above) to get this to work.

Ignore SSL errors when calling web-services in C#

Often when developing .NET systems that call secure web-services, I find myself running code against development and test servers with invalid SSL certificates. This causes service calls to fail and prevents me from making progress. To get round this potential blocker, I include the following snippet somewhere in the code before the service call is made:

ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };

This code only needs to appear once, and it is not good practice to leave it in production code as it means that data sent over HTTPS isn’t actually secure. Consider making the inclusion of this code configurable, or wrap it up in #DEBUG statements so that it is not compiled into production code.

Generic retry loops

Now more than ever I’m finding that the systems I write are required to communicate with a whole host of other systems. With “cloud computing” becoming ever more popular this trend will no doubt continue. Two obvious examples are databases and web-services (either directly or through APIs). Systems that rely on other applications and end-points naturally have extra failure points that we need to consider. More often than not, when calls to external end-points fail the problems are transient. That is to say if we try the same operation a little later the call magically works. There are many reasons why we experience transient errors, with some examples being insufficient band-width, transmission errors or servers being too busy to respond in a timely fashion. Of course, the end-point could be permanently down so our code should handle this gracefully, but usually pausing and retrying usually solves the problem.

Implementing a retry loop isn’t a difficult task, but given a class that calls a number of different methods of the same remote end-point, we don’t want to duplicate the retry code for each call. Fortunately, a relatively new feature in .NET makes implementing generic retry loops much easier. Using our old friend the console application (and with the same disclaimer about client profiles as always) I’ve written a calculator class that we will be calling using a retry loop:

using System;

namespace RetryLoops
{
    public class Calculator
    {
        private readonly Random random;

        public Calculator()
        {
            unchecked
            {
                random = new Random((int)DateTime.Now.Ticks);
            }
        }

        public int Add(int x, int y)
        {
            var value = random.Next(0, 5);

            if (value == 0)
            {
                throw new InvalidOperationException("Transient error!");
            }

            return x + y;
        }

        public int Subtract(int x, int y)
        {
            var value = random.Next(0, 5);

            if (value == 0)
            {
                throw new InvalidOperationException("Transient error!");
            }

            return x - y;
        }
    }
}

Note that the Add and Subtract methods have a 1 in 5 chance of throwing an exception when they’re called. This is to simulate transient errors. Note also that I’m using the seeding technique for random numbers that I introduced in my last post.

I’ve coded the Program class as follows:

using System;

namespace RetryLoops
{
    public static class Program
    {
        public static void Main()
        {
            var calculator = new Calculator();

            Console.WriteLine(calculator.Add(1, 2));
            Console.WriteLine(calculator.Subtract(1, 2));
            Console.WriteLine(calculator.Add(1, 2));
            Console.WriteLine(calculator.Subtract(1, 2));
            Console.WriteLine(calculator.Add(1, 2));
            Console.WriteLine(calculator.Subtract(1, 2));
            Console.WriteLine(calculator.Add(1, 2));
            Console.WriteLine(calculator.Subtract(1, 2));
            Console.WriteLine(calculator.Add(1, 2));
            Console.WriteLine(calculator.Subtract(1, 2));
            Console.WriteLine(calculator.Add(1, 2));
            Console.WriteLine(calculator.Subtract(1, 2));

            Console.ReadLine();
        }
    }
}

You don’t need to run this very many times before you get an unhandled exception.

Adding a method called ExecuteOperationWithRetry to the program and changing the main method to use this gives:

using System;
using System.Threading;

namespace RetryLoops
{
    public static class Program
    {
        public static void Main()
        {
            var calculator = new Calculator();

            Console.WriteLine(ExecuteOperationWithRetry(() => calculator.Add(1, 2)));
            Console.WriteLine(ExecuteOperationWithRetry(() => calculator.Subtract(1, 2)));
            Console.WriteLine(ExecuteOperationWithRetry(() => calculator.Add(1, 2)));
            Console.WriteLine(ExecuteOperationWithRetry(() => calculator.Subtract(1, 2)));
            Console.WriteLine(ExecuteOperationWithRetry(() => calculator.Add(1, 2)));
            Console.WriteLine(ExecuteOperationWithRetry(() => calculator.Subtract(1, 2)));
            Console.WriteLine(ExecuteOperationWithRetry(() => calculator.Add(1, 2)));
            Console.WriteLine(ExecuteOperationWithRetry(() => calculator.Subtract(1, 2)));
            Console.WriteLine(ExecuteOperationWithRetry(() => calculator.Add(1, 2)));
            Console.WriteLine(ExecuteOperationWithRetry(() => calculator.Subtract(1, 2)));
            Console.WriteLine(ExecuteOperationWithRetry(() => calculator.Add(1, 2)));
            Console.WriteLine(ExecuteOperationWithRetry(() => calculator.Subtract(1, 2)));

            Console.ReadLine();
        }

        private static TReturn ExecuteOperationWithRetry<TReturn>(Func<TReturn> operation)
        {
            const int MaximumAttempts = 10;
            var timeBetweenRetries = TimeSpan.FromMilliseconds(100);

            var attempts = 0;
            
            while (true)
            {
                try
                {
                    attempts++;
                    return operation.Invoke();
                }
                catch (Exception exception)
                {
                    // If the exception isn't transient then re-throw:
                    if (!(exception is InvalidOperationException))
                    {
                        throw;
                    }

                    // If we've had all our attempts already then re-throw:
                    if (attempts >= MaximumAttempts)
                    {
                        throw;
                    }

                    // Wait before making another attempt:
                    Console.WriteLine("Transient error encountered, retrying...");
                    Thread.Sleep((int)(timeBetweenRetries.TotalMilliseconds * Math.Pow(2, attempts - 1)));
                }
            }
        }
    }
}

As the operation parameter of ExecuteOperationWithRetry is of type Func we can execute and retry any function we like without caring about the specific implementation of the function. In your own implementation you’ll probably want to get MaximumAttempts and timeBetweenRetries from a configuration file rather than hard-coding them, and you’ll probably also want to alter the exceptions that you retry on to suit the type of end-point. The Console.WriteLine statement in ExecuteOperationWithRetry is included only for demonstration purposes, and note that we are using the Math.Pow function to ensure that we wait a little longer each time we hit a transient error. This means that on the first attempt we wait for 100 milliseconds, then 200, 400, 800, 1600, etc… Once we’ve had 10 attempts we assume that the end-point is permanently broken and allow the exception to bubble up where (hopefully!) it’ll be caught by the calling method and handled appropriately.

When I ran this I got the following:

Retries 01

…which proves the retry loop is doing its job.

To make the code even more generic we can add the following overload of the ExecuteOperationWithRetry method, enabling us to handle operations that don’t have return values:

private static void ExecuteOperationWithRetry(Action operation)
{
    Func<bool> operationWithReturn = () => 
    {
        operation.Invoke();
        return true;
    };

    ExecuteOperationWithRetry(operationWithReturn);
}

There is a lot of scope to improve this technique, and to tailor it to specific applications. In fact, it is possible to make a totally generic retry loop component that has the exceptions to retry, time to pause and number of attempts to make passed in as constructor arguments. I’ll cover this in a future article, but for now, this concludes my exploration of retry loops.