Help! My .NET application refuses to open more than 2 concurrent internet connections!

A while ago some colleagues and I were writing a .NET application that fired off web-requests to a third party web-API. The details of the application are unimportant, other than that it was a migration tool which sent data from one third party store to another. There was a lot of data to shift (~9TB) and it needed to be done very quickly. So, we ended up writing a multi-threaded application which could read in many pieces of the stored data concurrently, and fire them off to the third party web-API concurrently.

We wrote a beautiful, elegant and totally thread-safe application and all watched in anticipation as we fired it off for the first time, expecting to see data throughputs the likes of which had never been seen before.

Instead the performance was only slightly better than the first version of our application, which just sent data through one piece at a time in a single thread. After a lot of digging, it turned out that all .NET applications have a restriction on the number of network connections that can be opened. By default this number is two. Fortunately, this number can be changed using the configuration file.

Simply add the following to override this value:

<system.net>
    <connectionManagement>
        <add address = "*" maxconnection = "100" />
    </connectionManagement>
</system.net>

To demonstrate this in action, try running the following .NET console application without overriding the configuration file:

using System;
using System.Net;
using System.Threading;

namespace ConnectionTestApplication
{
    public static class Program
    {
        private const int ThreadCount = 100;
        private static volatile int _timesPageGot;
        
        public static void Main()
        {
            var timerThread = new Thread(TimerThreadMethod);

            var getPageThreads = new Thread[ThreadCount];
            for (var i = 0; i < ThreadCount; i++)
            {
                getPageThreads[i] = new Thread(GetPageMethod);
            }

            timerThread.Start();
            for (var i = 0; i < ThreadCount; i++)
            {
                getPageThreads[i].Start();
            }

            Console.ReadLine();
        }

        private static void TimerThreadMethod()
        {
            var seconds = 0;

            while (true)
            {
                Thread.Sleep(1000);
                seconds++;
                Console.WriteLine("Seconds: {0}", seconds);
            }
        }

        private static void GetPageMethod()
        {
            var request = WebRequest.Create("https://en.wikipedia.org/wiki/List_of_law_clerks_of_the_Supreme_Court_of_the_United_States");
            using (var response = request.GetResponse())
            {
                _timesPageGot++;
                Console.WriteLine("Page got: {0}", _timesPageGot);
            }
        }
    }
}

Note that this will download what is supposed to be the longest page on Wikipedia (at the time of writing) a whole bunch of times concurrently using threads:

List of law clerks of the Supreme Court of the United States

You’ll get something like this on the screen depending on the speed of your internet connection:

Concurrent Internet Connections Before

Note that it took under 12 seconds to download the page 100 times on my internet connection.

However, with the configuration file override set to allow 100 concurrent connections I get:

Concurrent Internet Connections After

This time it took under 3 seconds.

Result!

Preventing embedded YouTube videos from showing related videos after playing

I was recently made aware of a cool feature in YouTube that prevents the embedded player from automatically playing/showing related videos once it has finished playing. With this piece of advice you need never give competitors chance to get their videos shown on your page again, and can avoid those inappropriate thumbnails of scantily clad girls from appearing on your oh-so-serious business website.

By default, YouTube will give you a piece of HTML like the following to embed in your site:

<iframe width="420" height="315" src="https://www.youtube.com/embed/Msp0ezv764g" frameborder="0" allowfullscreen></iframe>

Simply add “rel=0” to the query string part of the src attribute and banish those nasty related videos forever. The altered version should look like this:

<iframe width="420" height="315" src="https://www.youtube.com/embed/Msp0ezv764g?rel=0" frameborder="0" allowfullscreen></iframe>

YouTube supports a whole host of other query string parameters that you might find useful. A full list is available on the following site on the YouTube Embedded Players and Player Parameters site.

Configure the .NET SmtpClient object using the configuration file

If you’re using the SmtpClient class in the System.Net.Mail namespace of the .NET Framework, there’s an easy way to set the properties of the client without having to wire them up in the code manually.

Simply add the following section to your application configuration file and the class will auto-magically wire up the properties at run-time:

<system.net>
  <mailSettings>
    <smtp deliveryMethod="Network">
      <network enableSsl="true" port="123" host="myhost" defaultCredentials="false" userName="myuser" password="mypassword"/>
    </smtp>
  </mailSettings>
</system.net>

There are a whole bunch more settings you can edit. See Microsoft’s <mailSettings> Element documentation page for more details.

Are your SSL certificates secure?

SSL encryption is vital when sending sensitive information over the internet from browser to server, but just how secure are your sites? I used to naively believe that all I had to do was whack a certificate on the web-server and all would be well. However, a friend of mine recently had one of their web-sites audited (I won’t name the site for obvious reasons!) and found more holes in their security defenses than a block of Swiss cheese.

As well as the certificate, here are some other things you need to take care of:

  • Protocols (SSL, TLS, PCT and their various version numbers)
  • Ciphers (DES, AES, etc…)
  • Hashes (SHA x, MD5, etc…)
  • Which key exchanges are enabled
  • The order that SSL ciphers are used

Fortunately, help is at hand using these two great tools:

You can use the Qualys SSL Labs SSL Server Test to find out how secure your site is. Just enter your URL in the box and they’ll give you a full report on how secure the SSL for your site is.

If there are issues the IIS Crypto tool from Nartac Software will help you make the right remedial registry changes without having to tinker around in their yourself, thus vastly reducing the possibility you’ll trash your server.

Clever stuff!

Testing your responsive mobile web-sites

Although there is NO substitute for testing your responsive mobile designs on physical devices, performing this kind of testing is often time consuming as it requires us to push code out to web-servers so we can get to it from our devices. Then of course we spot a bunch of issues, fix them, re-release and test again. This process often repeats a few times before we get it right.

It is possible to reduce the number of iterations by doing more “device like testing” in the browser. There are two aspects to control in order to get your browser to simulate a mobile browser more closely. The first is the user agent, and the second is the screen size.

There are many tools out there to help us alter the user agent. As I mentioned in my blog post entitled Essential FireFox add-ons for web-developers I tend to use the FireFox User Agent Switcher.

That just leaves the screen size. FireFox has a great tool called the Responsive Design View which allows you to set a custom screen size after pressing CTRL+SHIFT+M.

Once you’ve faked the user agent and reduced your screen size, you should be able to find more issues with your responsive designs in your own browser without the hassle of continually pushing code out to your web-server.

Here is a screen shot of FireFox pretending to be an iPhone 3:

Mobile Testing

Text truncation at a word boundary using C#

Following on from my last post on stripping HTML from text using C#, once I had removed all signs of HTML from the incoming text, I was also required to show a short preview of the text. I originally went with a truncation method, as follows:

namespace ExtensionMethods
{
    public static class StringExtensionMethods
    {
        public static string Truncate(this string text, int maximumLength)
        {
            if (string.IsNullOrEmpty(text))
            {
                return text;
            }

            return text.Length <= maximumLength ? text : text.Substring(0, maximumLength);
        }
    }
}

This works, but the results look a little odd if the truncate happens half-way through a word.

Instead, I came up this method to truncate at the first word break within the allowed number of characters:

using System.Linq;

namespace ExtensionMethods
{
    public static class StringExtensionMethods
    {
        private static readonly char[] Punctuation = {'.', ',', ';', ':'};

        public static string TruncateAtWordBoundary(this string text, int maximumLength)
        {
            if (string.IsNullOrEmpty(text))
            {
                return text;
            }

            if (text.Length <= maximumLength)
            {
                return text;
            }

            // If the character after the cut off is white space or punctuation 
            // then return what we've got using substring:
            var isCutOffWhiteSpaceOrPunctuation = char.IsWhiteSpace(text[maximumLength]) || Punctuation.Contains(text[maximumLength]);
            text = text.Substring(0, maximumLength);

            if (isCutOffWhiteSpaceOrPunctuation)
            {
                return text;
            }

            // Find the last white-space or punctuation and chop off there:
            var lastWhiteSpaceOrPunctuationPosition = 0;
            for (var i = text.Length - 1; i >= 0; i--)
            {
                if (char.IsWhiteSpace(text[i]) || Punctuation.Contains(text[i]))
                {
                    lastWhiteSpaceOrPunctuationPosition = i;
                    break;
                }
            }

            text = text.Substring(0, lastWhiteSpaceOrPunctuationPosition).Trim();

            return text;
        }
    }
}

While not perfect, this approach works a lot better. Please feel free to suggest improvements.

Beware! ASP.NET MVC ActionFilterAttributes are cached between requests!

My colleagues and I recently encountered a very strange bug in an ASP.NET MVC web-site which seemed to be to do with NHibernate. The error suggested that we were trying to use a session that had already been closed. We could not find any sign of a closed session anywhere within the controllers in our project. Eventually we pinned the error down to a repository (containing an NHibernate session) referenced in an ActionFilterAttribute. The guy that wrote the code assumed that the repository would be referenced freshly by the attribute each time it was used, but of course it wasn’t; the .NET engine was helpfully caching the attribute between calls so that some users were getting sessions intended for other users.

The moral of the story is to make sure your ActionFilterAttributes are always stateless.

The following simple demonstration highlights this. Here is an ActionFilterAttribute which stores the date and time in a field on construction:

using System;
using System.Web.Mvc;

namespace WebTest.Attributes
{
    public class TestActionFilterAttribute : ActionFilterAttribute
    {
        private readonly DateTime _now;

        public TestActionFilterAttribute()
        {
            _now = DateTime.Now;
        }
        public override void OnActionExecuting(ActionExecutingContext filterContext)
        {
            // Put the date into a ViewBag property:
            filterContext.Controller.ViewBag.DateTime = _now;
        }
    }
}

This is then used in a controller as follows:

using System;
using System.Text;
using System.Web.Mvc;
using WebTest.Attributes;

namespace WebTest.Controllers
{
    public class HomeController : Controller
    {
        [TestActionFilter]
        public ActionResult Index()
        {
            const string dateTimeFormat = "yyyy-MM-dd HH:mm:ss.fffffff";

            var text = new StringBuilder();

            text.AppendFormat("DateTime called directly: {0}<br/>", DateTime.Now.ToString(dateTimeFormat));
            text.AppendFormat("DateTime taken from ActionFilterAttribute: {0}<br/>", ViewBag.DateTime.ToString(dateTimeFormat));

            return Content(text.ToString());
        }
    }
}

If you refresh the page a few times you’ll notice that the time coming from the ViewBag (which originates from the attribute) remains the same while the current time continues to increase, proving that the attribute isn’t being instantiated for each web call. You have been warned!

Stripping HTML from text using C#

I recently had a situation where I needed to show some text received in HTML format as plain text. This is the method I now use for this purpose, implemented as an extension method:

using System.Linq;
using System.Text.RegularExpressions;

namespace ExtensionMethods
{
    public static class StringExtensionMethods
    {
        public static string StripHtml(this string text)
        {
            if (string.IsNullOrEmpty(text))
            {
                return text;
            }

            var tagRegex = new Regex(@"(?></?\w+)(?>(?:[^>'""]+|'[^']*'|""[^""]*"")*)>");
            var tagMatches = tagRegex.Matches(text);

            var commentRegex = new Regex(@"\<![ \r\n\t]*(--([^\-]|[\r\n]|-[^\-])*--[ \r\n\t]*)\>");
            var commentMatches = commentRegex.Matches(text);

            // Replace each tag match with an empty space:
            text = tagMatches.Cast<object>().Aggregate(text, (current, match) => current.Replace(match.ToString(), " "));

            // Replace each comment with an empty string:
            text = commentMatches.Cast<object>()
                .Aggregate(text, (current, match) => current.Replace(match.ToString(), string.Empty));

            // We also need to replace &nbsp; as this can mess up the system:
            text = text.Replace("&nbsp;", " ");

            // Trim and remove all double spaces:
            text = text.Trim().RemoveDoubleSpaces();

            return text;
        }

        public static string RemoveDoubleSpaces(this string text)
        {
            if (string.IsNullOrEmpty(text))
            {
                return text;
            }

            // Condense all double spaces to a single space:
            while (text.Contains("  "))
            {
                text = text.Replace("  ", " ");
            }

            return text;
        }
    }
}

The method RemoveDoubleSpaces was also needed, since after replacing HTML elements with empty space it is possible to end up with multiple empty spaces where a single space would do. This is quite a useful method in its own right, hence separating it out.

If you find any inputs which trip this method up, please let me know.

Using Visual Studio to run web-sites locally using real domains

Visual Studio is great for developing and testing web-sites, but by default it runs them under a localhost address using a randomly generated port number, as given in the property page for the web-site:

Visual Studio Real Domains 01

This is fine, but sometimes it’s useful to have sites run at a location that looks more like a real internet URL, especially if you plan on including functionality that manipulates the URL for redirects etc… I’ve also encountered issues creating cookies using the default http://localhost:xxxx address.

Fortunately, there’s a hosts file trick you can use to make Visual Studio run sites under more realistic URLs. First point 127.0.0.1 at the URL you want to use in your hosts file, for example:

127.0.0.1 www.testing.com

Note though that if the URL you are using points to a real web-site on the internet, you won’t be able to access that site again until you remove the entry from your host file.

Then edit the property pages so that you’re using the Visual Studio Development Server, a specific port (rather than an auto-assigned port) and a custom start URL containing your chosen port number and test URL, for example:

http://www.testing.com:1455

Your property pages should look like this:

Visual Studio Real Domains 02

Then, when you run your web-site using Visual Studio, it will use http://www.testing.com:1455 as the base URL. All paths relative to that should work as expected:

Visual Studio Real Domains 03

Solved: “Value does not fall within the expected range” error when opening app.config or web.config in Visual Studio 2012

I recently started getting the follow error in Visual Studio 2012 when I tried to open any configuration file:

Error: Value does not fall within the expected range

It took a bit of working out, but the solution for me was as follows:

  • Close Visual Studio
  • Browse to the top level folder where the solution is located in Windows Explorer
  • Search for and delete files matching the following two patterns: *.user, *.suo
  • Restart Visual Studio

That should do the trick.