Thursday, May 21, 2015

Create a Reverse Proxy Controlled By Sitecore

Reverse proxies can be an incredibly useful technology in your Sitecore implementation depending upon your needs. The basic idea is that a reverse proxy forwards requests on to other servers on behalf of the requesting client, sort of like a traffic cop. The responses from the servers behind the reverse proxy are then returned to the requesting client. This can be done in such a way that is completely transparent to the end-user.

The Use Case

So why bother? Well, as Grant Killian suggests over on his blog at least two scenarios come to mind (I'm sure all you very smart folks could undoubtedly name more!) I want to focus on the case of a reverse proxy sitting between the Internet and a set of web servers that includes one or more legacy web servers and a Sitecore instance.

I've kept the conceptual diagram above simple (no load-balanced servers, firewalls, cache servers, etc.) but the technique readily applies to an enterprise ecosystem. The basic strategy is as follows:

  1. A user tries to browse a page (perhaps one they have bookmarked) e.g. http://company.com/foobar.php
  2. The reverse proxy receives the request and "asks" Sitecore where to route the request
  3. Sitecore tells the reverse proxy if it can handle the request and, if so, what the URL should be.
  4. The reverse proxy rewrites the request and forwards it. For example, if Sitecore responded positively to the reverse proxy our URL might be transformed to http://sitecorecd.company.com/foo/bar
  5. Sitecore or the legacy server responds to the page request
  6. The reverse proxy rewrites the response so that the end-user is unaware the page they see was came from a different server than the one they contacted.

The payoff with this scenario is we can now manage incremental content migrations from legacy servers to Sitecore servers without any disruption to end-user experience. Bookmarks, campaign emails, RSS feeds, Google search result rankings....all of it will happily continue on as always regardless of whether the legacy web server or Sitecore actually answers the HTTP request. Powerful stuff! This technique is especially useful for clients that have a very large inventory of content and cannot or are unwilling to migrate everything all at once.

The Solution

The first order of business is setting up a reverse proxy in IIS. The goal is to have a dedicated web site in IIS as the reverse proxy. To do that we need to install the Application Request Routing (ARR) extension. Once ARR is installed we'll need to do perform the following configuration steps

  1. Open IIS Manager and select the server node. Double click on the Application Request Routing Cache icon.

  2. In the right-hand pane click Server Proxy Settings.

  3. Check the Enable proxy setting and uncheck the Reverse rewrite host in response headers option.

  4. Set the Response buffer and Response buffer threshold values for 8092 and then click Apply. The reason for this I discovered through the school of hard knocks: some pages were mysteriously causing YSODs. After digging through logs (more on that later) we found that the response from the server was literally truncated. The page was large enough that it was overflowing the response buffer and causing it to flush with only a part of the overall page.


Now that ARR is installed and configured at the server level we need to turn our attention to the reverse proxy site. Here is the secret sauce of our solution: rather than merely write in some rules for routing in the web.config we are going to create our own custom Rewrite Provider. This will allow us to execute our own code during the runtime of the reverse proxy. I followed this guide to develop my own custom provider; it should get you up and running.

So what does my rewrite provider do? At its heart it's just a very simple URL resolver. The provider is rather dumb (and it should be!) We want the reverse proxy to do as little processing as possible.

  1. First make a request to Sitecore to see if the requested page (rewritten with Sitecore's host header) can be served, i.e. does the web request return with a response status code < 400.
  2. If that fails, the reverse proxy contacts a web service that knows how to map a legacy URL onto a Sitecore URL. Thus a URL like /news/article.php?id=foobar in the legacy system can be mapped onto /news/articles/foobar for example.
  3. If steps 1 and 2 fail, then the request is routed to the legacy server.

You may be wondering if all of that still sounds like too much work given that every request passes through the reverse proxy. Fortunately ARR has very good built-in caching, so in practice, your most requested pages (and resources) will not be processed in code continuously.

Closing Thoughts

Beyond what's already been covered, I recommend you consider the following:

  1. You need some kind of logging strategy. I write to the event logs from the reverse proxy and Sitecore's logs during the runtime of Sitecore (for example, the URL mapping web service.)
  2. Perform load testing. ARR is remarkably good OOTB with its cache settings, but better to test and know than simply assume.
  3. Think ahead about sessions and how you will deal with them.
  4. Redundancy. Our solution uses more than one reverse proxy. As a side-note with my implementation, if Sitecore itself goes down, the reverse proxy will continue to function. All requests would simply go to the legacy server. Eventually this behavior may become undesirable, but early in a project's lifetime this can be a real selling point.
  5. You definitely need to create some outbound rewrite rules in your reverse proxy's web.config to deal with:
    1. Rewriting relative links in the response HTML
    2. Rewriting the Location in the response Header when the status code is a 3XX (a redirect) and the host name is your backend server. This will prevent the end-user's browser being redirected to http://legacy.company.com/foobar rather than http://company.com/foobar
  6. You should absolutely turn on Failed Request Tracing Rules. This is the logging function I discussed earlier that proved invaluable in diagnosing and resolving issues during development

Monday, April 27, 2015

SwitchMasterToWeb Woes

Scenario: You are using Sitecore 8—I think this also applies to 7.5—and have enabled the SwitchMasterToWeb.config. You now see see an exception related to 'master' database. For example:

  • Could not find configuration node: contentSearch/indexConfigurations/indexUpdateStrategies/syncMaster



Cause: Sitecore patches files found in the \Include folder first and then recursively patches all files found in sub-folders of \Include. In all cases, Sitecore follows alphabetical order when deciding which folder or file to examine next. This means that the SwitchMasterToWeb.config file placed in \Include will be merged before config files found in sub-folders. Some of those config files are for indexes and those index try to use the syncMaster strategy.

Resolution: Place the SwitchMasterToWeb.config file in a sub-folder like "zzzMustBeLast" and reload your site!

Saturday, April 25, 2015

NuGet Tip: Automatically Set 'Copy Local' Property to False

This is probably slightly off the beaten path for some of the Sitecore community, but I suspect quite a few of us use NuGet. Furthermore, of those that do, some not only consume but produce NuGet packages. One annoyance I've had is the good ol' Copy Local property of an assembly reference.


When this property is set to true, the assembly will be copied to the output directory of your project. This can be problematic. For example, with TDS, when you build your solution the output directory of your web application project gets copied to your Sitecore instance. Ideally, you don't want to overwrite or add DLLs to your Sitecore instance by accident. While you can prevent this from happening by maintaining an exclusion list in TDS, it's easy to forget.



How does this relate to NuGet packages?

Usually when you add a new package one or more DLL references are added to your project(s) and almost invariably the Copy Local property will be set to true. If you create your own NuGet package and wish to prevent this, here is a PowerShell script that will do the trick:


Simply add/overwrite the Install.ps1 script found in the "tools" folder of the NuGet package. Voila!

Friday, April 3, 2015

How to Map Sitecore Rules Field with Glass and TDS

tl;dr Copy lines 246 and 247 from here. Regenerate your Glass classes. 

Today a colleague asked how to map the rules field from Sitecore with Glass Mapper using TDS and T4 templates. It just so happens that I'd recently worked on this problem, and I thought I would share it with others here.

The basic issue is that the glassv3item.tt template doesn't know how to deal with the Rules field. The GetGlassFieldByType method is responsible for assigning a type to mapped field. It does this with a switch statement. Our rules field is falling all the way through to the default case which maps the field to an object. We need to add a case for the field.Type value when it equals "rules".

What type will we map to though? On first pass I thought something like XDocument would make a lot of sense. Problem is that doesn't work. The value is always null. I took a look inside the Glass.Mapper.Sc.dll at what I believe is the code responsible for returning a value. It looks to me like the mapping code isn't fully implemented, and Glass relies on a generic method that simply returns a string value.


Not a big deal. We can work with this.

After you've modified the T4 template and regenerated your Glass classes, you should have a mapped property on your class of type string. I've found for my purposes this is perfect for my needs, but that didn't stop me from extending my partial class with an XDocument property...just in case.

public partial class GeneratedClass
{
    public XDocument RuleAsXDocument
    {
        get { return XDocument.Parse(this.Rule); }
    }
}

Thursday, April 2, 2015

A SIMple Error

While installing Sitecore Instance Manager (SIM) I made a silly mistake that gave me pause. I figured I would document it here in the hope it might help someone else might benefit.

The final step in the installation wizard attempts to do a permission check. It is labeled as "File Systems permission" and, indeed, it does check this and even provides a handy "Grant" button if SIM does not have the permission it thinks it needs.


What can be a bit confusing is that even after seeing the success message above you still encounter an error dialog complaining that "You probably don't have necessary permissions set. Please try to click 'Grant' button before you proceed."


What's happening? Under the hood, SIM isn't just checking the file system, it is also trying to create a test database in SQL. If the SQL login SIM uses doesn't have the right to create a database, then SIM will fail this "file system" check with (in this specific case) a misleading error. The fix is simple, make sure your SQL login has the dbcreator role or higher. Thus my silly mistake: Of course SIM needs the ability to create databases why didn't I think of that sooner....DUH! :)


Rerun the last step and enjoy the wonders of SIM!

Saturday, March 28, 2015

On Second(ary) Thought...

Recently I posted my thoughts regarding the proper ratio of Solr cores to Sitecore indexes. In it, I mentioned the need to double the number of cores to support the SwitchOnRebuildSolrSearchIndex feature. It turns out, this isn't quite right. Creating a secondary core for the analytics index does no good and should be avoided.

You'll want to avoid a secondary core for the analytics index because trying to use one results in exception. The analytics index has an extra "group" parameter, and Sitecore cannot find a matching constructor. The SolrSearchIndex class allows for the group parameter but the SwitchOnRebuildSolrSearchIndex does not.

Why is this? As Adam Conn explains, the types of crawlers responsible for maintaining the analytics index are observers of data. This means that the crawler is notified when data is available and then passed that data. Therefore, rebuilding the analytics index would result in an empty index unless you rebuild the reporting database so that the analytics crawlers may observe (and index) the data. This is the reason why the UI doesn't provide the option of rebuilding the analytics index.

Tuesday, March 24, 2015

Solr + Glass = Castle Windsor Crashers?

(I know the title isn't technically true...don't judge me!)

I think it's safe to say most of us Sitecore developers are using some form of ORM these days, right? ...Right? :)

Of course you are, and – most likely – that means you are using Glass Mapper. Probably, you are also using Castle Windsor as your choice of Inversion of Control. You could use some other IoC, but there's a NuGet package that makes incorporating Castle Windsor + Glass into your Visual Studio solution quite easy.

So what if you read my series on Solr and were inspired to use it for Sitecore? Solr also requires an IoC. As I mentioned here, you may choose from Castle Windsor, AutoFac, Ninject, StructureMap, and Unity. It seems only natural to stick with the same IoC as Glass. Common sense says don't add more moving parts than necessary.

And this is why common sense isn't always common or sensible. The moment you deploy your Glass dependent (and thus Castle Windsor dependent) code to your Sitecore instance, you are going to be treated to some ugly YSODs. The issue is that Glass wants to use a higher version of Castle Windsor (3.2) than your Solr-enabled Sitecore instance; it wants version 3.1.

Luckily the fix is relatively painless. We are going to instruct .NET to redirect bindings of earlier versions of Castle.Core and Castle.Windsor to our later version. All you need to do is add the following section to your web.config: