tag:blogger.com,1999:blog-36692096298180997572024-03-14T03:52:43.404-04:00Sitecore /binPatrick's memory hole for Sitecore.Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.comBlogger21125tag:blogger.com,1999:blog-3669209629818099757.post-62389565100814406812017-12-15T08:49:00.000-05:002017-12-18T17:48:00.609-05:00An Approach to Blue/Green Sitecore Deployments<div class="separator" style="clear: both; display: none; text-align: center;">
<img border="0" data-original-height="1001" data-original-width="1000" height="320" src="https://4.bp.blogspot.com/-nALcghXib5A/WjLlTRGf-7I/AAAAAAAAA_A/zKOzPZnbkdIRRaTjboTkTRZDnwfJHD5-ACLcBGAs/s320/sitecore-blue-green.png" width="319" />
</div>
You may have heard someone speak of <a href="https://martinfowler.com/bliki/BlueGreenDeployment.html" target="_blank">blue-green deployments</a>, and you may have found yourself wondering what those colors have to do with software development. Simply put, blue-green deployments reduces downtime and risk when deploying changes. The central idea is to have two identical production environments labeled Blue and Green. At any given time one of the environments is live and the other is not. New code is always pushed to the non-live environment where it can be tested. Once validated, web traffic is pointed to the freshly updated environment. The other environment no longer serves traffic and can serve as a rollback in the unlikely event an issue surfaces in the newly live environment.<br />
<br />
Obviously, zero-downtime deployments are very valuable to enterprise clients. Since blue-green deployments minimize downtime (perhaps even to zero) it behooves us as Sitecore architects to utilize the strategy. Of course, if you have any familiarity with the complexity of a Sitecore environment, blue-green deployments may seem like an unattainable goal.<br />
<br />
My motivation for this blog post is to sketch out an approach for blue-green deployments with Sitecore. I'd like to think through the problem and demonstrate it is possible insofar as we can trust a thought-experiment.<br />
<br />
<h3>
The Challenge</h3>
The primary problem posed by Sitecore with blue-green deployments is the database layer. Since the database is a shared resource amongst all Sitecore servers in an environment, any change there can affect the entire environment. Additionally, since we are dealing with a CMS, we must expect that authors regularly introduce changes to the database.<br />
<br />
This database challenge is exacerbated by two factors:<br />
<ol>
<li>We cannot control the schema. Sitecore must own that.</li>
<li>We must actually think about two database layers (SQL and Mongo) -- in which some databases are inter-related -- as well as their attendant search indexes</li>
</ol>
<div>
<br /></div>
Point #1 should be self-evident. Sitecore's API encapsulates the database layer. Any changes to the database must be done through the API; this is de rigueur for any CMS and Sitecore is no exception.<br />
<br />
Point #2 probably requires a little more explanation. The databases in Mongo function as a very large "net" that captures all interaction data with visitors to the site. Mongo is organized in such a way to make writes very fast. The Reporting database in SQL represents data from Mongo that has been reorganized to support efficient reads so that report performance is optimized. The Analytics index lets us query Mongo data from the API efficiently. Thus, Mongo is the source of truth for visitor interaction data and the Reporting database and the Analytics index are coupled to it. This means any changes introduced to the data in Mongo must also be represented in SQL and in the Analytics index. We must treat those three systems as a unit.<br />
<br />
<br />
<h3>
A Solution</h3>
Clearly, we must mitigate the problems posed by the database layers. I believe we can, but let's first pose a few assumptions:<br />
<ol>
<li>Content Authors will be inactive during deployments.</li>
<li>We cannot use InProc mode for session state on CD servers.</li>
<li>Sticky sessions (server affinity) should be disabled.</li>
<li>Load-balancer supports configuration changes through scripting</li>
</ol>
<br />
<br />
<div style="text-align: center;">
<h4>
Step 0: Initial State</h4>
</div>
Now to the fun part! Before the deployment begins we imagine blue and green PROD environments where the green environment is live and one version ahead of the blue environment. For this initial state, I've greatly simplified the server topology. I'm representing all of Sitecore servers with a single app server and all databases as a single database. I've also eliminated search indexes.
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-LkvJLwxT6oU/Wihk5p5DnjI/AAAAAAAAA7Q/ZO9uYVU9L-EEKVck_dwxdN5J_gRq_sekACLcBGAs/s1600/BG0.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="176" data-original-width="583" height="192" src="https://3.bp.blogspot.com/-LkvJLwxT6oU/Wihk5p5DnjI/AAAAAAAAA7Q/ZO9uYVU9L-EEKVck_dwxdN5J_gRq_sekACLcBGAs/s640/BG0.png" width="640" /></a></div>
<br />
<br />
<div style="text-align: center;">
<h4>
Step 1: Synchronize Content</h4>
</div>
The first step in the deployment process is to synchronize all data managed by content authors in Sitecore from the live (green) environment to the blue environment. Typical examples of this sort of data would be the site pages and data items which we would expect to be descendants of /sitecore/content and media assets in the Media Library. One possible mechanism for automating the synchronization process is to use Razl. Here's a nice <a href="https://www.youtube.com/watch?v=hmoYpGb0zQ4" target="_blank">YouTube video</a> demonstrating the capability. It's important we synchronize content prior to deploying new code so that our deployment is working against the same Sitecore data as it would in a 'conventional' deployment to PROD. Note in the diagram below that SQL data in the blue environment is still version 0. The asterisk represents the addition of managed content synchronized from the green environment.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-fdanwurjK5Q/Wi6pPdDYwMI/AAAAAAAAA8k/otHEWMGarPcJOvSUbltBXhvANiMnKWLNQCLcBGAs/s1600/BG1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="259" data-original-width="587" height="281" src="https://4.bp.blogspot.com/-fdanwurjK5Q/Wi6pPdDYwMI/AAAAAAAAA8k/otHEWMGarPcJOvSUbltBXhvANiMnKWLNQCLcBGAs/s640/BG1.png" width="640" /></a></div>
<br />
<br />
<div style="text-align: center;">
<h4>
Step 2: Deploy New Version</h4>
</div>
The next step is to deploy our changes to the blue environment. I've represented this work performed by Octopus Deploy. It is my preferred tool for this job, but it's not the only one. Octopus manages making changes to all Sitecore servers (files placed on the file system and Sitecore items written to SQL) as well as ensuring we publish and rebuild indexes as required. When this step completes the blue environment will be one version ahead of the green environment (while retaining the synchronized content from green.)<br />
<blockquote class="tr_bq">
<br /><b>Update:</b> There is nothing, I believe, about this strategy the <b>requires</b> indexes to be rebuilt. There <i>could be</i> something about your particular solution that needs an index to be rebuilt. If so, this is the correct step to perform that work.</blockquote>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-yTWKATum2dI/Wi7Q8BVRMPI/AAAAAAAAA84/yvqjLpoBSS4kv_T1iXK2G_KND94jru_lACLcBGAs/s1600/BG2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="368" data-original-width="583" height="401" src="https://4.bp.blogspot.com/-yTWKATum2dI/Wi7Q8BVRMPI/AAAAAAAAA84/yvqjLpoBSS4kv_T1iXK2G_KND94jru_lACLcBGAs/s640/BG2.png" width="640" /></a></div>
<br />
<br />
<div style="text-align: center;">
<h4>
Step 3: Testing</h4>
</div>
At this point, the blue environment is ready for testing. The specific forms of testing are wide-open: starting with someone doing basic smoke-testing all the way through a full suite of automated tests.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-e94scf53stk/Wi7RRmdI3EI/AAAAAAAAA9A/QDv0G6RqNxEwJxcDi0J86OibQzWUVdEJACPcBGAYYCw/s1600/BG3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="357" data-original-width="585" height="390" src="https://4.bp.blogspot.com/-e94scf53stk/Wi7RRmdI3EI/AAAAAAAAA9A/QDv0G6RqNxEwJxcDi0J86OibQzWUVdEJACPcBGAYYCw/s640/BG3.png" width="640" /></a></div>
<br />
<br />
<div style="text-align: center;">
<h4>
Step 4: Change Connection Strings and Analytics Index</h4>
</div>
Now comes the hard part: dealing with databases. Fear not, however, we have a plan. Recall from step 1 that we've already dealt with any differences between blue and green due to changes made by content authors (also, we assume a content freeze during deployment.) In step 2 we deployed the the latest solution as well as published and rebuilt indexes if required. Therefore, the last hurdle is analytics and session data.
<br />
<br />
First, let's take a moment to think through analytics data. It all starts with the four Mongo databases. From the Mongo databases we have the derived data in the SQL Reporting database and the Analytics search index. So, really we need to think of these three subsystems as a unit. Another important consideration is that we never want to mix the live and non-live analytics data. For example, we don't want to dirty visitor interaction data with clicks generated by smoke-tests performed during a deployment. We also need to be careful with session data. Live users will be transitioned from the green environment to the blue environment. All of their serialized session data must remain valid and coherent.<br />
<br />
Great...how do we do this?<br />
<br />
We change the connection strings for the Mongo, Reporting, and Session databases for all Sitecore servers in the blue environment. We also modify the analytics index config to use the 'analytics_live' Solr core which is replicated from a corresponding 'analytics_live' Solr core in the green environment. In this way, we guarantee that all analytics and session data for a blue Sitecore server corresponds to live data.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-ipHWQHEsPaQ/WjLTgM2SJoI/AAAAAAAAA-k/uYNJDHZFEGclsHHe_1_NCnteI4E5n196QCLcBGAs/s1600/BlueGreen-Deployment-Diagrams-step-4.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1080" data-original-width="1124" height="614" src="https://1.bp.blogspot.com/-ipHWQHEsPaQ/WjLTgM2SJoI/AAAAAAAAA-k/uYNJDHZFEGclsHHe_1_NCnteI4E5n196QCLcBGAs/s640/BlueGreen-Deployment-Diagrams-step-4.gif" width="640" /></a></div>
<br />
<br />
<div style="text-align: center;">
<h4>
Step 5: Make Blue the Live Environment</h4>
</div>
We are now ready to redirect incoming traffic from the Internet to the blue environment, thus making it the live environment. At the same time, we also need to reverse the direction of Solr replication so that the blue analytics_live index is now the master index and green's version is the slave. Since we assume there are no sticky sessions. The traffic switch should be nearly instantaneous.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-9keKrsY1wOY/WjLT0aYGJ4I/AAAAAAAAA-s/-hjrXmlc2qYdeaXseOkxNc2KjV9re43mwCLcBGAs/s1600/BlueGreen-Deployment-Diagrams-step-5.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1080" data-original-width="1124" height="614" src="https://4.bp.blogspot.com/-9keKrsY1wOY/WjLT0aYGJ4I/AAAAAAAAA-s/-hjrXmlc2qYdeaXseOkxNc2KjV9re43mwCLcBGAs/s640/BlueGreen-Deployment-Diagrams-step-5.gif" width="640" /></a></div>
<br />
<br />
<div style="text-align: center;">
<h4>
Step 6: Finish Retiring the Green Environment</h4>
</div>
Finally, we must finish retiring the green environment from being live. This step is really just the inverse of step 4. That is, we change the connection strings for the Mongo, Reporting, and Session databases for all Sitecore servers in the green environment. Again, we also modify the analytics index config. This time, however, we wire up the analytics index in Sitecore to use the 'analytics' Solr core rather than the 'analytics_live' Solr core.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-Cp1qPZQXBUI/WjLT58nFL7I/AAAAAAAAA-w/7PR2k4EyVfc8S1_kOTRsmnO1OZq8yl4xACLcBGAs/s1600/BlueGreen-Deployment-Diagrams-step-6.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1080" data-original-width="1124" height="614" src="https://2.bp.blogspot.com/-Cp1qPZQXBUI/WjLT58nFL7I/AAAAAAAAA-w/7PR2k4EyVfc8S1_kOTRsmnO1OZq8yl4xACLcBGAs/s640/BlueGreen-Deployment-Diagrams-step-6.gif" width="640" /></a></div>
<br />
<br />
<br />
<h3>
Rollbacks</h3>
While I separated steps 4-6 are listed above as discrete steps to help illustrate the idea, as a practical matter we should think of them as a single continuous step. Even better, if we automate 4-6 as a single operation, the process of rolling back from blue to green becomes much easier.<br />
<br />
Imagine you finish step 6 and proudly watch traffic seamlessly flow to the blue servers only to discover that despite all the testing in step 3, an issue pops up related to the code just deployed. You could rollback to the green environment by inverting steps 4-6. That's a simple proposition if you took the time to automate 4-6 as a single operation.Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com1tag:blogger.com,1999:blog-3669209629818099757.post-47390227031413077542016-04-17T19:21:00.001-04:002016-04-17T19:21:04.737-04:00Sitecore Installer Re-visited<div class="separator" style="clear: both; display: none; text-align: center;">
<img border="0" src="https://4.bp.blogspot.com/-0CeLsi6wg9U/VZqedLC01ZI/AAAAAAAAAns/AHTm3VuTNdo/s1600/powershell.png" /></div>
Hard to believe, but it was close to a <a href="http://sitecoreblog.patrickperrone.com/2015/07/powershell-sitecore-install-script.html">year</a> ago that I released my <a href="https://github.com/patrickperrone/Sitecore-PowerShell-Installer" target="_blank">Sitecore PowerShell Installer script</a>. Since then I've continued to refine and improve upon it. Now seems like as good a time as any to take stock of the script.<br />
<br />
<h2>
Enhancements </h2>
<h4>
Installs 8.1 or 8.0</h4>
I've added a version checker in the script that uses Reflection to examine the version of the Sitecore.Kernel.dll assembly. This check then forces some behavioral differences during the script's runtime.<br />
<br />
<h4>
Robust Pre-Install Sanity Checking</h4>
Roughly 1/4 of the script is dedicated to validating configuration input and environmental health before any changes are made to the server. If I had a time machine, I would go back in time to the day I started the script and add a install counter. It's hard to pin down an accurate estimate of how many times I've used the script to install Sitecore, but I know I would need at least three digits! :)<br />
<br />
<h4>
More Sitecore Roles</h4>
Originally, the script was intended for CM/CD installs. Since then I have added more specialized roles like a Preview CD and a Publishing server.<br />
<br />
<h4>
Optionally Disable All Database Operations</h4>
Do you have to deal with intransigent DBAs? If, for some reason, you simply cannot allow the script to connect to SQL and write changes, then you can still use the script to automate everything else.<br />
<br />
<h4>
Full Compliance with MongoDB's Connection String Specification</h4>
If Mongo allows for it, then so does the script. You can review the specification <a href="https://docs.mongodb.org/manual/reference/connection-string/" target="_blank">here</a>.<br />
<br />
<h4>
Multiple Bindings in IIS</h4>
This is useful if you are running a multi-site instance or use load-balanced servers and wish to provide dedicated host names/IPs in addition to the load-balanced binding.<br />
<br />
<h4>
Change Default Admin Password</h4>
I worked very hard to ensure that servers were properly hardened during install. Yet, somehow, overlooked this basic but necessary change! Of course you could make this change manually, but if you wish to automate the installation of entire server environments this is a simple but important addition to the script. <br />
<br />
<h4>
Role-Based Config File Examples</h4>
The installer script's run-time is governed by a .config file that contains all the information needed to install Sitecore. To help people get started using the script, I've created a number of example configuration files according to Sitecore role.<br />
<br />
<h2>
Philosophy of Use</h2>
My general approach to managing and customizing Sitecore is through package deployment -- typically .update package. This, in turn, informs how I think about installing Sitecore. As a general rule of thumb, I want to make any system change that I cannot (easily) do through an .update package. I want to configure the server into a standard role that is suitable for smoke testing, but I don't want to turn an <i>installer</i> into an <i>application management </i>solution. This is same basic position I've maintained since I wrote the original version.<br />
<br />
What doe this mean in a practical sense? As much as possible, I try to avoid modifying stock .config files. There are a handful of exceptions, some of which I wouldn't have predicted back when I wrote my original blog post. Here are the list of files I do modify depending upon the specifics of the install:<br />
<br />
<ul>
<li>web.config</li>
<li>DataFolder.config</li>
<li>Sitecore.analytics.tracker.config</li>
<li>Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config.example</li>
<li>ScalabilitySettings.config</li>
<li>Sitecore.Publishing.Parallel.config</li>
</ul>
<br />
In addition, I will enable/disable any number of files depending upon the server role specified. For example, a CD server will have the SwitchMasterToWeb.config file enabled. Furthermore, I will place that file in an appropriate folder (e.g. zzzMustBeLast or Z.SwitchMasterToWeb).<br />
<br />
So what are some examples of what I <b>won't</b> do? My script will not 'turn on' Solr. Doing so depends upon having a Solr environment ready to go. I have provided a <a href="http://sitecoreblog.patrickperrone.com/2015/03/a-script-to-swap-search-in-sitecore.html">script</a> to switch between search providers however. I won't create new indexes or new indexing strategies if you provision a Preview CD server, nor will I ensure your indexing strategies are appropriately configured on a CD vs. a CM server. In my opinion, that is the job of some other process such as deploying an .update package that contains the appropriate .config file patches. The job of the install script is to ensure that each server is in a state ready to receive those packages and to eliminate the need to make any system changes outside the domain of an .update package.<br />
<br />
<h4>
Installer Configuration Tour</h4>
<div>
<iframe width="420" height="315" src="https://www.youtube.com/embed/xFp0cUWsLXA" frameborder="0" allowfullscreen></iframe>
</div>
<br />
<br />
<br />
<h2>
Future Enhancements</h2>
One idea I have is to isolate all sitecore config changes made by the installer to a single patch file. While the list of files that I make changes to today is fairly small, it would be an improvement if I made no modifications at all.<br />
<br />
How else could I improve the script? Actually, I'd like to ask you that, dear reader. What would you like to see done, if anything?Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com2tag:blogger.com,1999:blog-3669209629818099757.post-84302778412365264602015-08-10T08:48:00.000-04:002015-08-11T20:23:37.697-04:00Install Sitecore.Ship From NuGet with PowerShell<div class="separator" style="clear: both; display: none; text-align: center;">
<img border="0" height="320" src="http://3.bp.blogspot.com/-QiIXwqWKN3c/VcfT_FwG_yI/AAAAAAAAAoo/JHEX6hi4BbI/s320/SitecoreShip.png" width="320" /></div>
<a href="https://github.com/patrickperrone/Sitecore.Ship-PowerShell-Installer" target="_blank">Download</a> from GitHub. <br />
<br />
Now that I have a <a href="http://sitecoreblog.patrickperrone.com/2015/07/powershell-sitecore-install-script.html">solution for quickly installing Sitecore in higher environments</a>, I want a way to automate .update package installs. There are a number of clever solutions out there to assist with this. My choice is <a href="https://github.com/kevinobee/Sitecore.Ship" target="_blank">Sitecore.Ship</a>. The biggest advantage Sitecore.Ship has over other solutions I have seen is the ability to perform remote installations, including sending the package over the wire. Great stuff!<br />
<br />
The stumbling block for me, and possibly others, is that Sitecore.Ship isn't very easy to install. The typical technique is to use NuGet to introduce Sitecore.Ship into Visual Studio, and include it in your custom solution's deployment workstream. But...this is a chicken-and-egg problem for me. I want Sitecore.Ship installed from day one, because it is a building block of my deployment workstream <i>not</i> my custom solution!<br />
<br />
<h3>
PowerShell To The Rescue</h3>
Given my experience with PowerShell to install Sitecore my thought process was, "If PowerShell can install Sitecore, surely it can install Sitecore.Ship." (Yes, it surely can.) I've <a href="https://github.com/patrickperrone/Sitecore.Ship-PowerShell-Installer" target="_blank">created a script</a> that will consume one or more NuGet feeds to:<br />
<br />
<ol>
<li>Download Sitecore.Ship</li>
<li>Recursively download dependent packages</li>
<li>Create a web.config transform from Sitecore.Ship's NuGet packages and your chosen options </li>
<li>Install assemblies and apply config files</li>
<li>Write to an (optional) log file </li>
</ol>
<br />
<h3>
NuGet And You</h3>
As of this writing, if you want to use Sitecore.Ship with Sitecore 8 then you need version 0.4.0 of the Sitecore.Ship NuGet package. Unfortunately, this package isn't yet available on <a href="https://www.nuget.org/" target="_blank">nuget.org</a>. I downloaded the development branch of Sitecore.Ship from GitHub and built a NuGet package with Sitecore 8 update 4 assemblies. I published this NuGet package to Arke's private feed, so if you are an Arke employee, lucky you! If not, I've included the package in my GitHub repository to save you, dear reader, the trouble of generating the NuGet package. In any event, I expect that a public release of Sitecore.Ship 0.4.0+ will be available soon.<br />
<br />
The script supports basic authentication should you choose to host Sitecore.Ship or any dependent packages on a private feed. The script also supports search across multiple feeds. Thus, you could host a private build of Sitecore.Ship on a private feed, but pull dependent packages from public feed(s).<br />
<br />
<h3>
Version Testing</h3>
I've tested the install script against all versions of Sitecore 8 (initial release through update 4.) I've not tried the script with earlier versions of Sitecore, though I suspect it would <i>mostly</i> work. Depending upon demand or my own needs I may extend it to support earlier versions of Sitecore/Sitecore.Ship.<br />
<br />
Happy deployments!Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com0tag:blogger.com,1999:blog-3669209629818099757.post-22661010255276909462015-07-08T09:14:00.000-04:002015-07-08T09:14:24.620-04:00Basic Tips to Prevent Solr Downtime<div class="separator" style="clear: both; display: none;">
<img border="0" height="320" src="http://4.bp.blogspot.com/-Z4zIhxWwjrU/VPo8mj7_7UI/AAAAAAAAAcQ/z8Jr8LDxCZg/s1600/Solr_Logo_square.png" width="320" /></div>
If you've followed my <a href="http://sitecoreblog.patrickperrone.com/2015/02/sitecore-8-solr-part-13-install-tomcat.html">series</a> on installing Solr for Sitecore then you should have a shiny, new Solr instance somewhere in your environment happily indexing Sitecore data and returning results to queries. Hopefully, that never changes, but we all know that hiccups can happen. This post suggests a few things you can do to mitigate or prevent down-time.<br />
<br />
<h3>
Logging</h3>
If you find yourself troubleshooting, you'll be very glad to have Solr-specific logs to refer to. Given how easy this is to configure, you owe it to yourself to do so. Assuming you have the downloaded .zip from Solr:<br />
<ol>
<li>Copy the Jar files from <span style="font-family: "Courier New",Courier,monospace;">solr/example/lib/ext </span>to Tomcat's <span style="font-family: "Courier New",Courier,monospace;">lib/</span> folder.</li>
<li>Copy properties file from <span style="font-family: "Courier New",Courier,monospace;">solr/example/resources</span> to Tomcat's <span style="font-family: "Courier New",Courier,monospace;">lib/</span> folder.</li>
</ol>
All done! You will find your new Solr logs in the install path of Tomcat in the <span style="font-family: "Courier New",Courier,monospace;">logs/</span> folder.<br />
<br />
<h3>
RAM</h3>
When dealing with Solr there are two kinds of RAM to consider. One is the amount of RAM dedicated to the Java heap and the second is OS disk cache. While I can't give specific guidance on how much RAM you should devote and where, I will provide some general advice and guidance.<br />
<h4>
Java Heap</h4>
To set the Java heap size is pretty straightforward matter once you understand the implementation details of Tomcat for your machine. Mainly, this means what version of Tomcat are you running and which OS do you use. I'll be covering Tomcat 8 as a Windows service. If you differ from me in one or more regards, don't despair. Most of what I say still applies to you, you'll probably need to look a little to find the equivalent spots to make your setting changes.<br />
<br />
First, let's review the four different memory management parameters you may control<br />
<ul>
<li><span style="font-family: "Courier New",Courier,monospace;">Xms</span> - The minimum size of your heap</li>
<li><span style="font-family: "Courier New",Courier,monospace;">Xmx</span> - The maximum heap size</li>
<li><span style="font-family: "Courier New",Courier,monospace;">XX:PermSize</span> - Specifies the initial size allocated to the JVM at startup</li>
<li><span style="font-family: "Courier New",Courier,monospace;">XX:MaxPermSize</span> - If necessary, up this maximum will be allocated to the JVM during startup</li>
</ul>
<br />
Most likely, you won't need to worry about <span style="font-family: "Courier New",Courier,monospace;">XX:PermSize</span> and <span style="font-family: "Courier New",Courier,monospace;">XX:MaxPermSize</span> unless you see errors like <b>Java.lang.OutOfMemoryError: PermGen Space</b>. Much more likely, you will want to control the bounds on your already-running heap through <span style="font-family: "Courier New",Courier,monospace;">Xms</span> and <span style="font-family: "Courier New",Courier,monospace;">Xmx</span>. If you are running Tomcat as a Windows service then this is as simple as filling in a text box. For example:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-OqMZKClLDEA/VZmiFN-veuI/AAAAAAAAAnU/CLiBHhq0eTg/s1600/2015-07-05_17-15-42.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="309" src="http://4.bp.blogspot.com/-OqMZKClLDEA/VZmiFN-veuI/AAAAAAAAAnU/CLiBHhq0eTg/s320/2015-07-05_17-15-42.png" width="320" /></a></div>
<br />
The above screenshot shows the equivalent of setting <span style="font-family: "Courier New",Courier,monospace;">-Xms=256m</span> and <span style="font-family: "Courier New",Courier,monospace;">-Xmx=512m</span>. Additionally, I elected to specify the <span style="font-family: "Courier New",Courier,monospace;">XX:PermSize</span> as 128MB. <br />
<br />
As a final note on heap size, be aware that for heap sizes greater than 2GB, garbage collection can cause performance problems. Symptoms are occasional pauses in program execution during a full GC. This can be mitigated through GC tuning of your JVM or electing to use a commercial JVM. <br />
<h4>
Disk Cache </h4>
For disk cache, you would ideally have enough RAM to hold the entire index in memory. Whatever memory remains unused once the OS, running programs, and the Java heap have been satisfied is fair game for disk cache. Thus, if 12GB of RAM is unused, you could potentially fit 12GB of index data into memory before the OS is forced to start paging. In practice, you must use trial and error to find the right memory fit for your data and usage patterns.<br />
<br />
<h3>
Secondary Cores</h3>
Given that you have elected to use Solr, you probably treat search as a first-class citizen in your environment. If you aren't using secondary cores to provide data continuity during an index rebuild, you're simply <b>doing it wrong</b>. It helps that the <a href="http://www.skillcore.net/sitecore/using-switchonrebuildsolrsearchindex-solr-provider-for-sitecore" target="_blank">process for configuring secondary cores</a> is easy to follow.<br />
<br />
Note: every time a rebuild occurs, the <span style="font-family: "Courier New",Courier,monospace;">name</span> values in the <span style="font-family: "Courier New",Courier,monospace;">core.properties</span> files for the two related cores will swap. This is normal behavior, of course, but can be horribly confusing if you aren't aware of it. <b>I.e. don't just assume that the <span style="font-family: "Courier New",Courier,monospace;">name</span> of the core you are viewing matches the core's folder name in your Solr home directory!</b><br />
<br />
<h3>
Replication</h3>
This topic is actually quite broad and probably deserves a blog post or several all of its own. Nevertheless, we can at least imagine the base case of wishing to provision a second Solr instance that is slaved to a master instance. Fail-over will not be automatic although you <i>could</i> script it.<br />
<ol>
<li>Modify the <span style="font-family: "Courier New",Courier,monospace;">core.properties</span> file in your cores to set whether the core is a master or a slave.</li>
<ul>
<li>On Master</li>
<ul>
<li><span style="font-family: "Courier New",Courier,monospace;">enable.master=true</span></li>
<li><span style="font-family: "Courier New",Courier,monospace;">enable.slave=false</span></li>
</ul>
<li>On Slave</li>
<ul>
<li><span style="font-family: "Courier New",Courier,monospace;">enable.master=false</span></li>
<li><span style="font-family: "Courier New",Courier,monospace;">enable.slave=true</span></li>
</ul>
</ul>
<li>Modify <span style="font-family: "Courier New",Courier,monospace;">conf/solrconfig.xml</span> file in each core to include a request handler for replication. Below is a snippet of XML you can use. Simply replace the "remote_host" and "core_name" in the snippet's XML with your environment's values. Note: the way I have constructed this snippet means you can apply it "as is" to any core on your master OR your slave instance. The trick I used was to associate the state of the "enable" property for the master and slave elements with the value of the <span style="font-family: "Courier New",Courier,monospace;">enable.master</span> and <span style="font-family: "Courier New",Courier,monospace;">enable.slave</span> properties from the core's <span style="font-family: "Courier New",Courier,monospace;">core.properties</span> file which you should have set in step 1. This makes your bookkeeping duties a little less painful, especially if you ever find yourself swapping the master and slave around.</li>
</ol>
What to do in the event your master goes down? Edit the master/slave properties in the core.properties file and change the <span style="font-family: "Courier New",Courier,monospace;">ServiceBaseAddress</span> used by Solr in the <span style="font-family: "Courier New",Courier,monospace;">Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config</span> file. You should also (as soon as time allows) edit the Replication handler XML appropriately: either change the URL or comment it out entirely. <br />
<ol>
</ol>
<br />
<script src="https://gist.github.com/patrickperrone/d0e111ca9e9954896028.js"></script>
<span style="font-size: x-small;">Further Reading</span><br />
<ul>
<li><a href="https://wiki.apache.org/solr/SolrPerformanceProblems"><span style="font-size: x-small;">https://wiki.apache.org/solr/SolrPerformanceProblems</span></a></li>
<li><a href="https://cwiki.apache.org/confluence/display/solr/Index+Replication"><span style="font-size: x-small;">https://cwiki.apache.org/confluence/display/solr/Index+Replication</span></a></li>
<li><a href="http://wiki.apache.org/solr/SolrReplication"><span style="font-size: x-small;">http://wiki.apache.org/solr/SolrReplication</span></a></li>
</ul>
Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com2tag:blogger.com,1999:blog-3669209629818099757.post-19383682152646575882015-07-06T11:28:00.003-04:002016-04-25T14:11:57.480-04:00PowerShell Sitecore Install Script<div class="separator" style="clear: both; display: none; text-align: center;">
<img border="0" src="https://4.bp.blogspot.com/-0CeLsi6wg9U/VZqedLC01ZI/AAAAAAAAAns/AHTm3VuTNdo/s1600/powershell.png" /></div>
<div>
<b>Update (4-17-2016)</b>: <a href="http://sitecoreblog.patrickperrone.com/2016/04/sitecore-installer-re-visited.html" target="_blank">Here</a> I discuss enhancements I've made to the script since last year as well as a video tour of the config file used by the script.<br />
<br />
<iframe align="middle" allowfullscreen="" frameborder="0" height="315" src="https://www.youtube.com/embed/YHoxYMKoqYk" width="420"></iframe></div>
<br />
<br />
<a href="https://github.com/patrickperrone/Sitecore-PowerShell-Installer" target="_blank">Download</a> from GitHub.<br />
<br />
I've been searching for a solution to automating Sitecore installations in any environment higher than my personal development VM--for that we already have <a href="https://bitbucket.org/alienlab/sitecore-instance-manager/wiki/Home" target="_blank">SIM</a>. I can be stubborn and exacting, sometimes to a fault, and while a manual install affords me complete control over an environment, it also is horribly time-consuming. Also, if I'm being honest with myself (I'm sure this plainly obvious to you) this process is mistake-prone.<br />
<br />
<h3>
Search for a Solution</h3>
The following are the three most prominent examples of existing solutions I looked at, but I looked at many more. <br />
<h4>
Sitecore Instance Manager</h4>
I've been using SIM for a while now to manage my Sitecore instances on my development box. It's a wonderful solution, but it's not suitable for a production environment without a lot of post-install intervention. I also looked at the <a href="https://github.com/HedgehogDevelopment/BuildExtensions/tree/master/Hedgehog.Tds.Build.Sim.Console" target="_blank">console app</a> for SIM. Alas, while it seems to extend SIM to the command-line, it does not allow for greater flexibility in how SIM installs a Sitecore instance.<br />
<h4>
Sitecore's Installer</h4>
<a href="https://jermdavis.wordpress.com/2014/04/24/unattended-installs-of-sitecore/" target="_blank">Jeremy Davis</a> had the very clever idea of deconstructing Sitecore's .exe installer to get at the underlying .msi file. He successfully identified all of the command-line switches the .msi accepts. I very nearly settled on this approach. After all, one would expect Sitecore's own installer to follow Sitecore's installation guideline's recommendations. It does a better job than SIM, but you are also rather constrained in some of your options and that was a deal breaker for me. <br />
<h4>
PowerShell Script</h4>
All-star Alex Shyba wrote a PowerShell script to automate his installs. His use case is the same as mine for SIM, however: he built the script to install development instances. Like the previous solutions discussed, concerns such as file system permissions, user mappings in SQL, using a domain account for the application pool identity, and CD-hardening are left as post-install exercises for a human.<br />
<br />
<h3>
Inspiration</h3>
Alex's script gave me the push I needed to write my own. My goal is to completely automate a production-ready Sitecore CM or CD server install. Once you run my script the only thing left to do is install your desired modules. Actually, let's take a minute to unpack that, because buried inside that sentence is a subtle point on my deployment philosophy, and it impacts the way I designed my Sitecore installer. I believe that any Sitecore change that can be managed through a .update package (or .zip module) should. For me, this includes managing changes like SwitchMasterToWeb, scalability, and web.config amongst many others. Thus, my over-arching design philosophy for an automated install is to do everything I would normally in SQL, IIS, the file system, and (yes...) the .config files but only enough to create a working instance and no more. Once the installer is done, the instance should be 100% ready for management via .update packages. That is my goal.<br />
<br />
<h3>
The Solution</h3>
I decided to make my script available on GitHub for a couple reasons.<br />
<ul>
<li>I suspect and hope other will want to make use of it</li>
<li>Community feedback will help me improve it</li>
</ul>
<h4>
Major Features</h4>
<ul>
<li>Install Sitecore with or without the databases.</li>
<li>Script sanity checks SQL and input validation prior to making any changes</li>
<li>Write output to the screen and to a log file.</li>
<li>Fine-grained control of the application pool identity (built-in or domain account)</li>
<li>Assign recommended file system permissions on web server.</li>
<li>Add application pool identity to recommended local groups on web server.</li>
<li>Create user mappings for login in SQL.</li>
<li>Install database files on any valid path or UNC</li>
<li>SQL Login used during install doesn't have to be the same account executing the script.</li>
<li>May specifiy a host name and port used for MongoDB</li>
<li>May supply a Solr base address</li>
<li>Choose to use SQL as a session state server</li>
<li>Many CD-hardening options</li>
</ul>
<br />
One limitation of the script today is I do not support choosing MongoDB as a session state server. My suspicion is that this would be a very easy change to make, and I will be including it soon. The script is strictly limited to automating the Sitecore install itself, not MongoDB or Solr. While it's not necessary, it would be a good idea to provisions those applications first if you plan to use them. Speaking of Solr, if you do plan to use it, then be sure to check out my other <a href="http://sitecoreblog.patrickperrone.com/2015/03/a-script-to-swap-search-in-sitecore.html">PowerShell script to change the search provider from Lucene to Solr</a>.<br />
<br />
Finally, I built this script to install Sitecore 8.0. I've briefly tested it with Sitecore 7.5 and it mostly works, but breaks on some assumptions about the existence of .config files like <span style="font-family: "courier new" , "courier" , monospace;">SwitchMasterToWeb.config.example</span> and <span style="font-family: "courier new" , "courier" , monospace;">Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config.example</span>. Even earlier versions of Sitecore would need some more adjustments (example: deal with with differences in databases.) Depending upon the level of interest expressed I will consider making the script compatible with prior Sitecore releases.Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com2tag:blogger.com,1999:blog-3669209629818099757.post-50640284534094759612015-06-30T23:19:00.002-04:002015-07-01T10:29:39.306-04:00Unit Testing in Sitecore<div class="separator" style="clear: both; display: none; text-align: center;">
<img border="0" src="http://2.bp.blogspot.com/-IGWN2gZNqL8/VZNdxv-czsI/AAAAAAAAAm0/0h-xe-IjPUc/s1600/xunit-dot-net-small-logo.png" /></div>
Unit testing seems to be one of those topics that everyone generally agrees is a Good Idea, but when no one is looking it becomes fantastically easy to justify to oneself why for this project at this particular time it's ok to forgo unit tests.<br />
<br />
My purpose here isn't to convince (guilt) you into doing unit tests. Rather, I'd like to demonstrate that actually implementing them can be trivially easy. Now, of course, anyone can write a unit test that is easy but does it add value? Well, that'll depend upon you, of course. My advice is target the hot spots in your code first and anytime you encounter a bug write a unit test that validates the bug is resolved. Aiming for 100% code coverage is admirable but don't let the perfect be the enemy of the good. Ok—enough <strike>preaching</strike> philosophizing!<br />
<h3>
Tools</h3>
First a word about tools. I favor Glass (version 3 for the purpose of this blog post) as my Sitecore ORM; this will have some ramifications for unit tests. As far as a unit testing framework, I have settled upon xUnit. Is this terribly important? Not especially. NUnit is quite popular and powerful. The designers of NUnit actually wrote xUnit. You can view their reasons for doing so <a href="http://xunit.github.io/docs/why-did-we-build-xunit-1.0.html" target="_blank">here</a>. Regardless of which framework you select, most of what I document here will still apply, the main difference is the syntax. For mocking, I have come to really like <a href="https://github.com/sergeyshushlyapin/Sitecore.FakeDb" target="_blank">Sitecore.FakeDb</a>. It's an amazing product. Add a NuGet package to your project and you suddenly have to power to run all of Sitecore's API without the need of a website. There are other Sitecore mocking tools out there, but I highly recommend this one. Finally, for a bit of syntactic sugar I suggest <a href="https://www.nuget.org/packages/FluentAssertions/" target="_blank">Fluent Assertions</a>. Again, simply add the NuGet to your project and your assertion statements will read very nearly like English.<br />
<h3>
Some Lessons Learned</h3>
<ul>
<li>I don't need FakeDb for data mocking so long as I am working solely with Glass object. Since every Glass-mapped class is an implemenation of an interface, my mock test data can also simply implement that interface. </li>
<li>FakeDb is convenient, nonetheless, because it allows Glass's
<span style="font-family: "Courier New",Courier,monospace;">CreateFakeItem</span> method to 'just work' without having to go the extra
length of providing a testing database along with hand-(re)creating all
of the config settings necessary to connect Sitecore's API layer to a data provider. This means that for very simple API calls you could avoid mocking with FakeDb and do it all through Glass. In practice, however, I find FakeDb to be so fast as to leave me wonder if there is any advantage to this.'</li>
<li>I can (if I want) cast from a FakeDb item to a Glass object. Nathanael Mann <a href="http://cardinalcore.co.uk/category/testing/" target="_blank">warns</a> against this practice for performance reasons and because it blurs the line between unit testing and integration testing. In fairness, I am over-simplifying his thesis a bit, nonetheless, for me, the raw convenience of being able easily to mock Sitecore data is simply too powerful to ignore. As far as performance goes, once Glass's context has been created (~800ms hit) unit tests involving Glass run in less than 10ms even when casting. 10 seconds per 1000 tests? I can accept that.</li>
</ul>
<h3>
Everybody Loves an Example</h3>
<br />
<pre class="brush: csharp">namespace MyPOCO.Tests.Data.Domain
{
public class My_POCO_Tests
{
public class DoesQueryStringMatchMethod
{
public static DbTemplate GetMyPOCOTemplate()
{
return new DbTemplate(IMy_POCOConstants.TemplateName, IMy_POCOConstants.TemplateId)
{
new DbField(IMy_POCOConstants.Query_String_Value_To_MatchFieldName,
IMy_POCOConstants.Query_String_Value_To_MatchFieldId)
};
}
[Fact]
public void RequestedUrlHasUpperCaseValues()
{
//arrange
string requestedUrl = "http://www.google.com?foo=bar&color=RED";
GlassMapperContext.CreateContext();
using (var db = new Db
{
FakeDbHelper.GetMyPOCOTemplate(),
new DbItem("test", ID.NewID, IMy_POCOConstants.TemplateId)
{
{IMy_POCOConstants.Query_String_Value_To_MatchFieldName, requestedUrl}
}
})
{
global::Sitecore.Data.Items.Item home = db.GetItem("/sitecore/content/test");
My_POCO poco = home.GlassCast<My_POCO>();
//act
bool result = poco.DoesQueryStringMatch(requestedUrl);
//assert
result.Should().BeTrue();
}
}
}
}
}</pre>
<br />
So, what does this code demonstrate? It shows a Glass-mapped object that has a method I wish to unit test. The method is named <span style="font-family: "Courier New",Courier,monospace;">DoesQueryStringMatch</span>. Presumably, the method compares the requested URL against a field value and returns a boolean. Of course, a more realistic and <b>more useful</b> test would be against a method that had a more complicated job to do, but for purposes of illustrating the technique we'll stick with a contrived example.<br />
<br />
I could have mocked my Glass object without the need for FakeDb, and I do need to create my Glass context (that's the <span style="font-family: "Courier New",Courier,monospace;">GlassMapperContext.CreateContext()</span> method call.) Once that work is done, however, I am free and clear for all further unit tests where I may have need to test against Sitecore items—example: an action for the rules engine. I leverage the fact that Glass maintains constants about my template and its fields. Necessary? No, but very useful if I have to create an item with a complicated structure. Creating the Glass context is simple enough. I just replicate the code called in the <span style="font-family: "Courier New",Courier,monospace;">Start()</span> method of GlassMapperSc.cs.<br />
<br />
<br />Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com2tag:blogger.com,1999:blog-3669209629818099757.post-49240214000614642052015-05-22T20:54:00.001-04:002015-09-07T09:56:31.677-04:00MSBuild and TDS<div class="separator" style="clear: both; display: none; text-align: center;">
<img border="0" height="180" src="http://1.bp.blogspot.com/-ZQ4gsHfL4Dg/VV_QtmwuwnI/AAAAAAAAAmA/qPZEpph49l8/s320/hedgehog-logo.png" width="320" /></div>
I recently had a need to get continuous integration working for a project that uses Team Development for Sitecore (TDS)—most of us do right? :) While some of this blog post will deal specifically with creating a build definition in Team Foundation Server (TFS), the vast majority of this article applies to any software that uses MSBuild under the hood: TeamCity, Jenkins, CruiseControl, etc.<br />
<br />
The wrinkle in my requirements was that I could not install TDS on the build server. There's already a very helpful <a href="http://www.glass.lu/Blog/Archive/Working%20with%20TDS%20and%20TFS%20in%20the%20cloud" target="_blank">resource on this topic</a>, but I did find I had to do things slightly different in my environment. Additionally, I also made it a goal to remove the SlowCheetah dependency (I make use of xml transformations) from my build server. Finally, I ran into a couple of other small roadblocks that I thought I might as well document here while I was at it.<br />
<br />
<h3>
TDS</h3>
As I said, Mike Edwards has an immensely useful article that shows how to avoid installing TDS on your build server. The only things I will add are where I diverged from his steps. For clarity, Mike added a folder called TDSFiles at the root of his solution. I added a folder called <b>MSBuild Support Files</b> with a child folder called <b>TDS</b>.<br />
<br />
<ol>
<li>The <b>HedgehogDevelopmentSitecoreProject.targets</b> file has many references to the <b>HedgehogDevelopment.SitecoreProject.Tasks.dll</b>. For each one of these you need to modify the path. In my case, the correct path is no path at all. This was because (I assume) TFS used the working directory of the .targets file itself as a starting location—the .targets file and the DLL live side-by-side.
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-FNAf8Hzk1Nk/VV-1SmuW1_I/AAAAAAAAAkc/GtC24-aD4V4/s1600/2015-05-22_19-00-53.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="156" src="http://3.bp.blogspot.com/-FNAf8Hzk1Nk/VV-1SmuW1_I/AAAAAAAAAkc/GtC24-aD4V4/s640/2015-05-22_19-00-53.png" width="640" /></a></div>
<br />
</li>
<li>In the same .targets file you will also need to modify the paths of the <b>TdsService.asmx</b> and the <b>HedgehogDevelopment.SitecoreProject.Service.dll</b>. Here is a screenshot of my modifications.
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-yH7PMbDM8Kc/VV-3GI2Db-I/AAAAAAAAAks/8nvp9tq6H00/s1600/2015-05-22_19-05-36.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="66" src="http://4.bp.blogspot.com/-yH7PMbDM8Kc/VV-3GI2Db-I/AAAAAAAAAks/8nvp9tq6H00/s640/2015-05-22_19-05-36.png" width="640" /></a></div>
</li>
</ol>
<br />
<h3>
SlowCheetah</h3>
After corresponding with Hedgehog's <a href="https://www.hhogdev.com/blog.aspx?tag=Charlie+Turano" target="_blank">Charlie Turano,</a> I decided to eliminate MSBuild's dependency on SlowCheetah. This step is only necessary if you do not supply the DLLs and .targets files to MSBuild. One easy way of doing this is to simply include the "packages" folder from NuGet in source control. This guarantees that MSBuild will be able to make use of the files. In fact, this is how my solution was already set up. Nonetheless, TDS is perfectly capable of doing XML transformations during the build. I want to be ready should a future release of TDS completely replace SlowCheetah (a possibility since SlowCheetah's developer has said he will no longer maintain it.)<br />
<br />
This is very easy to do. Simply comment out the following line in any .csproj file that uses SlowCheetah<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/--V4DF8_pJOw/VV-8BCUX-gI/AAAAAAAAAlE/d6q8JMrhzH8/s1600/2015-05-22_19-29-24.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="34" src="http://2.bp.blogspot.com/--V4DF8_pJOw/VV-8BCUX-gI/AAAAAAAAAlE/d6q8JMrhzH8/s640/2015-05-22_19-29-24.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<h3>
Some Miscellaneous Issues</h3>
<ol>
<li>I encountered another .targets related issue. This time it was:<br /><br /><i>The imported project "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\WebApplications\Microsoft.WebApplication.targets" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.</i><br /><br />What's happening? Inside the .csproj file there is a variable $VSToolsPath getting set that ends up being used by MSBuild to resolve the path of the Microsoft.WebApplication.targets. You <i>could</i> modify the .csproj to prevent this behavior, but it's much easier to simply use a command-line switch like so: <br /><br /><span style="font-family: "Courier New",Courier,monospace;">msbuild myproject.csproj /p:VisualStudioVersion=12.0</span>. <br /><br />If you are using TFS then the fix is just as easy: in your build definition on the process tab set your MSBuild Arguments
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-Ct2pTjBoRZg/VV_CynGqPSI/AAAAAAAAAlY/ZxN731p1bQ0/s1600/2015-05-22_19-58-25.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="203" src="http://4.bp.blogspot.com/-Ct2pTjBoRZg/VV_CynGqPSI/AAAAAAAAAlY/ZxN731p1bQ0/s400/2015-05-22_19-58-25.png" width="400" /></a></div>
<br />
</li>
<li>I was receiving a post-build error:<br /><i><br />API restriction: The assembly 'file:///D:\Builds\6\XXXXXXXXXX\XXX-TestBuild\Binaries\_PublishedWebsites\TDS.MyProject\bin\MyProject.Tests.dll' has already loaded from a different location. It cannot be loaded from a new location within the same appdomain.</i><br /><br />The full explanation of what is happening is <a href="http://geekswithblogs.net/jakob/archive/2010/06/08/tfs-2010-build-dealing-with-the-api-restriction-error.aspx" target="_blank">here</a>. The resolution is again very simple. In the build definition make sure you do not recursively match all test DLLs:
<br /><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-m9OGryYP9gc/VV_JXqL_HqI/AAAAAAAAAls/HYrgfAz0St4/s1600/2015-05-22_20-24-21.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="185" src="http://2.bp.blogspot.com/-m9OGryYP9gc/VV_JXqL_HqI/AAAAAAAAAls/HYrgfAz0St4/s400/2015-05-22_20-24-21.png" width="400" /></a></div>
</li>
</ol>
Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com2tag:blogger.com,1999:blog-3669209629818099757.post-31949669415586545562015-05-21T08:44:00.000-04:002015-05-21T08:53:29.130-04:00Create a Reverse Proxy Controlled By Sitecore<div class="separator" style="clear: both; display: none; text-align: center;">
<img border="0" height="213" src="http://4.bp.blogspot.com/-FErQbGkYpto/VV1PCR1RmvI/AAAAAAAAAj8/5jOJlzJjZEs/s320/Traffic-at-night1.jpg" width="320" /></div>
Reverse proxies can be an incredibly useful technology in your Sitecore implementation depending upon your needs. The basic idea is that a reverse proxy forwards requests on to other servers on behalf of the requesting client, sort of like a traffic cop. The responses from the servers behind the reverse proxy are then returned to the requesting client. This can be done in such a way that is completely transparent to the end-user.<br />
<br />
<h3>
The Use Case</h3>
So why bother? Well, as Grant Killian suggests over on his <a href="https://grantkillian.wordpress.com/2015/01/15/the-deal-with-reverse-proxies-and-sitecore/" target="_blank">blog</a> at least two scenarios come to mind (I'm sure all you very smart folks could undoubtedly name more!) I want to focus on the case of a reverse proxy sitting between the Internet and a set of web servers that includes one or more legacy web servers and a Sitecore instance.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-l4q6-TTfAjo/VV0y1wFMDUI/AAAAAAAAAik/POXhLZwNBoo/s1600/ReverseProxyDiagram.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="208" src="http://3.bp.blogspot.com/-l4q6-TTfAjo/VV0y1wFMDUI/AAAAAAAAAik/POXhLZwNBoo/s400/ReverseProxyDiagram.png" width="400" /></a></div>
I've kept the conceptual diagram above simple (no load-balanced servers, firewalls, cache servers, etc.) but the technique readily applies to an enterprise ecosystem. The basic strategy is as follows:<br />
<br />
<ol>
<li>A user tries to browse a page (perhaps one they have bookmarked) e.g. http://company.com/foobar.php</li>
<li>The reverse proxy receives the request and "asks" Sitecore where to route the request</li>
<li>Sitecore tells the reverse proxy if it can handle the request and, if so, what the URL should be.</li>
<li>The reverse proxy rewrites the request and forwards it. For example, if Sitecore responded positively to the reverse proxy our URL might be transformed to http://sitecorecd.company.com/foo/bar </li>
<li>Sitecore or the legacy server responds to the page request</li>
<li>The reverse proxy rewrites the response so that the end-user is unaware the page they see was came from a different server than the one they contacted.</li>
</ol>
<br />
The payoff with this scenario is we can now manage incremental content migrations from legacy servers to Sitecore servers without any disruption to end-user experience. Bookmarks, campaign emails, RSS feeds, Google search result rankings....all of it will happily continue on as always regardless of whether the legacy web server or Sitecore actually answers the HTTP request. Powerful stuff! This technique is especially useful for clients that have a very large inventory of content and cannot or are unwilling to migrate everything all at once.<br />
<br />
<h3>
The Solution</h3>
The first order of business is setting up a reverse proxy in IIS. The goal is to have a dedicated web site in IIS as the reverse proxy. To do that we need to install the Application Request Routing (ARR) <a href="http://www.iis.net/downloads/microsoft/application-request-routing" target="_blank">extension</a>. Once ARR is installed we'll need to do perform the following configuration steps<br />
<br />
<ol>
<li>Open IIS Manager and select the server node. Double click on the <b>Application Request Routing Cache</b> icon.
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-uQTCq5iczdM/VV05bQhqbRI/AAAAAAAAAjA/mGUoeMfxTAQ/s1600/2015-05-20_21-47-57.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="283" src="http://3.bp.blogspot.com/-uQTCq5iczdM/VV05bQhqbRI/AAAAAAAAAjA/mGUoeMfxTAQ/s400/2015-05-20_21-47-57.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
</li>
<li>In the right-hand pane click <b>Server Proxy Settings</b>.
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-amuyiuuOUTE/VV06GWAOeiI/AAAAAAAAAjQ/8tfd7cFWonA/s1600/2015-05-20_19-52-27.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="163" src="http://1.bp.blogspot.com/-amuyiuuOUTE/VV06GWAOeiI/AAAAAAAAAjQ/8tfd7cFWonA/s400/2015-05-20_19-52-27.png" width="400" /></a></div>
<br />
</li>
<li>Check the <b>Enable proxy</b> setting and uncheck the <b>Reverse rewrite host in response headers</b> option.
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-oSEH4HQW4zQ/VV07DufSvrI/AAAAAAAAAjY/APcjrLvY8j0/s1600/2015-05-20_19-53-25.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="228" src="http://3.bp.blogspot.com/-oSEH4HQW4zQ/VV07DufSvrI/AAAAAAAAAjY/APcjrLvY8j0/s320/2015-05-20_19-53-25.png" width="320" /></a></div>
<br />
</li>
<li>Set the <b>Response buffer</b> and <b>Response buffer threshold</b> values for 8092 and then click <b>Apply</b>. The reason for this I discovered through the school of hard knocks: some pages were mysteriously causing YSODs. After digging through logs (more on that later) we found that the response from the server was literally truncated. The page was large enough that it was overflowing the response buffer and causing it to flush with only a part of the overall page.
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-GKNrAAB29Y0/VV089EvBFZI/AAAAAAAAAjo/4Ihssd53iFQ/s1600/2015-05-20_19-56-37.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="173" src="http://4.bp.blogspot.com/-GKNrAAB29Y0/VV089EvBFZI/AAAAAAAAAjo/4Ihssd53iFQ/s320/2015-05-20_19-56-37.png" width="320" /></a></div>
<br />
</li>
</ol>
<br />
Now that ARR is installed and configured at the server level we need to turn our attention to the reverse proxy site. Here is the secret sauce of our solution: rather than merely write in some rules for routing in the web.config we are going to create our own custom Rewrite Provider. This will allow us to execute our own code during the runtime of the reverse proxy. I followed this <a href="http://www.iis.net/learn/extensions/url-rewrite-module/developing-a-custom-rewrite-provider-for-url-rewrite-module" target="_blank">guide</a> to develop my own custom provider; it should get you up and running.<br />
<br />
So what does my rewrite provider do? At its heart it's just a very simple URL resolver. The provider is rather dumb (and it should be!) We want the reverse proxy to do as little processing as possible.<br /><br />
<ol>
<li>First make a request to Sitecore to see if the requested page (rewritten with Sitecore's host header) can be served, i.e. does the web request return with a response status code < 400.</li>
<li>If that fails, the reverse proxy contacts a web service that knows how to map a legacy URL onto a Sitecore URL. Thus a URL like /news/article.php?id=foobar in the legacy system can be mapped onto /news/articles/foobar for example.</li>
<li>If steps 1 and 2 fail, then the request is routed to the legacy server.</li>
</ol>
<br />
You may be wondering if all of that still sounds like too much work given that every request passes through the reverse proxy. Fortunately ARR has very good built-in caching, so in practice, your most requested pages (and resources) will not be processed in code continuously. <br />
<br />
<h3>
Closing Thoughts </h3>
Beyond what's already been covered, I recommend you consider the following:<br />
<br />
<ol>
<li>You need some kind of logging strategy. I write to the event logs from the reverse proxy and Sitecore's logs during the runtime of Sitecore (for example, the URL mapping web service.)</li>
<li>Perform load testing. ARR is remarkably good OOTB with its cache settings, but better to test and know than simply assume.</li>
<li>Think ahead about sessions and how you will deal with them.</li>
<li>Redundancy. Our solution uses more than one reverse proxy. As a side-note with my implementation, if Sitecore itself goes down, the reverse proxy will continue to function. All requests would simply go to the legacy server. Eventually this behavior may become undesirable, but early in a project's lifetime this can be a real selling point.</li>
<li>You definitely need to create some outbound rewrite rules in your reverse proxy's web.config to deal with:</li>
<ol>
<li>Rewriting relative links in the response HTML</li>
<li>Rewriting the Location in the response Header when the status code is a 3XX (a redirect) and the host name is your backend server. This will prevent the end-user's browser being redirected to http://legacy.company.com/foobar rather than http://company.com/foobar</li>
</ol>
<li>You <b>should absolutely turn on </b><a href="http://www.iis.net/learn/troubleshoot/using-failed-request-tracing/using-failed-request-tracing-rules-to-troubleshoot-application-request-routing-arr" target="_blank">Failed Request Tracing Rules</a>. This is the logging function I discussed earlier that proved invaluable in diagnosing and resolving issues during development</li>
</ol>
Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com3tag:blogger.com,1999:blog-3669209629818099757.post-39661094386042208862015-04-27T13:18:00.000-04:002015-05-11T21:26:21.962-04:00SwitchMasterToWeb Woes<div class="separator" style="clear: both; display: none; text-align: center;">
<img border="0" height="320" src="http://2.bp.blogspot.com/-MciILAn-gzM/VTulZNuFgOI/AAAAAAAAAgw/3BtQS3RNFTE/s1600/tag_icon.png" width="320" /></div>
<i><b>Scenario</b></i>: You are using Sitecore 8—I think this also applies to 7.5—and have enabled the SwitchMasterToWeb.config. You now see see an exception related to 'master' database. For example:<br />
<br />
<ul style="list-style-type: none;">
<li>
<span style="font-size: large;"><i>Could not find configuration node: contentSearch/indexConfigurations/indexUpdateStrategies/syncMaster</i></span><br />
</li>
<li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-DayD4xmFVOw/VTuhGwpzmPI/AAAAAAAAAgk/kALjOEz-tWM/s1600/2015-04-25_09-50-19.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="348" src="http://3.bp.blogspot.com/-DayD4xmFVOw/VTuhGwpzmPI/AAAAAAAAAgk/kALjOEz-tWM/s1600/2015-04-25_09-50-19.png" width="640" /></a></div>
<br />
</li>
</ul>
<br />
<i><b>Cause</b></i>: Sitecore patches files found in the \Include folder first and then recursively patches all files found in sub-folders of \Include. In all cases, Sitecore follows alphabetical order when deciding which folder or file to examine next. This means that the SwitchMasterToWeb.config file placed in \Include will be merged before config files found in sub-folders. Some of those config files are for indexes and those index try to use the syncMaster strategy.<br />
<br />
<i><b>Resolution</b></i>: Place the SwitchMasterToWeb.config file in a sub-folder like "zzzMustBeLast" and reload your site!Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com2tag:blogger.com,1999:blog-3669209629818099757.post-71407679639344898232015-04-25T11:07:00.001-04:002015-04-25T11:07:28.476-04:00NuGet Tip: Automatically Set 'Copy Local' Property to False<div class="separator" style="clear: both; display: none; text-align: center;">
<img border="0" src="http://4.bp.blogspot.com/-YUVZIFwxlRM/VTusNvJcuoI/AAAAAAAAAhM/yk2TTvXi8P4/s1600/364px-NuGet_project_logo.svg.png" height="319" width="320" /></div>
This is probably slightly off the beaten path for some of the Sitecore community, but I suspect quite a few of us use NuGet. Furthermore, of those that do, some not only consume but produce NuGet packages. One annoyance I've had is the good ol' Copy Local property of an assembly reference.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-oDoqNXeAFDw/VTupny0j8mI/AAAAAAAAAg8/qoyvgUgZEDg/s1600/2015-04-25_10-48-55.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-oDoqNXeAFDw/VTupny0j8mI/AAAAAAAAAg8/qoyvgUgZEDg/s1600/2015-04-25_10-48-55.png" height="97" width="400" /></a></div>
<br />
When this property is set to true, the assembly will be copied to the output directory of your project. This can be problematic. For example, with TDS, when you build your solution the output directory of your web application project gets copied to your Sitecore instance. Ideally, you don't want to overwrite or add DLLs to your Sitecore instance by accident. While you can prevent this from happening by maintaining an exclusion list in TDS, it's easy to forget.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-qmZh6ZmSKZ4/VTuqZGOH96I/AAAAAAAAAhE/gxzm7e6mp3U/s1600/2015-04-25_10-52-54.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-qmZh6ZmSKZ4/VTuqZGOH96I/AAAAAAAAAhE/gxzm7e6mp3U/s1600/2015-04-25_10-52-54.png" height="245" width="400" /></a></div>
<br />
<br />
How does this relate to NuGet packages?<br />
<br />
Usually when you add a new package one or more DLL references are added to your project(s) and almost invariably the Copy Local property will be set to true. If you create your own NuGet package and wish to prevent this, here is a PowerShell script that will do the trick:<br />
<br />
<script src="https://gist.github.com/patrickperrone/ac4922db06ec2b393de3.js"></script>
<br />
Simply add/overwrite the Install.ps1 script found in the "tools" folder of the NuGet package. Voila!Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com0tag:blogger.com,1999:blog-3669209629818099757.post-13474114853412993362015-04-03T22:21:00.002-04:002015-04-19T22:37:17.314-04:00How to Map Sitecore Rules Field with Glass and TDS<div class="separator" style="clear: both; display: none; text-align: center;">
<img border="0" src="http://2.bp.blogspot.com/-Lkl20N14TuU/VR9KTuyUVUI/AAAAAAAAAgA/WKfQeCswkzQ/s1600/Glass-logo-250.png" /></div>
<i><b>tl;dr</b> Copy lines 246 and 247 from <a href="https://gist.github.com/patrickperrone/9626cccbd044cc418539" target="_blank">here</a>. Regenerate your Glass classes. </i><br />
<br />
Today a colleague asked how to map the rules field from Sitecore with Glass Mapper using TDS and T4 templates. It just so happens that I'd recently worked on this problem, and I thought I would share it with others here.<br />
<br />
The basic issue is that the glassv3item.tt template doesn't know how to deal with the Rules field. The <span style="font-family: "Courier New",Courier,monospace;">GetGlassFieldByType</span> method is responsible for assigning a type to mapped field. It does this with a <span style="font-family: "Courier New",Courier,monospace;">switch</span> statement. Our rules field is falling all the way through to the <span style="font-family: "Courier New",Courier,monospace;">default</span> case which maps the field to an <span style="font-family: "Courier New",Courier,monospace;">object</span>. We need to add a <span style="font-family: "Courier New",Courier,monospace;">case</span> for the <span style="font-family: "Courier New",Courier,monospace;">field.Type</span> value when it equals "rules".<br />
<br />
What type will we map to though? On first pass I thought something like <span style="font-family: "Courier New",Courier,monospace;">XDocument</span> would make a lot of sense. Problem is that doesn't work. The value is always null. I took a look inside the Glass.Mapper.Sc.dll at what I believe is the code responsible for returning a value. It looks to me like the mapping code isn't fully implemented, and Glass relies on a generic method that simply returns a <span style="font-family: "Courier New",Courier,monospace;">string</span> value.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-7vIbgd2g9FM/VR9JTAiP6sI/AAAAAAAAAf4/tAlUCHLXc9k/s1600/2015-03-24_14-02-07.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-7vIbgd2g9FM/VR9JTAiP6sI/AAAAAAAAAf4/tAlUCHLXc9k/s1600/2015-03-24_14-02-07.png" height="194" width="320" /></a></div>
<br />
Not a big deal. We can work with this. <br />
<br />
<code data-gist-highlight-line="246-247" data-gist-id="9626cccbd044cc418539" data-gist-line="188,246-255"></code>
After you've modified the T4 template and regenerated your Glass classes, you should have a mapped property on your class of type <span style="font-family: "Courier New",Courier,monospace;">string</span>. I've found for my purposes this is perfect for my needs, but that didn't stop me from extending my partial class with an <span style="font-family: "Courier New",Courier,monospace;">XDocument</span> property...just in case.<br />
<br />
<pre class="brush: csharp">public partial class GeneratedClass
{
public XDocument RuleAsXDocument
{
get { return XDocument.Parse(this.Rule); }
}
}
</pre>
Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com0tag:blogger.com,1999:blog-3669209629818099757.post-1646581816958667392015-04-02T23:18:00.000-04:002015-04-25T10:36:41.281-04:00A SIMple ErrorWhile installing Sitecore Instance Manager (SIM) I made a silly mistake that gave me pause. I figured I would document it here in the hope it might help someone else might benefit.<br />
<br />
The final step in the installation wizard attempts to do a permission check. It is labeled as "File Systems permission" and, indeed, it does check this and even provides a handy "Grant" button if SIM does not have the permission it thinks it needs.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-OYxBnCc8oq0/VR3_FOD2viI/AAAAAAAAAfI/pooYasNFbQw/s1600/2015-04-02_22-12-35.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-OYxBnCc8oq0/VR3_FOD2viI/AAAAAAAAAfI/pooYasNFbQw/s1600/2015-04-02_22-12-35.png" height="249" width="320" /></a></div>
<br />
What can be a bit confusing is that even after seeing the success message above you still encounter an error dialog complaining that "You probably don't have necessary permissions set. Please try to click 'Grant' button before you proceed." <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-1R_VymyhmA4/VR3_nrZ246I/AAAAAAAAAfQ/_STdMzt6EL4/s1600/2015-04-02_22-12-48.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-1R_VymyhmA4/VR3_nrZ246I/AAAAAAAAAfQ/_STdMzt6EL4/s1600/2015-04-02_22-12-48.png" height="251" width="320" /></a></div>
<br />
What's happening? Under the hood, SIM isn't just checking the file system, it is also trying to create a test database in SQL. If the SQL login SIM uses doesn't have the right to create a database, then SIM will fail this "file system" check with (in this specific case) a misleading error. The fix is simple, make sure your SQL login has the dbcreator role or higher. Thus my silly mistake: <i>Of course SIM needs the ability to create databases why didn't I think of that sooner....DUH!</i> :)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-Rqva4fXUgLo/VR4Dyr7UVuI/AAAAAAAAAfc/IURKAjgl3v0/s1600/2015-04-02_22-14-25.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-Rqva4fXUgLo/VR4Dyr7UVuI/AAAAAAAAAfc/IURKAjgl3v0/s1600/2015-04-02_22-14-25.png" height="198" width="320" /></a></div>
<br />
Rerun the last step and enjoy the wonders of SIM!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-5Mkybb0jziM/VR4DymopJMI/AAAAAAAAAfg/Dt0g8pOGt-Y/s1600/2015-04-02_22-14-42.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-5Mkybb0jziM/VR4DymopJMI/AAAAAAAAAfg/Dt0g8pOGt-Y/s1600/2015-04-02_22-14-42.png" height="249" width="320" /></a></div>
Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com5tag:blogger.com,1999:blog-3669209629818099757.post-66728677439403236592015-03-28T01:23:00.000-04:002015-04-30T20:48:16.466-04:00On Second(ary) Thought...<div class="separator" style="clear: both; display: none; text-align: center;">
<img border="0" src="http://4.bp.blogspot.com/-uHxpIuqeg2U/VRY4deZwA5I/AAAAAAAAAew/aSVRTZq-hJg/s1600/Setting.png" />
</div>
Recently I <a href="http://sitecoreblog.patrickperrone.com/2015/03/a-solr-core-nucopia.html">posted</a> my thoughts regarding the proper ratio of Solr cores to Sitecore indexes. In it, I mentioned the need to double the number of cores to support the SwitchOnRebuildSolrSearchIndex feature. It turns out, this isn't quite right. Creating a secondary core for the analytics index does no good and should be avoided.<br />
<br />
You'll want to avoid a secondary core for the analytics index because trying to use one results in exception. The analytics index has an extra "group" parameter, and Sitecore cannot find a matching constructor. The SolrSearchIndex class allows for the group parameter but the SwitchOnRebuildSolrSearchIndex does not.<br />
<br />
Why is this? As <a href="https://www.sitecore.net/learn/blogs/technical-blogs/getting-to-know-sitecore/posts/2014/10/introducing-the-sitecore-analytics-index.aspx" target="_blank">Adam Conn explains</a>, the types of crawlers responsible for maintaining the analytics index are <i>observers</i> of data. This means that the crawler is notified when data is available and then passed that data. Therefore, rebuilding the analytics index would result in an empty index unless you <a href="http://blog.horizontalintegration.com/2015/01/10/rebuilding-sitecore-8-analytics-indexes/" target="_blank">rebuild the reporting database</a> so that the analytics crawlers may <i>observe</i> (and index) the data. This is the reason why the UI doesn't provide the option of rebuilding the analytics index.Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com0tag:blogger.com,1999:blog-3669209629818099757.post-40734630116488133812015-03-24T23:06:00.000-04:002015-04-30T20:48:42.993-04:00Solr + Glass = Castle Windsor Crashers?<div class="separator" style="clear: both; display: none; text-align: center;">
<img border="0" src="http://1.bp.blogspot.com/-r8R0769L1yE/VRIl4SpQpdI/AAAAAAAAAeY/o3qb19gljoU/s1600/Sand_Castle_Washing_Away.jpg" height="320" width="320" />
</div>
(I know the title isn't technically true...don't judge me!) <br />
<br />
I think it's safe to say most of us Sitecore developers are using some form of ORM these days, right? ...Right? :)<br />
<br />
Of course you are, and – most likely – that means you are using <a href="http://www.glass.lu/Mapper/Sc" target="_blank">Glass Mapper</a>. Probably, you are also using Castle Windsor as your choice of Inversion of Control. You could use some other IoC, but there's a NuGet package that makes incorporating Castle Windsor + Glass into your Visual Studio solution quite easy.<br />
<br />
So what if you read my series on <a href="http://sitecoreblog.patrickperrone.com/2015/02/sitecore-8-solr-part-13-install-tomcat.html">Solr</a> and were inspired to use it for Sitecore? Solr also requires an IoC. As I mentioned here, you may choose from Castle Windsor, AutoFac, Ninject, StructureMap, and Unity. It seems only natural to stick with the same IoC as Glass. Common sense says don't add more moving parts than necessary.<br />
<br />
And this is why common sense isn't always common or sensible. The moment you deploy your Glass dependent (and thus Castle Windsor dependent) code to your Sitecore instance, you are going to be treated to some ugly YSODs. The issue is that Glass wants to use a higher version of Castle Windsor (3.2) than your Solr-enabled Sitecore instance; it wants version 3.1.<br />
<br />
Luckily the fix is relatively painless. We are going to instruct .NET to redirect bindings of earlier versions of Castle.Core and Castle.Windsor to our later version. All you need to do is add the following section to your web.config:<br />
<br />
<script src="https://gist.github.com/patrickperrone/dc4f49946a46e005cce2.js"></script>Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com0tag:blogger.com,1999:blog-3669209629818099757.post-28105418953433253182015-03-07T23:00:00.000-05:002015-04-19T22:40:41.599-04:00A Script to Swap Search in Sitecore<div class="separator" style="clear: both; display: none;">
<img border="0" src="http://4.bp.blogspot.com/-srQBg7zdblI/VPvPLcnHbFI/AAAAAAAAAc4/E3rU6kRYrQc/s1600/GitHub-Mark-120px-plus.png" /></div>
A colleague of mine had the very good idea of saving some time when switching Sitecore between Lucene and Solr (and back again) with a console program. I decided to rewrite it in PowerShell. Help yourselves!
<br />
<br />
<script src="https://gist.github.com/patrickperrone/59b8745ee8b8ff9045b5.js"></script>Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com0tag:blogger.com,1999:blog-3669209629818099757.post-46115866002856953422015-03-06T12:18:00.000-05:002015-05-20T16:01:25.614-04:00A Solr Core-nucopia?<div class="separator" style="clear: both; display: none;">
<img border="0" height="320" src="http://4.bp.blogspot.com/-Z4zIhxWwjrU/VPo8mj7_7UI/AAAAAAAAAcQ/z8Jr8LDxCZg/s1600/Solr_Logo_square.png" width="320" /></div>
[<i>N.B.: If you haven't already, check out my series of posts (<a href="http://sitecoreblog.patrickperrone.com/2015/02/sitecore-8-solr-part-13-install-tomcat.html">1</a>, <a href="http://sitecoreblog.patrickperrone.com/2015/02/sitecore-8-solr-part-23-install-solr.html">2</a>, and <a href="http://sitecoreblog.patrickperrone.com/2015/02/sitecore-8-solr-part33-configur-ageddon.html">3</a>) that walk through installing Solr for Sitecore.</i>] <br />
<br />
Given that there isn't yet a search scaling guide for Sitecore 8.0, the current, best authoritative source of guidance is the <a href="http://sdn.sitecore.net/upload/sitecore7/75/sitecore_search_scaling%20guide_sc75-usletter.pdf" target="_blank">Sitecore Search Scaling Guide for 7.5</a>. There is an interesting line in the guide that recommends creating "separate cores for each Sitecore index." This was necessary to avoid inconsistent and unexpected results.<br />
<br />
While digging through the <a href="https://dev.sitecore.net/~/media/Downloads/Sitecore_Experience_Platform/8_0/Sitecore_Experience_Platform_8_update-1/Secure/Sitecore_8_0_update-1_Release_Notes.ashx" target="_blank">release notes for Sitecore 8 update 1</a> I found a good, technical description of the risk:<br />
<br />
<blockquote class="tr_bq">
When two indexes were configured to use the same SOLR core, it was impossible to differentiate the index data between the indexes. As a result, index data related in one index would override the index data in the other index. This has been fixed so that the _uniqueid index field value has been extended with information about the index name. (426743)</blockquote>
<br />
Out of curiosity I decided to validate the fix. Here is a query from one of my Solr cores connected to a Sitecore 8 update 2 instance (a big thumbs-up to Solr's built-in admin tool!)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-Va6DC3VSdY8/VPnRoFgDzQI/AAAAAAAAAbg/AdF5LaB7AE8/s1600/2015-03-06_11-09-40.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-Va6DC3VSdY8/VPnRoFgDzQI/AAAAAAAAAbg/AdF5LaB7AE8/s1600/2015-03-06_11-09-40.png" /></a></div>
<br />
Let's breakdown the taxonomy of this _uniqueid:<br />
<ol style="list-style-type: none;">
<li>
<br />
<span style="font-family: "Courier New",Courier,monospace;">sitecore://<database name>/<item id>?lang=<language name>&ver=<version number>&ndx=<index name></span><br />
</li>
</ol>
<br />
Clearly, the index name is now a part of the key! Does this mean you can disregard the advice in the Search Scaling Guide? In a word, yes. <b>A more important question is, "Should you?" </b>Well, probably not. Here are some reasons why:<br />
<br />
<ol>
<li>Any single Sitecore index (Solr core) rebuild is less expensive since there is less data. Thus, the rebuild is quicker.</li>
<li>When reviewing statistics about a core in Solr's core admin, facts about the core such as the number of documents easily translate to facts about the Sitecore index.</li>
<li>Probably most important of all, it's possible to tune the cache and core's settings as necessary per Sitecore index. Undoubtedly, usage patterns will vary per Sitecore index. So should the strategies you implement to tune the Solr core responsible for that Sitecore index.</li>
</ol>
<br />
<b>Update (3-8-2015)</b>: In Sitecore update 2, one Solr core was removed and two were added. I updated the paragraph below to reflect this.<br />
<br />
Keep in mind, these advantages do come at a cost. There will be some amount of overhead incurred per core. Also, there is the management headache of maintaining many cores. As of Sitecore 8 update 2 there are 13 indexes for a vanilla install. If you want to take advantage of the <a href="http://www.skillcore.net/sitecore/using-switchonrebuildsolrsearchindex-solr-provider-for-sitecore" target="_blank">SwitchOnRebuildSolrSearchIndex feature</a> (while an index rebuilds, Sitecore can still return search results for that index) then you will need to add an additional core for each Sitecore index that uses this feature. That is a possible <strike>26</strike> 25 (<a href="http://sitecoreblog.patrickperrone.com/2015/03/on-secondary-thought.html">see here for an explanation why the total number changed</a>) cores to manage!<br />
<br />
I'm interested in other people's opinions on this topic. Let me know what you all think.Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com0tag:blogger.com,1999:blog-3669209629818099757.post-50923324226426402592015-02-28T08:16:00.000-05:002015-09-03T21:25:03.484-04:00Sitecore 8 + Solr (part 3/3): Configur-ageddon<div class="separator" style="clear: both; display: none;">
<img border="0" height="320" src="http://4.bp.blogspot.com/-Z4zIhxWwjrU/VPo8mj7_7UI/AAAAAAAAAcQ/z8Jr8LDxCZg/s1600/Solr_Logo_square.png" width="320" /></div>
So far in this series we have installed <a href="http://sitecoreblog.patrickperrone.com/2015/02/sitecore-8-solr-part-13-install-tomcat.html">Tomcat</a> and <a href="http://sitecoreblog.patrickperrone.com/2015/02/sitecore-8-solr-part-23-install-solr.html">Solr</a>. Hopefully it has been a relatively easy process. Unfortunately, I can't promise today's post will be as easy. I've tried to balance making the configuration process as pain-free as possible, while not shielding anyone from the details that tripped me up. My goal is to take (all?) the guesswork out of this process. Let me know how I did!<br />
<br />
[<i>N.B.: As I was finishing up this final post I saw that <a href="https://dev.sitecore.net/Downloads/Sitecore_Experience_Platform/8_0/Sitecore_Experience_Platform_8_update-2.aspx" target="_blank">update 2 for Sitecore</a> was released. A quick perusal of the release notes leaves me feeling fairly confident that these instructions -- created with Sitecore 8 update 1 in mind -- should still be valid for update 2. No doubt I will soon need to upgrade my Sitecore environments to update 2 in the near future. Rest assured any issue(s) I encounter I will document here.</i>]<br />
<br />
<b>Update (3-3-2015, 3-8-2015)</b>: As promised, I have updated this guide to comply with Sitecore 8 update 2. This means that if you want to upgrade from update 1, then you will need to create a couple of new cores and delete an old one (see step 6) and replace a DLL (see step 9). If you are starting fresh from update 2, then don't worry about any of this and dive right in!<br />
<br />
<br />
<ol>
<li>Stop the Tomcat service.</li>
<li>Go to the root of your Solr instance, in my case, D:\solr. We need to modify the "collection1" directory to serve as one of the cores (folders with config files and index data) required by Sitecore.</li>
<ol type="a">
<li>Rename the folder to "sitecore_analytics_index"</li>
<li>Inside the folder you will find a file called “core.properties” which you may edit with a text editor. You need to change the “name” value to the name of your core. For example, when I create the “sitecore_analytics_index” core, then I will edit the “core.properties” file to have the value <span style="font-family: "Courier New",Courier,monospace;">name=sitecore_analytics_index</span><br /><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-8NP9gPKgL0Y/VPyhSi36A4I/AAAAAAAAAdY/EeaqVuUx8Kg/s1600/2015-03-08_15-20-33.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-8NP9gPKgL0Y/VPyhSi36A4I/AAAAAAAAAdY/EeaqVuUx8Kg/s1600/2015-03-08_15-20-33.png" /></a></div>
<br />
</li>
</ol>
<li>
Start the Tomcat service and visit the Solr administration page. You may need to reload your browser page if it was already up. Click on the “Core Admin” menu item. If you modified the old “collection1” core correctly you should now see a sitecore_analytics_index core.<br /><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-ZHyOwVapnoY/VPyiMIVI-5I/AAAAAAAAAdg/9u7vC4hi7Yg/s1600/2015-03-08_15-24-43.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="217" src="http://2.bp.blogspot.com/-ZHyOwVapnoY/VPyiMIVI-5I/AAAAAAAAAdg/9u7vC4hi7Yg/s1600/2015-03-08_15-24-43.png" width="320" /></a></div>
<br />
</li>
<li>Stop the Tomcat service. Now we need to fix the schema of our “sitecore_analytics_index” core, otherwise Sitecore cannot parse the xml correctly. Edit the file at \sitecore_analytics_index\conf\schema.xml according to Sitecore’s knowledge base article: <a href="https://kb.sitecore.net/articles/227897" target="_blank">https://kb.sitecore.net/articles/227897</a>. Don’t forget to define the field type for pint since we are using a version of Solr later than 4.9! Start Tomcat and reload the Core Admin page.</li>
<li>Next we must generate a new, Sitecore-specific schema. Sitecore provides a tool for this. Navigate to the Control Panel of your Sitecore instance. Look for the “Generate the Solr Schema.xml file” link and click it. Provide a path for the source and target files (they can’t be the same file.) Once you have generated your new schema, replace the old schema with it. Restart the Tomcat service and make sure the core loads correctly. </li>
<li>A vanilla install of Sitecore 8 update 2 requires 13 cores to work correctly. So far we have one, but don’t despair, now that we have a generated a schema this process is much easier. Essentially, we are going to use our sitecore_analytics_index core as a template to create the others. To do this:</li>
<ol type="a">
<li>Copy the sitecore_analytics_index folder.</li>
<li>Repeat steps 2a and 2b for each copy.</li>
<li>When you are done, your Solr home folder should contain the following cores<br /><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-l00Ue4NM7Wc/VPyjuXrLAcI/AAAAAAAAAds/JF3GxVy5-BM/s1600/2015-03-04_23-14-53.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="201" src="http://4.bp.blogspot.com/-l00Ue4NM7Wc/VPyjuXrLAcI/AAAAAAAAAds/JF3GxVy5-BM/s1600/2015-03-04_23-14-53.png" width="320" /></a></div>
<br />
</li>
<li>If you have done everything correctly you should be able to restart Tomcat and see all the cores listed above on the Core Admin page.</li>
</ol>
<li><b>Update (3-7-2015)</b>: I decided to create a <a href="https://gist.github.com/patrickperrone/59b8745ee8b8ff9045b5" target="_blank">PowerShell shortcut</a> for this step. Save yourself time!<br /><br />Still with me? Hang in there, we are halfway home! Next, we must tell Sitecore to start using Solr instead of Lucene. This is done by appending or removing “disabled” as an extension of a configuration file’s name.</li>
<ol type="a">
<li>Config files to <b>DISABLE</b>:<br /><br />
<span style="font-size: xx-small;">
\App_Config\Include\Sitecore.ContentSearch.Lucene.DefaultConfigurations.config.disabled<br />
\App_Config\Include\Sitecore.ContentSearch.Lucene.Index.Analytics.config.disabled<br />
\App_Config\Include\Sitecore.ContentSearch.Lucene.Index.Core.config.disabled<br />
\App_Config\Include\Sitecore.ContentSearch.Lucene.Index.Master.config.disabled<br />
\App_Config\Include\Sitecore.ContentSearch.Lucene.Index.Web.config.disabled<br />
\App_Config\Include\Sitecore.ContentSearch.Lucene.Indexes.Sharded.Core.config.example (left as is)<br />
\App_Config\Include\Sitecore.ContentSearch.Lucene.Indexes.Sharded.Master.config.example (left as is)<br />
\App_Config\Include\Sitecore.ContentSearch.Lucene.Indexes.Sharded.Web.config.example (left as is)<br />
\App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Lucene.Index.Master.config.disabled<br />
\App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Lucene.Index.Web.config.disabled<br />
\App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Lucene.IndexConfiguration.config.disabled<br />
\App_Config\Include\ContentTesting\Sitecore.ContentTesting.Lucene.IndexConfiguration.config.disabled<br />
\App_Config\Include\FXM\Sitecore.FXM.Lucene.Index.DomainsSearch.config.disabled<br />
\App_Config\Include\ListManagement\Sitecore.ListManagement.Lucene.Index.List.config.disabled<br />
\App_Config\Include\ListManagement\Sitecore.ListManagement.Lucene.IndexConfiguration.config.disabled<br />
\App_Config\Include\Social\Sitecore.Social.Lucene.Index.Master.config.disabled<br />
\App_Config\Include\Social\Sitecore.Social.Lucene.Index.Web.config.disabled<br />
\App_Config\Include\Social\Sitecore.Social.Lucene.IndexConfiguration.config.disabled</span><br /><br />
</li>
<li>Config files to <b>ENABLE</b>:<br /><br />
<span style="font-size: xx-small;">
\App_Config\Include\Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config<br />
\App_Config\Include\Sitecore.ContentSearch.Solr.Index.Analytics.config<br />
\App_Config\Include\Sitecore.ContentSearch.Solr.Index.Core.config<br />
\App_Config\Include\Sitecore.ContentSearch.Solr.Index.Master.config<br />
\App_Config\Include\Sitecore.ContentSearch.Solr.Index.Web.config<br />
\App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Solr.Index.Master.config<br />
\App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Solr.Index.Web.config<br />
\App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Solr.IndexConfiguration.config<br />
\App_Config\Include\ContentTesting\Sitecore.ContentTesting.Solr.IndexConfiguration.config<br />
\App_Config\Include\FXM\Sitecore.FXM.Solr.Index.DomainsSearch.config<br />
\App_Config\Include\ListManagement\Sitecore.ListManagement.Solr.Index.List.config<br />
\App_Config\Include\ListManagement\Sitecore.ListManagement.Solr.IndexConfiguration.config<br />
\App_Config\Include\Social\Sitecore.Social.Solr.Index.Master.config<br />
\App_Config\Include\Social\Sitecore.Social.Solr.Index.Web.config<br />
\App_Config\Include\Social\Sitecore.Social.Solr.IndexConfiguration.config</span><br /><br />
</li>
</ol>
<li>So I know that last step was pretty tedious, but if you've made it this far then the rest will be easier. Download the <a href="https://dev.sitecore.net/~/media/Downloads/Sitecore_Experience_Platform/8_0/Sitecore_Experience_Platform_8_update-1/Secure/Sitecore,-d-,Solr,-d-,Support_1,-d-,0,-d-,0_rev,-d-,_150120.ashx" target="_blank">Solr Support Package</a> from Sitecore and extract the contents of the zip file.</li>
<li>Copy the following DLLs from the Solr Support Package into the \bin folder of your Sitecore Instance [<i>N.B.: Castle Windsor is my Inversion of Control preference as Glass also uses it. Aside from Castle Windsor, Sitecore supports AutoFac, Ninject, StructureMap, and Unity.</i>]:
<br /><br />
</li>
<ul style="list-style-type: none;">
<li>Castle.Facilities.SolrNetIntegration.dll</li>
<li>Microsoft.Practices.ServiceLocation.dll</li>
<li>Sitecore.ContentSearch.Linq.Solr.dll</li>
<li>Sitecore.ContentSearch.SolrProvider.CastleWindsorIntegration.dll</li>
<li>Sitecore.ContentSearch.SolrProvider.dll</li>
<li>SolrNet.dll<br /><br /><b>Update (3-3-2015)</b>: If you are upgrading from Sitecore 8 update 1, then you only need to replace Sitecore.ContentSearch.SolrProvider.XXXXXIntegration.dll with the latest version from the Solr Support Package. All other DLLs remain unchanged.</li>
</ul>
<br />
<li>Download the Nuget package for <a href="https://www.nuget.org/api/v2/package/castle.windsor/3.1.0" target="_blank">Castle Windsor</a>. Unzip the package by renaming the extension from .nupkg to .zip and extracting its contents. Copy the Castle.Windsor.dll from \lib\net40-client to the \bin folder of your Sitecore instance.</li>
<li>Repeat step 10 for the <a href="https://www.nuget.org/api/v2/package/castle.core/3.1.0">Castle.Core</a> Nuget package. Copy the Castle.Core.dll from \lib\net40-client to the \bin folder of your Sitecore instance.</li>
<li>Since we are going to use IoC, we need to make our Sitecore instance aware of it by replacing the Application directive in the global.asax file with the following:
<br /><br />
<span style="font-family: "Courier New",Courier,monospace;"><%@Application Language='C#' Inherits="Sitecore.ContentSearch.SolrProvider.CastleWindsorIntegration.WindsorApplication" %></span>
<br /><br />
</li>
<li>In order for Sitecore to talk to Solr, we need to give it a URL. This setting is maintained in the Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config file (remember, your address may differ from mine):
<br /><br />
<span style="font-family: "Courier New",Courier,monospace;"><setting name="ContentSearch.Solr.ServiceBaseAddress" value="http://tomcat:8080/solr" /></span>
<br /><br />
</li>
<li>SOOO CLOSE! At this point, you are ready to test how badly you have broken your Sitecore instance! With any luck, when you browse to your Sitecore site you won't encounter any yellow screens of death. If you see a YSOD complaining about "Connection error to search provider [Solr] : Unable to connect to [http://tomcat:8080/solr]" then you are likely either missing a core or made typo when creating one. Assuming you are error free, the final step is to re-index. Go to the Control Panel and look for the "Indexing manager" link. Select all indexes and click the "Rebuild" button.</li>
<li>Drink a beer; you deserve it!</li>
</ol>
Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com11Atlanta, GA, USA33.7489954 -84.387982433.3266004 -85.0334294 34.1713904 -83.7425354tag:blogger.com,1999:blog-3669209629818099757.post-14115550932665398482015-02-27T13:29:00.001-05:002015-04-19T22:41:45.911-04:00Sitecore 8 + Solr (part 2/3): Install SolrIn <a href="http://sitecoreblog.patrickperrone.com/2015/02/sitecore-8-solr-part-13-install-tomcat.html">part one</a> of this series, I covered the installation of Tomcat. Now we are ready to get an instance of Solr up and running.<br />
<br />
<ol>
<li>Download Solr 4.10.3 from <a href="http://lucene.apache.org/solr/downloads.html">http://lucene.apache.org/solr/downloads.html</a>. Since we are installing on a Windows machine, you will want to get the .zip file version of the download.</li>
<li>Extract the contents of the .zip to a temporary location of your choice.</li>
<li>Find the \dist folder in the extracted solr-4.10.3 directory. Rename the solr-4.10.3.war to solr.war and copy the file to the Tomcat’s \webapps folder. The path in my environment was C:\Program Files\Apache Software Foundation\Tomcat 8.0\webapps.</li>
<li>Create an empty Solr home folder. This will be the permanent place of residence on your machine for your Solr instance. For example, D:\solr is where I put my Solr instance.</li>
<li>Find the \example\solr folder in the extracted solr-4.10.3 directory. Copy the contents of \example\solr to the empty Solr home folder you just created in step 4.</li>
<li>Find the \example\lib\ext folder in the extracted solr-4.10.3 directory. Copy the contents of \example\lib\ext to Tomcat’s \lib folder. The path in my environment was C:\Program Files\Apache Software Foundation\Tomcat 8.0\lib.</li>
<li>Set the home directory for Solr in Tomcat. This is done by adding a new Java option with the Monitor Tomcat program. In my case, the option was <span style="font-family: "Courier New",Courier,monospace;">-Dsolr.solr.home=D:\solr</span>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-4458QlALa1w/VO_OCZAZ6NI/AAAAAAAAAYk/nhnSHFNL3m4/s1600/2015-02-23_14-42-58.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-4458QlALa1w/VO_OCZAZ6NI/AAAAAAAAAYk/nhnSHFNL3m4/s1600/2015-02-23_14-42-58.png" height="305" width="320" /></a></div>
<br />
</li>
<li>Stop/start Tomcat and try browsing to your Solr instance. For example, <a href="http://tomcat:8080/solr">http://tomcat:8080/solr</a>.
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-Ntn5qYZ6DWs/VO_TC8wa7EI/AAAAAAAAAYw/O1DQZ_y_dcc/s1600/2015-02-26_21-13-54.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-Ntn5qYZ6DWs/VO_TC8wa7EI/AAAAAAAAAYw/O1DQZ_y_dcc/s1600/2015-02-26_21-13-54.png" height="204" width="320" /></a></div>
<br />
</li>
</ol>
In the <a href="http://sitecoreblog.patrickperrone.com/2015/02/sitecore-8-solr-part33-configur-ageddon.html">third and final post</a> of this series I'll show you how to get Sitecore and your shiny, new Solr instance working together.Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com0tag:blogger.com,1999:blog-3669209629818099757.post-83784793728775682582015-02-26T15:00:00.000-05:002015-05-28T11:43:07.871-04:00Sitecore 8 + Solr (part 1/3): Install TomcatThis is part one of a three-part series intended to guide you step-by-step through the process of making Sitecore 8 and Solr work together. There are some good resources on the Internet, but none of them are a soup-to-nuts walkthrough, and unfortunately there are many pitfalls along the way especially if you are unfamiliar with tools like Apache Tomcat or Solr -- even if you are, the current state of Sitecore's documentation for working with Solr isn't complete for Sitecore 8. I'm sure that will change soon! :)<br />
<br />
So, in this first post our goal is to get Apache Tomcat running on a Windows machine installed as a Windows service. Onwards!<br />
<br />
<ol>
<li>Since Solr and Tomcat both require the Java Runtime Environment (JRE) to run, we'll need to start with installing the correct JRE for our machine. This guide uses <a href="http://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html" target="_blank">jre-8u31-windows-x64.exe</a>.</li>
<li><a href="http://tomcat.apache.org/download-80.cgi" target="_blank">Download</a> and install the <b>Windows Service Installer</b> for Tomcat. This guide uses version 8.0.18 of Tomcat, but you can safely use a more recent version. There are a few decisions to make with the installation wizard. You may choose to install the Host Manager component; this will expose a host manager gui to Tomcat’s built-in administration page.<br /><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-5r3koQjJvaA/VO90VQysTRI/AAAAAAAAAXU/wcIqSxKmZ5g/s1600/2015-02-25_15-52-59.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="248" src="http://2.bp.blogspot.com/-5r3koQjJvaA/VO90VQysTRI/AAAAAAAAAXU/wcIqSxKmZ5g/s1600/2015-02-25_15-52-59.png" width="320" /></a></div>
<br />
</li>
<li>The default port used by Tomcat is 8080 (used to be 8983) but you may elect to use some other port. Don’t choose port 80 unless you are prepared to tinker with IIS to re-route requests meant for Tomcat to Tomcat; it’s not really worth the effort. Nevertheless, here is a link to a helpful resource: <a href="https://confluence.atlassian.com/display/JIRA/Integrating+JIRA+with+IIS">https://confluence.atlassian.com/display/JIRA/Integrating+JIRA+with+IIS</a>.<br /><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-SCgUb-wBet0/VO91liMTsLI/AAAAAAAAAXc/CEflksYXm1o/s1600/2015-02-25_15-58-30.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="249" src="http://1.bp.blogspot.com/-SCgUb-wBet0/VO91liMTsLI/AAAAAAAAAXc/CEflksYXm1o/s1600/2015-02-25_15-58-30.png" width="320" /></a></div>
<br />
If you do not supply an optional user name and password, then you will need to manually edit the conf/tomcat-users.xml file (found under the Tomcat install path) in order to manage Tomcat from the built-in administrative web site. If you do find yourself editing the tomcat-users.xml file, don’t forget to stop-restart Tomcat for those changes to take effect. You can do this with the Monitor Tomcat tool:<br /><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-r2f1DX1vxEQ/VO94jOkHhbI/AAAAAAAAAX0/IY6L_tw66UY/s1600/2015-02-25_16-16-11b.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="307" src="http://3.bp.blogspot.com/-r2f1DX1vxEQ/VO94jOkHhbI/AAAAAAAAAX0/IY6L_tw66UY/s1600/2015-02-25_16-16-11b.png" width="320" /></a></div>
<br />
</li>
<li>Test the Tomcat installation by pointing your browser to <a href="http://localhost:8080/">http://localhost:8080</a> assuming that you used port 8080 as your HTTP Connector Port.<br /><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-SJglJ4nN8CU/VO_HjFEl-UI/AAAAAAAAAYY/U4IHJJsqhMw/s1600/2015-02-26_20-24-58.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="236" src="http://4.bp.blogspot.com/-SJglJ4nN8CU/VO_HjFEl-UI/AAAAAAAAAYY/U4IHJJsqhMw/s1600/2015-02-26_20-24-58.png" width="320" /></a></div>
<br />
</li>
<li>By default, the Tomcat Windows service startup type is “Manual.” This means that after rebooting your machine you will need to remember to manually restart the Tomcat service. Change the Tomcat Windows Service startup to “Automatic.”<br /><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-evjXRAeZCIU/VO94WOVgvmI/AAAAAAAAAXs/IrROM3NgoQs/s1600/2015-02-25_16-16-11.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="307" src="http://1.bp.blogspot.com/-evjXRAeZCIU/VO94WOVgvmI/AAAAAAAAAXs/IrROM3NgoQs/s1600/2015-02-25_16-16-11.png" width="320" /></a></div>
<br />
</li>
<li>As a final, optional step, you might elect to add an alias to Tomcat so that the website can be accessed using something other than localhost. For example, you could add the tag <span style="font-family: "Courier New",Courier,monospace;"><Alias>tomcat</Alias></span> to the conf/server.xml file. Don't forget—if you use an alias then make sure you either update your DNS server to use that alias or edit your hosts file in Windows. :)<br /><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-xMbR7I_LbzI/VO95iQ2NtGI/AAAAAAAAAX8/KVOCsWY1ri4/s1600/2015-02-25_16-30-48.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="62" src="http://3.bp.blogspot.com/-xMbR7I_LbzI/VO95iQ2NtGI/AAAAAAAAAX8/KVOCsWY1ri4/s1600/2015-02-25_16-30-48.png" width="320" /></a></div>
<br />
</li>
</ol>
At this point you should have a working Tomcat installation that will be available even after a reboot of your machine. In the <a href="http://sitecoreblog.patrickperrone.com/2015/02/sitecore-8-solr-part-23-install-solr.html">next post</a> of this series we will tackle installing Solr on top of Tomcat.Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com2tag:blogger.com,1999:blog-3669209629818099757.post-58136505206971832412015-02-01T18:24:00.003-05:002015-02-28T10:30:06.928-05:00MongoDB University Course for .NET Developers<div class="separator" style="clear: both; display: none;">
<img border="0" src="http://3.bp.blogspot.com/-aelvt4P5Tv8/VM7mFHwhwRI/AAAAAAAAAXI/HpnS1yQkh18/s1600/mongodb_icon.png" />
</div>
Given that MongoDB is an integral part of Sitecore's xDB architecture starting with Sitecore 7.5, it stands to reason that our clients and colleagues will look to us, the Sitecore experts, for guidance. So what can us mere mortals who aren't already <a href="http://en.wikipedia.org/wiki/NoSQL" target="_blank">NoSQL</a> experts do to prepare ourselves? Enroll in a course taught by NoSQL experts, of course!<br />
<br />
Now, I know what you might be thinking, "Online courses are of limited value." And, normally, I'm right there with you. The quality of the material on offer from <a href="https://university.mongodb.com/courses/catalog" target="_blank">MongoDB University</a> is truly worth the investment of your time, however. The only catch was none of the courses on offer were tailored for .NET developers. Last year I decided to dive in anyway with the Node.js course -- it's excellent, of course -- but just recently MongoDB University has <a href="https://university.mongodb.com/courses/M101N/about" target="_blank">expanded</a> their course catalog with something for us .NET folks! Be prepared to devote several hours a week, and in return you will get a solid foundation on which to build solutions ranging from CRUD, to schema design, to performance tuning and application engineering.Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com0tag:blogger.com,1999:blog-3669209629818099757.post-11204996902505473822015-02-01T13:51:00.000-05:002015-07-08T09:27:04.899-04:00Solr 4.8 and Higher with Sitecore - Schema Issue Resolved<div class="separator" style="clear: both; display: none;">
<img border="0" height="320" src="http://4.bp.blogspot.com/-Z4zIhxWwjrU/VPo8mj7_7UI/AAAAAAAAAcQ/z8Jr8LDxCZg/s1600/Solr_Logo_square.png" width="320" /></div>
Dan Solovay has a great <a href="http://www.dansolovay.com/2013/05/setting-up-solr-with-sitecore-7.html" target="_blank">blog post</a> that details how to setup Solr with Sitecore 7. One potential problem stood out though: Sitecore didn't play well with Solr 4.8.x or higher due to an assumption Sitecore made about Solr's schema.<br />
<br />
If you, like me, read this and had reason to hesitate then I say worry not! Sitecore has provided a solution that will allow you to deploy Solr 4.8 and later to your environment. In a nutshell, the fix is to modify the Solr's schema.xml file.<br />
<br />
From the <a href="https://kb.sitecore.net/articles/227897" target="_blank">kb article</a>:<br />
<br />
<blockquote class="tr_bq">
<ol style="-webkit-text-stroke-width: 0px; background: transparent; border: 0px; color: #606060; font-family: verdana, Arial, sans-serif; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; list-style: none; margin: 0px; orphans: auto; outline: 0px; padding: 0px 0px 0px 1em; text-align: left; text-indent: 0px; text-transform: none; vertical-align: baseline; white-space: normal; widows: auto; word-spacing: 0px;">
<li style="background: transparent; border: 0px; font-size: 12px; list-style: decimal; margin: 0px 0px 0.5em 1em; outline: 0px; padding: 0px; vertical-align: baseline;">Make the following changes in the default<span class="Apple-converted-space"> </span><b style="background: transparent; border: 0px; font-size: 12px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">schema.xml<span class="Apple-converted-space"> </span></b>file shipped with Solr:</li>
<ul style="background: transparent; border: 0px; font-size: 12px; list-style: none; margin: 0px; outline: 0px; padding: 0px 0px 0px 3em; vertical-align: baseline;">
<li style="background: transparent; border: 0px; font-size: 12px; list-style: disc; margin: 0px 0px 0.5em 1em; outline: 0px; padding: 0px; vertical-align: baseline;">enclose all <field> and <dynamicField> elements in the <fields> tag.</li>
<li style="background: transparent; border: 0px; font-size: 12px; list-style: disc; margin: 0px 0px 0.5em 1em; outline: 0px; padding: 0px; vertical-align: baseline;">enclose all <fieldType> elements in the <types> tag.</li>
</ul>
<li style="background: transparent; border: 0px; font-size: 12px; list-style: decimal; margin: 0px 0px 0.5em 1em; outline: 0px; padding: 0px; vertical-align: baseline;">Pass the modified<span class="Apple-converted-space"> </span><b style="background: transparent; border: 0px; font-size: 12px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">schema.xml<span class="Apple-converted-space"> </span></b>file to the<span class="Apple-converted-space"> </span><b style="background: transparent; border: 0px; font-size: 12px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Build Solr Schema Wizard</b><span class="Apple-converted-space"> </span>to add the Sitecore specific specific configuration.</li>
<li style="background: transparent; border: 0px; font-size: 12px; list-style: decimal; margin: 0px 0px 0.5em 1em; outline: 0px; padding: 0px; vertical-align: baseline;">Put the resulting file to the configuration folder of the Solr core.</li>
<li style="background: transparent; border: 0px; font-size: 12px; list-style: decimal; margin: 0px 0px 0.5em 1em; outline: 0px; padding: 0px; vertical-align: baseline;">If you use Solr 4.9 or later, ensure that the following field type is defined in the<span class="Apple-converted-space"> </span><b style="background: transparent; border: 0px; font-size: 12px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">schema.xml</b><span class="Apple-converted-space"> </span>file:</li>
<pre class="nocode prettyprint prettyprinted" style="background: rgb(238, 238, 238); border: none; font-size: 12px; margin: 10px; outline: 0px; overflow-x: auto; padding: 10px; vertical-align: baseline; width: auto;"><fieldType name="pint" class="solr.IntField"/></pre>
<li style="background: transparent; border: 0px; font-size: 12px; list-style: decimal; margin: 0px 0px 0.5em 1em; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="https://wiki.apache.org/solr/CoreAdmin#RELOAD" style="background: transparent; border: 0px; color: #003b77; font-size: 12px; margin: 0px; outline: none; padding: 0px; text-decoration: none; vertical-align: baseline; word-break: break-all;">Reload the core</a><span class="Apple-converted-space"> </span>to apply schema changes.</li>
</ol>
</blockquote>
Patrickhttp://www.blogger.com/profile/04029478911734299188noreply@blogger.com0