On Second(ary) Thought...

Recently I posted my thoughts regarding the proper ratio of Solr cores to Sitecore indexes. In it, I mentioned the need to double the number of cores to support the SwitchOnRebuildSolrSearchIndex feature. It turns out, this isn't quite right. Creating a secondary core for the analytics index does no good and should be avoided.

You'll want to avoid a secondary core for the analytics index because trying to use one results in exception. The analytics index has an extra "group" parameter, and Sitecore cannot find a matching constructor. The SolrSearchIndex class allows for the group parameter but the SwitchOnRebuildSolrSearchIndex does not.

Why is this? As Adam Conn explains, the types of crawlers responsible for maintaining the analytics index are observers of data. This means that the crawler is notified when data is available and then passed that data. Therefore, rebuilding the analytics index would result in an empty index unless you rebuild the reporting database so that the analytics crawlers may observe (and index) the data. This is the reason why the UI doesn't provide the option of rebuilding the analytics index.

Read More

A Script to Swap Search in Sitecore

A colleague of mine had the very good idea of saving some time when switching Sitecore between Lucene and Solr (and back again) with a console program. I decided to rewrite it in PowerShell. Help yourselves!

Read More

A Solr Core-nucopia?

[N.B.: If you haven't already, check out my series of posts (1, 2, and 3) that walk through installing Solr for Sitecore.]

Given that there isn't yet a search scaling guide for Sitecore 8.0, the current, best authoritative source of guidance is the Sitecore Search Scaling Guide for 7.5. There is an interesting line in the guide that recommends creating "separate cores for each Sitecore index." This was necessary to avoid inconsistent and unexpected results.

While digging through the release notes for Sitecore 8 update 1 I found a good, technical description of the risk:

When two indexes were configured to use the same SOLR core, it was impossible to differentiate the index data between the indexes. As a result, index data related in one index would override the index data in the other index. This has been fixed so that the _uniqueid index field value has been extended with information about the index name. (426743)

Out of curiosity I decided to validate the fix. Here is a query from one of my Solr cores connected to a Sitecore 8 update 2 instance (a big thumbs-up to Solr's built-in admin tool!)

Let's breakdown the taxonomy of this _uniqueid:

1. 
   sitecore://<database name>/<item id>?lang=<language name>&ver=<version number>&ndx=<index name>
   

Clearly, the index name is now a part of the key! Does this mean you can disregard the advice in the Search Scaling Guide? In a word, yes. A more important question is, "Should you?" Well, probably not. Here are some reasons why:

1. Any single Sitecore index (Solr core) rebuild is less expensive since there is less data. Thus, the rebuild is quicker.
2. When reviewing statistics about a core in Solr's core admin, facts about the core such as the number of documents easily translate to facts about the Sitecore index.
3. Probably most important of all, it's possible to tune the cache and core's settings as necessary per Sitecore index. Undoubtedly, usage patterns will vary per Sitecore index. So should the strategies you implement to tune the Solr core responsible for that Sitecore index.

Update (3-8-2015): In Sitecore update 2, one Solr core was removed and two were added. I updated the paragraph below to reflect this.

Keep in mind, these advantages do come at a cost. There will be some amount of overhead incurred per core. Also, there is the management headache of maintaining many cores. As of Sitecore 8 update 2 there are 13 indexes for a vanilla install. If you want to take advantage of the SwitchOnRebuildSolrSearchIndex feature (while an index rebuilds, Sitecore can still return search results for that index) then you will need to add an additional core for each Sitecore index that uses this feature. That is a possible 26 25 (see here for an explanation why the total number changed) cores to manage!

I'm interested in other people's opinions on this topic. Let me know what you all think.

Read More

Sitecore 8 + Solr (part 3/3): Configur-ageddon

So far in this series we have installed Tomcat and Solr. Hopefully it has been a relatively easy process. Unfortunately, I can't promise today's post will be as easy. I've tried to balance making the configuration process as pain-free as possible, while not shielding anyone from the details that tripped me up. My goal is to take (all?) the guesswork out of this process. Let me know how I did!

[N.B.: As I was finishing up this final post I saw that update 2 for Sitecore was released. A quick perusal of the release notes leaves me feeling fairly confident that these instructions -- created with Sitecore 8 update 1 in mind -- should still be valid for update 2. No doubt I will soon need to upgrade my Sitecore environments to update 2 in the near future. Rest assured any issue(s) I encounter I will document here.]

Update (3-3-2015, 3-8-2015): As promised, I have updated this guide to comply with Sitecore 8 update 2. This means that if you want to upgrade from update 1, then you will need to create a couple of new cores and delete an old one (see step 6) and replace a DLL (see step 9). If you are starting fresh from update 2, then don't worry about any of this and dive right in!

1. Stop the Tomcat service.
2. Go to the root of your Solr instance, in my case, D:\solr. We need to modify the "collection1" directory to serve as one of the cores (folders with config files and index data) required by Sitecore.
1. Rename the folder to "sitecore_analytics_index"
2. Inside the folder you will find a file called “core.properties” which you may edit with a text editor. You need to change the “name” value to the name of your core. For example, when I create the “sitecore_analytics_index” core, then I will edit the “core.properties” file to have the value name=sitecore_analytics_index
   
   
   
   
   
3. Start the Tomcat service and visit the Solr administration page. You may need to reload your browser page if it was already up. Click on the “Core Admin” menu item. If you modified the old “collection1” core correctly you should now see a sitecore_analytics_index core.
   
   
   
   
   
4. Stop the Tomcat service. Now we need to fix the schema of our “sitecore_analytics_index” core, otherwise Sitecore cannot parse the xml correctly. Edit the file at \sitecore_analytics_index\conf\schema.xml according to Sitecore’s knowledge base article: https://kb.sitecore.net/articles/227897. Don’t forget to define the field type for pint since we are using a version of Solr later than 4.9! Start Tomcat and reload the Core Admin page.
5. Next we must generate a new, Sitecore-specific schema. Sitecore provides a tool for this. Navigate to the Control Panel of your Sitecore instance. Look for the “Generate the Solr Schema.xml file” link and click it. Provide a path for the source and target files (they can’t be the same file.) Once you have generated your new schema, replace the old schema with it. Restart the Tomcat service and make sure the core loads correctly.
6. A vanilla install of Sitecore 8 update 2 requires 13 cores to work correctly. So far we have one, but don’t despair, now that we have a generated a schema this process is much easier.  Essentially, we are going to use our sitecore_analytics_index core as a template to create the others. To do this:
1. Copy the sitecore_analytics_index folder.
2. Repeat steps 2a and 2b for each copy.
3. When you are done, your Solr home folder should contain the following cores
   
   
   
   
   
4. If you have done everything correctly you should be able to restart Tomcat and see all the cores listed above on the Core Admin page.
7. Update (3-7-2015): I decided to create a PowerShell shortcut for this step. Save yourself time!
   
   Still with me? Hang in there, we are halfway home! Next, we must tell Sitecore to start using Solr instead of Lucene. This is done by appending or removing “disabled” as an extension of a configuration file’s name.
1. Config files to DISABLE:
   
   \App_Config\Include\Sitecore.ContentSearch.Lucene.DefaultConfigurations.config.disabled
   \App_Config\Include\Sitecore.ContentSearch.Lucene.Index.Analytics.config.disabled
   \App_Config\Include\Sitecore.ContentSearch.Lucene.Index.Core.config.disabled
   \App_Config\Include\Sitecore.ContentSearch.Lucene.Index.Master.config.disabled
   \App_Config\Include\Sitecore.ContentSearch.Lucene.Index.Web.config.disabled
   \App_Config\Include\Sitecore.ContentSearch.Lucene.Indexes.Sharded.Core.config.example (left as is)
   \App_Config\Include\Sitecore.ContentSearch.Lucene.Indexes.Sharded.Master.config.example (left as is)
   \App_Config\Include\Sitecore.ContentSearch.Lucene.Indexes.Sharded.Web.config.example (left as is)
   \App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Lucene.Index.Master.config.disabled
   \App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Lucene.Index.Web.config.disabled
   \App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Lucene.IndexConfiguration.config.disabled
   \App_Config\Include\ContentTesting\Sitecore.ContentTesting.Lucene.IndexConfiguration.config.disabled
   \App_Config\Include\FXM\Sitecore.FXM.Lucene.Index.DomainsSearch.config.disabled
   \App_Config\Include\ListManagement\Sitecore.ListManagement.Lucene.Index.List.config.disabled
   \App_Config\Include\ListManagement\Sitecore.ListManagement.Lucene.IndexConfiguration.config.disabled
   \App_Config\Include\Social\Sitecore.Social.Lucene.Index.Master.config.disabled
   \App_Config\Include\Social\Sitecore.Social.Lucene.Index.Web.config.disabled
   \App_Config\Include\Social\Sitecore.Social.Lucene.IndexConfiguration.config.disabled
   
   
2. Config files to ENABLE:
   
   \App_Config\Include\Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config
   \App_Config\Include\Sitecore.ContentSearch.Solr.Index.Analytics.config
   \App_Config\Include\Sitecore.ContentSearch.Solr.Index.Core.config
   \App_Config\Include\Sitecore.ContentSearch.Solr.Index.Master.config
   \App_Config\Include\Sitecore.ContentSearch.Solr.Index.Web.config
   \App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Solr.Index.Master.config
   \App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Solr.Index.Web.config
   \App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Solr.IndexConfiguration.config
   \App_Config\Include\ContentTesting\Sitecore.ContentTesting.Solr.IndexConfiguration.config
   \App_Config\Include\FXM\Sitecore.FXM.Solr.Index.DomainsSearch.config
   \App_Config\Include\ListManagement\Sitecore.ListManagement.Solr.Index.List.config
   \App_Config\Include\ListManagement\Sitecore.ListManagement.Solr.IndexConfiguration.config
   \App_Config\Include\Social\Sitecore.Social.Solr.Index.Master.config
   \App_Config\Include\Social\Sitecore.Social.Solr.Index.Web.config
   \App_Config\Include\Social\Sitecore.Social.Solr.IndexConfiguration.config
   
   
8. So I know that last step was pretty tedious, but if you've made it this far then the rest will be easier. Download the Solr Support Package from Sitecore and extract the contents of the zip file.
9. Copy the following DLLs from the Solr Support Package into the \bin folder of your Sitecore Instance [N.B.: Castle Windsor is my Inversion of Control preference as Glass also uses it. Aside from Castle Windsor, Sitecore supports AutoFac, Ninject, StructureMap, and Unity.]:
   
   
• Castle.Facilities.SolrNetIntegration.dll
• Microsoft.Practices.ServiceLocation.dll
• Sitecore.ContentSearch.Linq.Solr.dll
• Sitecore.ContentSearch.SolrProvider.CastleWindsorIntegration.dll
• Sitecore.ContentSearch.SolrProvider.dll
• SolrNet.dll
  
  Update (3-3-2015): If you are upgrading from Sitecore 8 update 1, then you only need to replace Sitecore.ContentSearch.SolrProvider.XXXXXIntegration.dll with the latest version from the Solr Support Package. All other DLLs remain unchanged.


10. Download the Nuget package for Castle Windsor. Unzip the package by renaming the extension from .nupkg to .zip and extracting its contents. Copy the Castle.Windsor.dll from \lib\net40-client to the \bin folder of your Sitecore instance.
11. Repeat step 10 for the Castle.Core Nuget package. Copy the Castle.Core.dll from \lib\net40-client to the \bin folder of your Sitecore instance.
12. Since we are going to use IoC, we need to make our Sitecore instance aware of it by replacing the Application directive in the global.asax file with the following:
    
    <%@Application Language='C#' Inherits="Sitecore.ContentSearch.SolrProvider.CastleWindsorIntegration.WindsorApplication" %>
    
    
13. In order for Sitecore to talk to Solr, we need to give it a URL. This setting is maintained in the Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config file (remember, your address may differ from mine):
    
    <setting name="ContentSearch.Solr.ServiceBaseAddress" value="http://tomcat:8080/solr" />
    
    
14. SOOO CLOSE! At this point, you are ready to test how badly you have broken your Sitecore instance! With any luck, when you browse to your Sitecore site you won't encounter any yellow screens of death. If you see a YSOD complaining about "Connection error to search provider [Solr] : Unable to connect to [http://tomcat:8080/solr]" then you are likely either missing a core or made typo when creating one. Assuming you are error free, the final step is to re-index. Go to the Control Panel and look for the "Indexing manager" link. Select all indexes and click the "Rebuild" button.
15. Drink a beer; you deserve it!

Read More

Solr 4.8 and Higher with Sitecore - Schema Issue Resolved

Dan Solovay has a great blog post that details how to setup Solr with Sitecore 7. One potential problem stood out though: Sitecore didn't play well with Solr 4.8.x or higher due to an assumption Sitecore made about Solr's schema.

If you, like me, read this and had reason to hesitate then I say worry not! Sitecore has provided a solution that will allow you to deploy Solr 4.8 and later to your environment. In a nutshell, the fix is to modify the Solr's schema.xml file.

From the kb article:

1. Make the following changes in the default schema.xml file shipped with Solr:
• enclose all <field> and <dynamicField> elements in the <fields> tag.
• enclose all <fieldType> elements in the <types> tag.
2. Pass the modified schema.xml file to the Build Solr Schema Wizard to add the Sitecore specific specific configuration.
3. Put the resulting file to the configuration folder of the Solr core.
4. If you use Solr 4.9 or later, ensure that the following field type is defined in the schema.xml file:

<fieldType name="pint" class="solr.IntField"/>

5. Reload the core to apply schema changes.

Read More