Saturday, February 28, 2015

Sitecore 8 + Solr (part 3/3): Configur-ageddon

So far in this series we have installed Tomcat and Solr. Hopefully it has been a relatively easy process. Unfortunately, I can't promise today's post will be as easy. I've tried to balance making the configuration process as pain-free as possible, while not shielding anyone from the details that tripped me up. My goal is to take (all?) the guesswork out of this process. Let me know how I did!

[N.B.: As I was finishing up this final post I saw that update 2 for Sitecore was released. A quick perusal of the release notes leaves me feeling fairly confident that these instructions -- created with Sitecore 8 update 1 in mind -- should still be valid for update 2. No doubt I will soon need to upgrade my Sitecore environments to update 2 in the near future. Rest assured any issue(s) I encounter I will document here.]

Update (3-3-2015, 3-8-2015): As promised, I have updated this guide to comply with Sitecore 8 update 2. This means that if you want to upgrade from update 1, then you will need to create a couple of new cores and delete an old one (see step 6) and replace a DLL (see step 9). If you are starting fresh from update 2, then don't worry about any of this and dive right in!


  1. Stop the Tomcat service.
  2. Go to the root of your Solr instance, in my case, D:\solr. We need to modify the "collection1" directory to serve as one of the cores (folders with config files and index data) required by Sitecore.
    1. Rename the folder to "sitecore_analytics_index"
    2. Inside the folder you will find a file called “core.properties” which you may edit with a text editor. You need to change the “name” value to the name of your core. For example, when I create the “sitecore_analytics_index” core, then I will edit the “core.properties” file to have the value name=sitecore_analytics_index


  3. Start the Tomcat service and visit the Solr administration page. You may need to reload your browser page if it was already up. Click on the “Core Admin” menu item. If you modified the old “collection1” core correctly you should now see a sitecore_analytics_index core.


  4. Stop the Tomcat service. Now we need to fix the schema of our “sitecore_analytics_index” core, otherwise Sitecore cannot parse the xml correctly. Edit the file at \sitecore_analytics_index\conf\schema.xml according to Sitecore’s knowledge base article: https://kb.sitecore.net/articles/227897. Don’t forget to define the field type for pint since we are using a version of Solr later than 4.9! Start Tomcat and reload the Core Admin page.
  5. Next we must generate a new, Sitecore-specific schema. Sitecore provides a tool for this. Navigate to the Control Panel of your Sitecore instance. Look for the “Generate the Solr Schema.xml file” link and click it. Provide a path for the source and target files (they can’t be the same file.) Once you have generated your new schema, replace the old schema with it. Restart the Tomcat service and make sure the core loads correctly.
  6. A vanilla install of Sitecore 8 update 2 requires 13 cores to work correctly. So far we have one, but don’t despair, now that we have a generated a schema this process is much easier.  Essentially, we are going to use our sitecore_analytics_index core as a template to create the others. To do this:
    1. Copy the sitecore_analytics_index folder.
    2. Repeat steps 2a and 2b for each copy.
    3. When you are done, your Solr home folder should contain the following cores


    4. If you have done everything correctly you should be able to restart Tomcat and see all the cores listed above on the Core Admin page.
  7. Update (3-7-2015): I decided to create a PowerShell shortcut for this step. Save yourself time!

    Still with me? Hang in there, we are halfway home! Next, we must tell Sitecore to start using Solr instead of Lucene. This is done by appending or removing “disabled” as an extension of a configuration file’s name.
    1. Config files to DISABLE:

      \App_Config\Include\Sitecore.ContentSearch.Lucene.DefaultConfigurations.config.disabled
      \App_Config\Include\Sitecore.ContentSearch.Lucene.Index.Analytics.config.disabled
      \App_Config\Include\Sitecore.ContentSearch.Lucene.Index.Core.config.disabled
      \App_Config\Include\Sitecore.ContentSearch.Lucene.Index.Master.config.disabled
      \App_Config\Include\Sitecore.ContentSearch.Lucene.Index.Web.config.disabled
      \App_Config\Include\Sitecore.ContentSearch.Lucene.Indexes.Sharded.Core.config.example (left as is)
      \App_Config\Include\Sitecore.ContentSearch.Lucene.Indexes.Sharded.Master.config.example (left as is)
      \App_Config\Include\Sitecore.ContentSearch.Lucene.Indexes.Sharded.Web.config.example (left as is)
      \App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Lucene.Index.Master.config.disabled
      \App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Lucene.Index.Web.config.disabled
      \App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Lucene.IndexConfiguration.config.disabled
      \App_Config\Include\ContentTesting\Sitecore.ContentTesting.Lucene.IndexConfiguration.config.disabled
      \App_Config\Include\FXM\Sitecore.FXM.Lucene.Index.DomainsSearch.config.disabled
      \App_Config\Include\ListManagement\Sitecore.ListManagement.Lucene.Index.List.config.disabled
      \App_Config\Include\ListManagement\Sitecore.ListManagement.Lucene.IndexConfiguration.config.disabled
      \App_Config\Include\Social\Sitecore.Social.Lucene.Index.Master.config.disabled
      \App_Config\Include\Social\Sitecore.Social.Lucene.Index.Web.config.disabled
      \App_Config\Include\Social\Sitecore.Social.Lucene.IndexConfiguration.config.disabled


    2. Config files to ENABLE:

      \App_Config\Include\Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config
      \App_Config\Include\Sitecore.ContentSearch.Solr.Index.Analytics.config
      \App_Config\Include\Sitecore.ContentSearch.Solr.Index.Core.config
      \App_Config\Include\Sitecore.ContentSearch.Solr.Index.Master.config
      \App_Config\Include\Sitecore.ContentSearch.Solr.Index.Web.config
      \App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Solr.Index.Master.config
      \App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Solr.Index.Web.config
      \App_Config\Include\Sitecore.Marketing.Definitions.MarketingAssets.Repositories.Solr.IndexConfiguration.config
      \App_Config\Include\ContentTesting\Sitecore.ContentTesting.Solr.IndexConfiguration.config
      \App_Config\Include\FXM\Sitecore.FXM.Solr.Index.DomainsSearch.config
      \App_Config\Include\ListManagement\Sitecore.ListManagement.Solr.Index.List.config
      \App_Config\Include\ListManagement\Sitecore.ListManagement.Solr.IndexConfiguration.config
      \App_Config\Include\Social\Sitecore.Social.Solr.Index.Master.config
      \App_Config\Include\Social\Sitecore.Social.Solr.Index.Web.config
      \App_Config\Include\Social\Sitecore.Social.Solr.IndexConfiguration.config


  8. So I know that last step was pretty tedious, but if you've made it this far then the rest will be easier. Download the Solr Support Package from Sitecore and extract the contents of the zip file.
  9. Copy the following DLLs from the Solr Support Package into the \bin folder of your Sitecore Instance [N.B.: Castle Windsor is my Inversion of Control preference as Glass also uses it. Aside from Castle Windsor, Sitecore supports AutoFac, Ninject, StructureMap, and Unity.]:

    • Castle.Facilities.SolrNetIntegration.dll
    • Microsoft.Practices.ServiceLocation.dll
    • Sitecore.ContentSearch.Linq.Solr.dll
    • Sitecore.ContentSearch.SolrProvider.CastleWindsorIntegration.dll
    • Sitecore.ContentSearch.SolrProvider.dll
    • SolrNet.dll

      Update (3-3-2015): If you are upgrading from Sitecore 8 update 1, then you only need to replace Sitecore.ContentSearch.SolrProvider.XXXXXIntegration.dll with the latest version from the Solr Support Package. All other DLLs remain unchanged.

  10. Download the Nuget package for Castle Windsor. Unzip the package by renaming the extension from .nupkg to .zip and extracting its contents. Copy the Castle.Windsor.dll from \lib\net40-client to the \bin folder of your Sitecore instance.
  11. Repeat step 10 for the Castle.Core Nuget package. Copy the Castle.Core.dll from \lib\net40-client to the \bin folder of your Sitecore instance.
  12. Since we are going to use IoC, we need to make our Sitecore instance aware of it by replacing the Application directive in the global.asax file with the following:

    <%@Application Language='C#' Inherits="Sitecore.ContentSearch.SolrProvider.CastleWindsorIntegration.WindsorApplication" %>

  13. In order for Sitecore to talk to Solr, we need to give it a URL. This setting is maintained in the Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config file (remember, your address may differ from mine):

    <setting name="ContentSearch.Solr.ServiceBaseAddress" value="http://tomcat:8080/solr" />

  14. SOOO CLOSE! At this point, you are ready to test how badly you have broken your Sitecore instance! With any luck, when you browse to your Sitecore site you won't encounter any yellow screens of death. If you see a YSOD complaining about "Connection error to search provider [Solr] : Unable to connect to [http://tomcat:8080/solr]" then you are likely either missing a core or made typo when creating one. Assuming you are error free, the final step is to re-index. Go to the Control Panel and look for the "Indexing manager" link. Select all indexes and click the "Rebuild" button.
  15. Drink a beer; you deserve it!

11 comments:

  1. I'm stuck in #5.
    Status: Generating... Queued. for hours... :(

    ReplyDelete
  2. Hi Graciax8,

    Is analytics enabled but there is no connection to MongoDB? Here is a link to my schema file: https://gist.github.com/patrickperrone/669c36e40192913ec6f4

    ReplyDelete
    Replies
    1. Hi Patrick. I followed your instructions and file versions but we have different schema. You have additional fields that I dont have. Hmm..

      Anyways, I checked on the solr admin object browser and the schema has been updated. you can see now the fields, types and dynamicfields tags, but on the Sitecore "Generate the Solr Schema.xml file" it is still generating. :/

      Delete
    2. The right question would be how will I know if the core loads correctly?

      Delete
  3. Hey Graciax8,

    Had the same issue where sitecore seemed to stall when generating the schema. I closed all browsers and did an iisreset. Seemed to do the trick and generated the file in a few seconds afterwards. I would double check you file to make sure you have all of the field types and field/dynamic fields wrapped per the KB instructions in step 4. If you're still having issues, make sure your target schema location is somewhere that can be written to.

    Cheers.

    ReplyDelete
  4. Thanks Patrick this has been invaluable!

    For Sitecore 8.1 Update 1 - there are 2 new cores:

    sitecore_marketingdefinitions_master
    sitecore_marketingdefinitions_web

    And marketingdefiniions no longer exists.

    ReplyDelete
    Replies
    1. Yep, you are absolutely correct! I sorely need to revisit this topic for 8.1 + 5.x Solr. :)

      Delete
  5. After switching to Sitecore.ContentSearch.SolrProvider.CastleWindsorIntegration.WindsorApplication
    My application_start method is no longer called :(

    public class Application : WindsorApplication
    {
    protected new void Application_Start()
    {
    ...

    ReplyDelete
  6. This comment has been removed by the author.

    ReplyDelete
  7. Hi, I have completed all the step and after than when i rebuild the index its show me an error:

    Job started: Index_Update_IndexName=sitecore_master_index|#Exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> SolrNet.Exceptions.SolrConnectionException:

    400341org.apache.solr.common.SolrExceptionorg.apache.solr.common.SolrExceptionundefined field _indexname400

    ---> System.Net.WebException: The remote server returned an error: (400) Bad Request.
    at System.Net.HttpWebRequest.GetResponse()
    at HttpWebAdapters.Adapters.HttpWebRequestAdapter.GetResponse()
    at SolrNet.Impl.SolrConnection.GetResponse(IHttpWebRequest request)
    at SolrNet.Impl.SolrConnection.PostStream(String relativeUrl, String contentType, Stream content, IEnumerable`1 parameters)
    --- End of inner exception stack trace ---
    at SolrNet.Impl.SolrConnection.PostStream(String relativeUrl, String contentType, Stream content, IEnumerable`1 parameters)
    at SolrNet.Impl.SolrConnection.Post(String relativeUrl, String s)
    at SolrNet.Commands.DeleteCommand.Execute(ISolrConnection connection)
    at SolrNet.Impl.SolrBasicServer`1.SendAndParseHeader(ISolrCommand cmd)
    at Sitecore.ContentSearch.SolrProvider.SolrSearchIndex.Reset()
    at Sitecore.ContentSearch.SolrProvider.SolrSearchIndex.PerformRebuild(Boolean resetIndex, Boolean optimizeOnComplete, IndexingOptions indexingOptions, CancellationToken cancellationToken)
    at Sitecore.ContentSearch.SolrProvider.SolrSearchIndex.Rebuild(Boolean resetIndex, Boolean optimizeOnComplete)
    --- End of inner exception stack trace ---
    at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
    at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
    at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
    at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
    at (Object , Object[] )
    at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
    at Sitecore.Jobs.Job.ThreadEntry(Object state)


    could anybody face that type of problem, if yes and have a solution , plz provide me.

    Thanks,
    Pawan

    ReplyDelete
  8. FYI, for Sitecore 8.1 Update 2, you'll need to add 'Sitecore.Speak.ContentSearch.Solr.config.example' to the list of configs.

    ReplyDelete