Content (Un)Availability: Missing items in content search in Sitecore 9.1 update 1 (9.1.1)


Sitecore Content Availability Calendar



Even after reading tons of documents, blogs and carefully following the installation guides, it is possible to commit human errors. Through this blog, I would like to share an experience from one of the careless oversight that came back and bit us.

Scenario: 

  • We have configured Publishing Service 4 with Sitecore 9.1 update 1.
  • Followed all the installation steps (well mostly, otherwise no point to writing this blog right?).
  • Published content and as the title suggests, couldn't find it using Content Search on Sitecore_Web_Index.

What we verified:

Before I dive into the content of this blog, here are the questions we answered to cover the basic troubleshoot from our past leanings

  1. Is content published properly (completely and successfully)?
  2. Is cache cleared in CD servers?
  3. Is item available in web database configured to CD roles?
  4. Is re-indexing/ crawl completed on CD servers (If applicable)?

For all of the above questions, the answer is yes. So, then what is stopping us from seeing the content??
So, we asked ourselves one more question. 

Are we able to see the content using master index? And the answer was yes. 

This was the ray of hope we needed. Knowing search is not broken, we dig little deeper into our investigation. Our codebase is same on CM and CD. Content search configurations are identical. So, we checked Sitecore logs to identify any anomaly patterns. And to our luck, we quickly found 1, log generated for Solr query.

When we use ContentSearch API and generate the Solr request, the request query is logged into Sitecore log files. These are the logs generated for us on CM and CD

On CD:

2019-08-12T03:51:37  PID[11252] Information 10692 03:51:37 INFO  Solr Query - ?q=((((versionsunrisedate_s:[* TO 2019-08-12T03:51:37.361Z] AND versionsunsetdate_s:{2019-08-12T03:51:37.361Z TO *]) AND publishablefrom_s:[* TO 2019-08-12T03:51:37.361Z]) AND publishableto_s:{2019-08-12T03:51:37.361Z TO *]) AND (_content:(abc) AND (_template:(ea3ab59e0fb54a6699a521ebe499b0a2) OR _template:(3cba1536cbef4ea2a6632e1d41b3c618) OR _template:(cd6d939c907c402ba8a120bc27b8e037) OR _template:(ace57d85288d40b38883ba70b98a69a0) OR _template:(cf9ca02f2f9d4a6399fd8fdb756d888c) OR _template:(46663e05a6b8422a8e1336cd2b041278) OR _template:(389f874cccd64d9682c9e57bd4e9b915) OR _template:(6de771d153c04793b42aca1d8d678525) OR _template:(aebe3399d9b94adc9fea573185650569) OR _template:(fb0892ca098647bf998ecba0c47bb3cb) OR _template:(ace57d85288d40b38883ba70b98a69a0))))&rows=1000000&fq=(_fullpath:(*) AND _language:(en))&fq=_indexname:(sitecore_web_index)&wt=xml&sort=__smallcreateddate_tdt desc

On CM:

2019-08-12T03:53:14  PID[10372] Information 14992 03:53:14 INFO  Solr Query - ?q=(_content:(abc) AND (_template:(ea3ab59e0fb54a6699a521ebe499b0a2) OR _template:(3cba1536cbef4ea2a6632e1d41b3c618) OR _template:(cd6d939c907c402ba8a120bc27b8e037) OR _template:(ace57d85288d40b38883ba70b98a69a0) OR _template:(cf9ca02f2f9d4a6399fd8fdb756d888c) OR _template:(46663e05a6b8422a8e1336cd2b041278) OR _template:(389f874cccd64d9682c9e57bd4e9b915) OR _template:(6de771d153c04793b42aca1d8d678525) OR _template:(aebe3399d9b94adc9fea573185650569) OR _template:(fb0892ca098647bf998ecba0c47bb3cb) OR _template:(ace57d85288d40b38883ba70b98a69a0)))&rows=1000000&fq=(_fullpath:(*) AND _language:(en))&fq=_indexname:(sitecore_web_index)&wt=xml&sort=__smallcreateddate_tdt desc

If you focus on the highlighted part, there are few fields like versionsunrisedate_s, versionsunsetdate_s, publishablefrom_s, publishableto_s that are added to query in an 'AND' clause. Fundamentally, these are introduced into Sitecore due to installation of publishing service module.

Unfortunately, there isn't a ton of content for Publishing Service on Sitecore document site

But, you can read about the release notes and download installation guide if you are certified Sitecore developer from  https://dev.sitecore.net/Downloads/Sitecore_Publishing_Service/41/Sitecore_Publishing_Service_410.aspx

Root Cause:

We used SolrCloud in our application and on our CM role, we have noticed that on CM role "Sitecore.Publishing.Service.ContentAvailability.solr.config" is disabled. 

We reviewed this file closely and quickly noticed that we have to enable this file as it is the source for adding above missing fields


Knowing we must enable the file, we quickly renamed the file on Azure AppService Editor.



With these fields added, we did a quick publish of content again and re-indexed the sitecore_web_index and this time all the  4 computed fields are added to the search index. Voila!! we started seeing the content on Content Delivery.

After understanding the root-cause, we spent some more time to understand the content availability feature. There is a very good blog post about it here by Bo Breitinghttps://thegrumpycoder.com/post/167975001471/sitecore-publishing-server-content-availability


From Content Availability Section of Sitecore Publishing Service Installation Guide:

The content availability feature ensures that the valid version of an item is always available in the target database at the time of publishing. In this way, you do not have to perform a publishing operation every time an item version expires and the next version should be displayed.

When you enable content availability and publish an item, the currently valid item version and all the versions that are valid for future publishing are moved from the source database to the target database. A new pipeline that is enabled in the content availability configuration file automatically clears the Sitecore item cache when an item version expires and then, when a contact accesses the item, the next valid version is displayed.

Important If you are using HTML caching on a rendering, the Sitecore item cache does not automatically clear. In addition, if you use a data source inside a rendering, and if the data source item switches to display a new version, the hosting rendering is not updated because there is nothing that indicates that an update of a

Additional Reading:

  1. http://www.sugnl.net/~/media/SUGNL/Meetings/SUGNL%20Meeting%20May%202019%20Virtual%20Affairs/Downloads/SUGNL%20meetup%2020190515%20%20Ivan%20Lieckens%20%20Sitecore%20Publishing%20Strategies.pdf
Hope this blog helps others to solve content unavailability issue. Please comment, if you like the blog or to share your experiences. And do let me know if you would like me to cover certain topic in Sitecore for you.

Happy Problem Solving!!!

Comments