Sitecore and Solr – Installation errors on Sitecore 7.5

Today I’ve been setting up Sitecore and Solr using the instructions found on the SDN. When generating the schema file, you might get an error like the following:

unknown field ‘_uniqueid’

This is when you load the core admin screen to check that the default core ‘itembuckets’ has been loaded correctly.  If you read the comments in the generated file it
will tell you that this value should be set to ‘id’. Like so:

<uniqueKey>id</uniqueKey>

I’ve not done any Solr setup in quite a while so I’m still testing the installation. Hopefully this is correct and will save someone some time!

Lucinq Features – Date range searching

Pulling items by date or date range out of Lucene is quite a useful process. To do it accurately, we need to store the date values in a field that can be parsed appropriately. In Lucinq, our fluent Lucene wrapper, we currently achieve this by converting the DateTime value into an integer using the ticks property. This has a resolution down to 100ns which should be enough for anyones needs!

Using this, it allows us to create a range based search by comparing the ticks values between two dates. Then, for Lucene this is a simple integer comparison (taken directly from our test suite):

LuceneSearch luceneSearch = new LuceneSearch(IndexDirectory);

IQueryBuilder queryBuilder = new QueryBuilder();
DateTime february = DateTime.Parse("01/02/2013");
DateTime end = DateTime.Parse("28/02/2013");

queryBuilder.Setup(
	x => x.WildCard(BBCFields.Description, "food", Matches.Always),
	x => x.Filter(DateRangeFilter.Filter(BBCFields.PublishDateObject, february, end))
);

ILuceneSearchResult result = luceneSearch.Execute(queryBuilder);
List<NewsArticle> data = Mapper.Map<List<Document>, List<NewsArticle>>(result.GetTopItems());

This is similar to the way you’d think about doing it in SQL. Behind the scenes, SQL Server stores datetime values with very high precision.

Its worth noting that we’ve used the date based range as a filter in this example. We can also apply a date range directly and have it work like so:

LuceneSearch luceneSearch = new LuceneSearch(IndexDirectory);

IQueryBuilder queryBuilder = new QueryBuilder();
DateTime month = DateTime.Parse("01/02/2013");

queryBuilder.Setup(
	x => x.DateRange(BBCFields.PublishDateObject, month, month.AddDays(28)),
	x => x.WildCard(BBCFields.Description, "food")
);

ILuceneSearchResult result = luceneSearch.Execute(queryBuilder);
List<NewsArticle> data = Mapper.Map<List<Document>, List<NewsArticle>>(result.GetTopItems());

I think this is a good example of picking the correct field type for the job. By avoiding using a string we gain a lot more relevant operations without issues in casting the data. A case study for this would be a project I completed some time ago where cooking times needed to act as filters for a search. We were unable to achieve consistent results until the cooking time was stored as an integer value rather than a string. Which is something you would expect in code but potentially neglect in a search index.

Resources

Sitecore 7 Search – Searching By Inherited Template

After reading a great article this article from a former colleague (Richard Seal – Lead Sitecore Architect @ LightMaker US), I remembered that I to had solved this issue slightly differently some 12 – 18 months back using (originally Lucinq and then) the stock Sitecore 7 search. Non-the less, it fits a lot of what I have been saying in my Don’t Fight The Framework series – in particular – Please Don’t Base Your Code on Exact Templates, so I thought I would cover it again for those who don’t already know.

The problem is that Sitecore doesn’t ( / didn’t) provide a default way of searching for item’s based on their template inheritance hierarchy which means that normally in the out of the box solution you have to search on template (breaking everything I have said in my previous article above). This can be particularly useful for example if we wanted to return ‘All News Articles’ in our Site but you actually have implementations of ‘Press Release News Article’, ‘Industry News Article’ and ‘Budget News Article’

This article describes how I have achieved similar to Richard’s solution has done with Fortis (sorry Richard – I am a Glass fan 😉 ) with the stock Sitecore 7 Search.

I do have to note that I believe Sitecore (at the time I wrote this code originally) provided a similar field in it’s default configuration, but when I wrote this, it was broken (from memory, it only returned the first three levels or something).

Continue reading

Don’t Fight the Framework – Sitecore Tooling

Following on in this series, I found another topic I thought I would cover off from my travels. I guess the theme for this is that ‘just cause its the normal way, doesn’t mean it’s the ONLY way’ with a particular focus on choosing the right tooling from the options that Sitecore ships with. The following is an example of a consistent trend I have encountered and goes a little something like this.

Tree Traversal vs Lucene (or Lucinq or SC7 Search)
Firstly on this point, I will say this – Why are developers so scared of Lucene?? its an amazingly fast solution to a raft of problems and hyper fast to boot. People are quite quick to be jumping up and down about NoSql solutions (which I also love), but as developers you can solve similar problems with Lucene, it is in essence a form of unstructured data store and very mature.
Continue reading

Lucene – Support complex navigation using Collectors

Building a navigation system for a news or blog based on years and months (like an archive as an example)  requires that we know what years and months are available for:

  1. The top level – as in when we open the home page
  2. Expanded – whenever we are viewing a month or article page and need to expand accordingly
  3. Dynamically – when we need to build the navigation in one shot, perhaps as an accordion or similar.

Note – we are assuming the following structure:

Year/
    +-Month
    +-Month/
           Article A
           Article B
    +-Month
Year/  
Year/ 

Its possible to code around these by doing Axes calls on the current item and getting the parents and siblings but this doesn’t scale well. Especially if you need to start dropping into sibling months to hide and show depending if they’re empty or not. And we haven’t even talked about languages.

An alternative approach is to use your Lucene index and scan it for all your news items in the current language and group them by year and month and store counts for each. Then you can cache this for even more speed.
Continue reading

Searching Using Lucinq with Sitecore 7 default indexes

In this post I will be looking at using Lucinq to drive search using the default sitecore 7 indexing.

Lucinq provides the ability to use the generic forms of the standard Term, Phrase, Wildcard and other methods. This allows the assignment of an expression to denote which of the fields is the target of the query.

In the line below, you can see an example of this in action

queryBuilder.Term(t => t.Content, "value")

This tells Lucinq to read the IndexField attribute from the Content field of the SearchResultItem class and use that as the field for the Lucene query that is to be sent down.

* Note, the .HasBaseTemplate() method of Lucinq’s query extensions requires a change to the default Sitecore indexing settings. *

Here is an example in situe – this can be can in unit tests WITHOUT the need to run the sitecore initialise pipeline (unlike SC 7 search).

// the index lives in the appdata folder, so using a Server.MapPath to get the real path for Lucene.
string searchIndexPath = Server.MapPath(Settings.IndexFolder + @"/lucinqwebindex");
using (SitecoreSearch search = new SitecoreSearch(searchIndexPath))
{
     ISitecoreQueryBuilder queryBuilder = new SitecoreQueryBuilder(SitecoreMode.Sitecore7);
     queryBuilder.TemplateId(ContentIds.HomeTemplateId); // add template id to the query
     
     // below users the sitecore index field attribute, its the same as 
     // queryBuilder.Term("content", "value");
     queryBuilder.Term(t => t.Content, "value");// set up the term based on the in built sitecore 7 class.
     ISitecoreSearchResult searchResult = search.Execute(qb);
     int totalResults = searchResult.TotalHits; // gets the total number of results
     List items = searchResult.GetPagedItems(1, 10);
}

More information can be found on Lucinq on github here

https://github.com/cardinal252/Lucinq/wiki/

Searching with Lucinq and Glass Mapper

In this post I will cover using Lucinq with sitecore – in particular returning the results through the Glass Sitecore Mapper.

Returning results from lucene is something we as Sitecore developers have been used to for some time. Lucene provides hyper fast searching and its flexible storage makes it great for indexing content. However, in use, the API is somewhat verbose and not necessarily the easiest to understand without reading up on it first (I still would recommend the book Lucene In Action to get a grounding in the product).

I have a few issues with the search layers that Sitecore have implemented over the years which I won’t go into in this post.
Continue reading