Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Thomas 151 posts 326 karma points
    Mar 31, 2015 @ 11:30
    Thomas
    0

    Examine with Custom DB does not search Greek

    Hi,

    I am facing the following problem on a production environment. Recently i create a custom ISimpleDataService class which collects data from custom tables in database. I have also create the examine config settings as follows:

      <add name="CountriesCitiesIndexer"
         type="Examine.LuceneEngine.Providers.SimpleDataIndexer, Examine"
         dataService="UmbracoAddons.Infrastructure.Examine.CountriesCitiesService, UmbracoAddons"
         analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net"
         indexTypes="CustomData"
         indexSet="CountriesCitiesFinderIndexSet"
         runAsync="false"/>
    

    and

      <add name="CountriesCitiesSearcher"
           type="Examine.LuceneEngine.Providers.LuceneSearcher, Examine"
           analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net"
           indexSet="CountriesCitiesFinderIndexSet" />
    

    and the indexset is

    The problem that i am facing is when i add on the searcher tool a city or country name in english (for example London), then i am getting results, when i add the name in Greek i am not getting. I am sure that the Greek name exists because the ISimpleDataService returns the Greek names also from the database custom tables.

    Any help please? It is urgent as this is a production environment.

    Regards

    Thomas

  • Dave Woestenborghs 3325 posts 11170 karma points MVP 5x admin c-trib
    Mar 31, 2015 @ 13:51
    Dave Woestenborghs
    0

    Can you post your code for your dataservice ?

    dave

  • Thomas 151 posts 326 karma points
    Apr 01, 2015 @ 08:19
    Thomas
    0

    Hi, thank you for the reply. I found that the problem was in the searchCriteria and the compiled query. For some reason that i cannot understand using the following part of code it didn't work

    var searcher = ExamineManager.Instance.SearchProviderCollection["ArtTravelCountriesCitiesSearcher"]; var searchCriteria = searcher.CreateSearchCriteria(Examine.SearchCriteria.BooleanOperation.Or);

    var query = searchCriteria.GroupedOr(new string[] { "Name" }, searchTerm); or var query = searchCriteria.Field("Name", searchTerm);

    var searchResults = searcher.Search(query.Compile()).OrderByDescending(x => x.Score).TakeWhile(x => x.Score > 0.05f);

    but using the following code it works

    var searcher = ExamineManager.Instance.SearchProviderCollection["ArtTravelCountriesCitiesSearcher"]; var searchResults = searcher.Search(searchTerm, true).OrderByDescending(x => x.Score);

  • Dave Woestenborghs 3325 posts 11170 karma points MVP 5x admin c-trib
    Apr 01, 2015 @ 08:41
    Dave Woestenborghs
    0

    Glad to see you got it solved. But still interested in the dataservice code. Maybe this can come in useful for myself at some point.

    Dave

  • Thomas 151 posts 326 karma points
    Apr 01, 2015 @ 08:45
    Thomas
    0

    Following is the source from the DataService

    public class ArtTravelCountriesCitiesService : ISimpleDataService { public IEnumerable

            var dataSets = new List<SimpleDataSet>();
            var i = 1; //unique id for each doc
    
            foreach (var item in items)
            {
                try
                {
                    var simpleDataSet = new SimpleDataSet { NodeDefinition = new IndexedNode(), RowData = new Dictionary<string, string>() };
    
                    simpleDataSet = MapCountryCityItemToSimpleDataIndexItem(item, simpleDataSet, i, indexType);
    
                    dataSets.Add(simpleDataSet);
                }
                catch (Exception ex)
                {
                    Log.Add(LogTypes.Error, i, "error processing item " + item.CityCountryName + " " + ex);
                }
    
                i++;
            }
    
            return dataSets;
        }
    
        private SimpleDataSet MapCountryCityItemToSimpleDataIndexItem(SearchCountriesCitiesListModel item, SimpleDataSet simpleDataSet, int index, string indexType)
        {
            simpleDataSet.NodeDefinition.NodeId = index;
            simpleDataSet.NodeDefinition.Type = indexType;
            simpleDataSet.RowData.Add("Id", item.Id.ToString());
            simpleDataSet.RowData.Add("CityCountryId", item.CityCountryId.ToString());
            simpleDataSet.RowData.Add("CityCountryUrl", item.CityCountryUrl.ToString());
            simpleDataSet.RowData.Add("CityCountryName", RemoveAccent(item.CityCountryName, true));
            simpleDataSet.RowData.Add("IsCountry", item.isCountry.ToString());
            simpleDataSet.RowData.Add("IsCity", item.isCity.ToString());
    
            return simpleDataSet;
        }
    
        char[] accentsArray = { 'ά', 'έ', 'ί', 'ό', 'ύ', 'ή', 'ς', 'ώ', 'Ά', 'Έ', 'Ί', 'Ό', 'Ύ', 'Ή', 'Ώ' };
        char[] replacementArray = { 'α', 'ε', 'ι', 'ο', 'υ', 'η', 'σ', 'ω', 'Α', 'Ε', 'Ι', 'Ο', 'Υ', 'Η', 'Ω' };
        private string RemoveAccent(string text, bool upper = false)
        {
            if (upper)
            {
                return RemoveAccent(text).ToUpperInvariant();
            }
            return RemoveAccent(text).ToLowerInvariant();
        }
    
        private string RemoveAccent(string accentedStr)
        {
            char[] replacement = replacementArray;
            char[] accents = accentsArray;
            string temp = new string(replacement).ToUpper();
            char[] upperReplacement = temp.ToCharArray();
            temp = new string(accents).ToUpper();
            char[] upperAccents = temp.ToCharArray();
    
            StringBuilder returnString = new StringBuilder();
    
            if (accents != null && replacement != null && accentedStr.IndexOfAny(accents) > -1)
            {
                returnString.Length = 0;
                returnString.Append(accentedStr);
                for (int i = 0; i < accents.Length; i++)
                {
                    returnString.Replace(accents[i], replacement[i]);
                }
    
                return returnString.ToString();
            }
            else if (accents != null && replacement != null && accentedStr.IndexOfAny(upperAccents) > -1)
            {
                returnString.Length = 0;
                returnString.Append(accentedStr);
                for (int i = 0; i < upperAccents.Length; i++)
                {
                    returnString.Replace(upperAccents[i], upperReplacement[i]);
                }
    
                return returnString.ToString();
            }
            else
                return accentedStr;
        }
    }
    
  • This forum is in read-only mode while we transition to the new forum.

    You can continue this topic on the new forum by tapping the "Continue discussion" link below.

Please Sign in or register to post replies