Elasticsearch .NET (using NEST)

Among the many wonderful benefits of Elasticsearch is its scalability, and this is, of course, possible with the cool integrations you can do with it. In this third and last part of the tutorial series, you’ll be learning how to integrate a database and the plugins you may want to add to your ES installation.

Chapter Six: Database

As mentioned in Part 1 regarding Elasticsearch, it is not best practice to rely on ES as your database. So if your platform or system needs one, it is advisable that you set up a separate database for it. For the purposes of the demo project in this tutorial, you’ll set up a MySQL database.

MySQL Database

The design of the database for this demo project is DATABASE FIRST. So you’ll be implementing a reverse engineering migration.

1. To create a new table, simply add a table in the MySQL workbench and save the configured table.

2. After creating the table, simply paste the following command below in the package manager console. This would reverse engineer the migration and would auto-generate the context and models.

Scaffold-DbContext:
"server=localhost;port=3306;user=root;password=yourpassword;database=yourdatabase" MySQL.Data.EntityFrameworkCore -OutputDir Models -f

Note: Make sure you've set up your database and that it has data before you perform the migration. Check Elasticsearch .NET (using NEST) - Part 2 for this.

If you have successfully configured the database, then you will be able to add data items (person or employee) in your demo project and receive a notification indicating this.

Chapter Seven: Plugins

After configuring the database, the next thing to do is install the plugins you need for the demo project.

How to install on 7.5.1 on Windows

1. To install any plugin on version 7.5.1 in a Windows machine, simply navigate to the root folder of your Elasticsearch.

2. Once you are in the Elastic folder, simply type the following:

.\bin\elasticsearch-plugin.bat install NameOfThePlugin

After executing the command, it would display that it’s downloading and it will let you know once it is done.

How to install ICU ( International Components for Unicode ) ANALYSIS

The ICU plugin is a widely used library that provides support for different Unicodes (a unique number assigned to each character in every language). The plugin integrates the Lucene ICU module into Elasticsearch and extends the default Unicode library the module has. When integrated, it provides a better output when creating an analysis of Unicode libraries and languages.

Installing ICU Analysis plugin in Windows

1. There are two ways to install this plugin.

Download the zip file and extract it manually to the plugin folder.

Install any plugin on version 7.5.1 on a Windows machine, and simply navigate to the root folder of your elasticsearch.

2. Once you are in the elastic folder, simply type the following:
.\bin\elasticsearch-plugin.bat install analysis-icu

After executing the command it would display that it’s downloading and it will let you know once it is done.

Once the ICU analysis plugin has been installed, there are two ways of testing it. One is through Kibana and the other one is through the backend of your demo project.

ICU Analysis through Kibana

1. Open your Kibana Dev Tools in http://localhost:5601 and click the Tool icon (wrench).

2. Next, copy the code below.


    
        PUT /icu_collation_analyzer?pretty
        {
          "settings": {
            "analysis": {
              "filter": {
                "german_phone_book": {
                  "type":     "icu_collation",
                  "language": "de",
                  "country":  "DE",
                  "variant":  "@collation=phonebook"
                   }
                },
                "analyzer": {
                  "german_phone_book":{
                    "tokenizer": "keyword",
                      "filter": [ "german_phone_book" ]
                   }
                }
             }
           },
           "mappings": {
              "properties": {
                 "title": {
                   "type": "text",
                   "fields": {
                   "sort": {
                     "type":"text",
                     "fielddata": true,
                     "analyzer": "german_phone_book
                   }
                 }
               }
             }
          }
        }

The example will set up a phonebook containing all the German names. Always set fielddata:true so it would be sortable in the 7.5 elastic version. The default setting in Kibana is false due to performance issues in large amounts of data.

Note: Delete all existing indices and nodes, or restart it after installing the plugin.

Next, set up the index and add the plugin (ICU) as an analyzer. The name of the index is icu_collation_analyzer.

3. After we have created the Index, it's time to insert data (documents) into it.

4. Sort the data using collated sort:

Check the output. The order of the docs is going to be based on the German set phonebook collation.

ICU Analysis through NEST

NEST has a built-in ICU Analysis which can be seen here:

Elasticsearch-net / src / Nest / Analysis / Plugins / ICU

It will index and create tokens based on the configuration of your index. Try adding the special character below and try to search it using the “All” filter.

Special Character --> 你好

Definition of terms:

Poco - Plain old CLR object.
CLR - Common Language Runtime
REST - Representational State Transfer
JSON - Javascript Object Notation
ELK - Elasticsearch Logstash Kibana
DUCET - Default Unicode Collation Element Table
ICU - International Components for Unicode
NFD - Normalization Form Canonical Decomposition
NFC - Normalization Form Canonical Composition
NFKD - Normalization Form Compatilibitity Decomposition
NFKC - Normalization Form Compatibility Composition

If you have gotten to this part, then I congratulate you on learning the basics of Elasticsearch. Hope this tutorial helps you on your future projects. But if you want to review everything from the beginning, kindly refer to the Parts 1 & 2 of this tutorial.

Elasticsearch .NET (using NEST) - Part 3/3