3 Developing Search Applications

3.1 Task: Highlight the search terms in the response of a query

Example 1: Creating search queries w/highlighting for blog posts

Requirements

Perform a search query which highlights the search term “elasticsearch”

Steps

Open the Kibana Console or Use a REST Client

Create and populate the Index

POST /blog_posts/_bulk
{ "index": { "_id": "1" } }
{ "title": "Introduction to Elasticsearch", "content": "Elasticsearch is a powerful search engine." }
{ "index": { "_id": "2" } }
{ "title": "Advanced Elasticsearch Techniques", "content": "This guide covers advanced features of Elasticsearch." }
{ "index": { "_id": "3" } }
{ "title": "Elasticsearch Performance Tuning", "content": "Learn how to optimize Elasticsearch for better performance." }

Create a search query using the highlight clause (the field being searched must match the field to be highlighted)

GET /blog_posts/_search
{
  "query": {
    "match": {
      "content": "elasticsearch"
    }
  },
  "highlight": {
    "fields": {
      "content": {}
    }
  }
}

Test

Confirm the index exists
```
GET /blog_posts
```

Execute the query and confirm that the content field has highlighting

{
  ...
  "hits": {
    "hits": [
      {
        "_id": "1",
        "_source": {
          "title": "Introduction to Elasticsearch",
          "content": "Elasticsearch is a powerful search engine."
        },
        "highlight": {
          "content": [
            "<em>Elasticsearch</em> is a powerful search engine."
          ]
        }
      }
      // Additional documents...
    ]
  }
}

Considerations

Field Selection: The highlight field in the search request specifies which fields to highlight. In this example, we highlight the content field.
Performance: Highlighting can impact search performance, especially on large datasets. It is essential to balance the need for highlighting with performance considerations.

Clean-up (optional)

Delete the index
```
DELETE blog_posts
```

Documentation

Example 2: Creating search queries w/highlighting for customer order data

Requirements

An orders index with documents containing customer order information including customer_name, order_date, products, total_price.
A search query to retrieve orders
- Search for Product A and a range for price called total_price
- Highlight the search terms in the products nested object

Steps

Open the Kibana Console or use a REST client.

Create the orders index by indexing some documents

POST /orders/_bulk
{ "index": { "_id": "1" } }
{ "customer_name": "John Doe", "order_date": "2022-01-01", "products": [{ "name": "Product A", "price": 10.99 },{ "name": "Product B", "price": 5.99 }], "total_price": 16.98 }
{ "index": { "_id": "2" } }
{ "customer_name": "Jane Smith", "order_date": "2022-01-15", "products": [{ "name": "Product B", "price": 5.99 },{ "name": "Product C", "price": 7.99 }], "total_price": 13.98 }
{ "index": { "_id": "3" } }
{ "customer_name": "Bob Johnson", "order_date": "2022-02-01", "products": [{ "name": "Product A", "price": 10.99 },{ "name": "Product C", "price": 7.99 }], "total_price": 18.98 }

Execute a search query with highlighting including custom pre_tags and post_tags

GET orders/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match_phrase": {
            "products.name": "product a"
          }
        },
        {
          "range": {
            "total_price": {
              "gt": 10
            }
          }
        }
      ]
    }
  },
  "highlight": {
    "fields": {
      "products.name": {
        "pre_tags": [
          "<b>"
        ],
        "post_tags": [
          "</b>"
        ]
      }
    }
  }
}

Test

Confirm the index exists
```
GET /orders
```

Execute the query and confirm that products.name has highlighting

{
  ...
    "hits": [
      {
        "_index": "orders",
        "_id": "1",
        "_score": 1.603535,
        "_source": {
          "customer_name": "John Doe",
          "order_date": "2022-01-01",
          "products": [
            {
              "name": "Product A", "price": 10.99
            },
            {
              "name": "Product B", "price": 5.99
            }
          ],
          "total_price": 16.98
        },
        "highlight": {
          "products.name": [
            "<b>Product A</b>"
          ]
        }
      },
      {
        "_index": "orders",
        "_id": "3",
        "_score": 1.603535,
        "_source": {
          "customer_name": "Bob Johnson",
          "order_date": "2022-02-01",
          "products": [
            {
              "name": "Product A", "price": 10.99
            },
            {
              "name": "Product C", "price": 7.99
            }
          ],
          "total_price": 18.98
        },
        "highlight": {
          "products.name": [
            "<b>Product A</b>"
          ]
        }
      }
    ]
  }
}

Considerations

Highlighting is used to emphasize the search terms in the response, making it easier to see why a document matched the query.
The highlight section in the search query specifies which fields to highlight and how to format the highlighted text.
Nested objects (products) are highlighted using the fields section with dot notation (products.name, products.price).

Clean-up (optional)

Delete the index
```
DELETE orders
```

Documentation

3.2 Task: Sort the results of a query by a given set of requirements

Example 1: Creating Search Queries w/ Sorting for e-commerce products

Requirements

Search for e-commerce product data in an index named products.
Sort the results by two criteria:
- Primary Sort: In descending order by product price (highest to lowest).
- Secondary Sort: In ascending order by product name (alphabetically).

Steps

Open the Kibana Console or use a REST client.

Create the products index

PUT products
{
  "mappings": {
    "properties": {
      "name": {
        "type": "keyword"
      },
      "price": {
        "type": "float"
      }
    }
  }
}

Index some documents

PUT /products/_bulk
{"index":{},"action":"index","_id":"1"}
{"name":"Headphones","price":79.99}
{"index":{},"action":"index","_id":"2"}
{"name":"Smartwatch","price":249.99}
{"index":{},"action":"index","_id":"3"}
{"name":"Laptop","price":1299.99}
{"index":{},"action":"index","_id":"4"}
{"name":"Wireless Speaker","price":99.99}

Define a query to match_all and then perform the primary sort of price highest to lowest.

GET /products/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "price": {
        "order": "desc"
      }
    }
  ]
}

Define a query to perform the secondary sort of name in alphabetical order (asc).

GET /products/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "name": {
        "order": "asc"
      }
    }
  ]
}

Combine the two sorts and their impact on the results (try the sort with name first and price second and see how the results change)

GET /products/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "price": {
        "order": "desc"
      }
    },
    {
      "name": {
        "order": "asc"
      }
    }
  ]
}

Test

Confirm the index exists
```
GET /products
```

Run the search queries and examine the response

{
  ...
    "hits": [
      {
        "_index": "products",
        "_id": "nXpN-pABRRh1FLFi7Ks8",
        "_score": null,
        "_source": {
          "name": "Laptop",
          "price": 1299.99
        },
        "sort": [
          1299.99,
          "Laptop"
        ]
      },
      {
        "_index": "products",
        "_id": "nHpN-pABRRh1FLFi7Ks8",
        "_score": null,
        "_source": {
          "name": "Smartwatch",
          "price": 249.99
        },
        "sort": [
          249.99,
          "Smartwatch"
        ]
      },
      {
        "_index": "products",
        "_id": "nnpN-pABRRh1FLFi7Ks8",
        "_score": null,
        "_source": {
          "name": "Wireless Speaker",
          "price": 99.99
        },
        "sort": [
          99.99,
          "Wireless Speaker"
        ]
      },
      {
        "_index": "products",
        "_id": "m3pN-pABRRh1FLFi7Ks8",
        "_score": null,
        "_source": {
          "name": "Headphones",
          "price": 79.99
        },
        "sort": [
          79.99,
          "Headphones"
        ]
      }
    ]
  }
}

Considerations

The sort clause defines the sorting criteria.
An array of sort definitions is specified, prioritizing them from top to bottom.
In this example, price is sorted in descending order (desc), while name is sorted in ascending order (asc).

Clean-up (optional)

Delete the index

DELETE products

Documentation

Sort Search Results

3.3 Task: Implement pagination of the results of a search query

There is only one example here as pagination is rather simple with very few configuration options.

Example 1: Creating pagination queries for an e-commerce product catalog

Requirements

An index named products with documents containing fields like name, price, category, description, etc.
Implement pagination to retrieve search results in batches of 2 documents at a time.

Steps

Open the Kibana Console or use a REST client.

Index sample products documents

POST /products/_bulk
{"index":{"_id":1}}
{"name":"Product A","price":99.99,"category":"Electronics","description":"High-quality product"}
{"index":{"_id":2}}
{"name":"Product B","price":49.99,"category":"Books","description":"Best-selling novel"}
{"index":{"_id":3}}
{"name":"Product C","price":149.99,"category":"Electronics","description":"Top-rated gadget"}
{"index":{"_id":4}}
{"name":"Product D","price":29.99,"category":"Clothing","description":"Stylish t-shirt"}
{"index":{"_id":5}}
{"name":"Product E","price":19.99,"category":"Books","description":"Classic literature"}

Define the initial search query with pagination (notice the use of size)

GET products/_search
{
  "size": 2, 
  "query": {
    "match_all": {}
  },
  "from": 0
}

To retrieve the next page of results, use one of two methods:

Update the from field with the document count to proceed from (not the document id)

GET products/_search
{
  "size": 2, 
  "query": {
    "match_all": {}
  },
  "from": 2
}

OR If you are sorting the documents as well as paginating then you can use the search_after parameter along with the sort values from the last hit in the previous page

// search with sort on page 1
GET /products/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    "_doc"
  ],
  "size": 2
}

// second search using the sort value 
// from the last document of the previous search
GET /products/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    "_doc"
  ],
  "size": 2,
  "search_after": [6]
}

Test

Confirm the index exists
```
GET /products
```

Execute the initial search query to retrieve the first 2 documents

{
  ...
    "hits": [
      {
        "_index": "products",
        "_id": "1",
        "_score": 1,
        "_source": {
          "name": "Product A",
          "price": 99.99,
          "category": "Electronics",
          "description": "High-quality product"
        }
      },
      {
        "_index": "products",
        "_id": "2",
        "_score": 1,
        "_source": {
          "name": "Product B",
          "price": 49.99,
          "category": "Books",
          "description": "Best-selling novel"
        }
      }
    ]
  }
}

Change from to 2 and execute it again to get the next 2 items.

{
  ...
    "hits": [
      {
        "_index": "products",
        "_id": "3",
        "_score": 1,
        "_source": {
          "name": "Product C",
          "price": 149.99,
          "category": "Electronics",
          "description": "Top-rated gadget"
        }
      },
      {
        "_index": "products",
        "_id": "4",
        "_score": 1,
        "_source": {
          "name": "Product D",
          "price": 29.99,
          "category": "Clothing",
          "description": "Stylish t-shirt"
        }
      }
    ]
  }
}

Considerations

The size parameter specifies the number of documents to retrieve per page.
The from parameter is used for the initial query to start from the beginning.
The search_after parameter can be used for subsequent queries to retrieve the next page of results based on the sort values from the last hit or simply update the from parameter to start with the next group starting from a certain number of items in the search results. The following are required to use search_after
- The sort parameter is used to ensure consistent ordering of results across pages.
- The _doc field is used as a tiebreaker to ensure a stable sort order.

Clean-up (optional)

Delete the index
```
DELETE products
```

Documentation

3.4 Task: Define and use index aliases

Example 1: Creating Index Aliases for Customer Data

This is an example of the simplest kind of alias.

Requirements

Create multiple indices for customer data (e.g., customers-2024-01, customers-2024-02).
Create an alias that points to these indices.
Use the alias to perform search operations across all customer indices.

Steps

Open the Kibana Console or use a REST Client.

Create the 2 indices by indexing sample documents

POST /customers-2024-01/_bulk
{ "index": { "_id": "1" } }
{ "name": "John Doe", "email": "john.doe@example.com", "signup_date": "2024-01-15" }
{ "index": { "_id": "2" } }
{ "name": "Jane Smith", "email": "jane.smith@example.com", "signup_date": "2024-01-20" }

POST /customers-2024-02/_bulk
{ "index": { "_id": "1" } }
{ "name": "Alice Johnson", "email": "alice.johnson@example.com", "signup_date": "2024-02-05" }
{ "index": { "_id": "2" } }
{ "name": "Bob Brown", "email": "bob.brown@example.com", "signup_date": "2024-02-10" }

Create an alias for the two indices

POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "customers-*",
        "alias": "customers"
      }
    }
  ]
}

Execute a search query using the alias and confirm 4 documents returned
```
GET /customers/_search
```

Execute a search query for John Doe’s record

GET customers/_search
{
  "query": {
    "match": {
      "name": "john doe"
    }
  }
}

Test

Verify the alias was created
```
GET /_alias/customers
```

Confirm 4 documents returned when executing the test query

GET /customers/_search
{
  "query": {
    "match_all": {}
  }
}

Or just

GET /customers/_search

Considerations

Alias Flexibility: Using an alias allows for flexibility in managing indices. The alias can point to multiple indices, making it easier to manage and query data across time-based indices.
Index Patterns: Ensure that the alias name (customers) is descriptive and clearly indicates its purpose.
Performance: Searching using an alias is efficient and does not introduce significant overhead compared to searching directly on indices.

Clean-up (optional)

Delete the aliases

DELETE customers-2024-01/_alias/customers
DELETE customers-2024-02/_alias/customers

Delete the 2 indices

DELETE customers-2024-01
DELETE customers-2024-02

Documentation

Example 2: Creating index aliases for logging data with filtering and routing

This is a slightly more complex use of an index alias. It includes a custom configuration for each index defined in the alias and any custom filtering and/or routing that is required.

Requirements

Three indices (logs_2022, logs_2023, and logs_2024) with documents containing log data (message, level, timestamp)
An index alias (logs) that points to all three indices with filtering and routing based on the log level
- logs_2022
  - filter on level ERROR
  - routing to error
- logs_2023
  - filter on level INFO
  - routing to info
- logs_2024
  - filter on level DEBUG
  - routing to debug
A search query against the message field to retrieve documents from the alias

Steps

Open the Kibana Console or use a REST client.

Create the logs_2022, logs_2023, and logs_2024 indices

PUT logs_2022
{
  "mappings": {
    "properties": {
      "message": {
        "type": "text"
      },
      "level": {
        "type": "keyword"
      },
      "timestamp": {
        "type": "date"
      }
    }
  }
}

PUT logs_2023
{
  "mappings": {
    "properties": {
      "message": {
        "type": "text"
      },
      "level": {
        "type": "keyword"
      },
      "timestamp": {
        "type": "date"
      }
    }
  }
}

PUT logs_2024
{
  "mappings": {
    "properties": {
      "message": {
        "type": "text"
      },
      "level": {
        "type": "keyword"
      },
      "timestamp": {
        "type": "date"
      }
    }
  }
}

Create an index alias (logs) with filtering and routing (this must be done before indexing any documents)

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "logs_2022",
        "alias": "logs",
        "filter": {
          "term": {
            "level": "ERROR"
          }
        },
        "routing": "error"
      }
    },
    {
      "add": {
        "index": "logs_2023",
        "alias": "logs",
        "filter": {
          "term": {
            "level": "INFO"
          }
        },
        "routing": "info"
      }
    },
    {
      "add": {
        "index": "logs_2024",
        "alias": "logs",
        "filter": {
          "term": {
            "level": "DEBUG"
          }
        },
        "routing": "debug"
      }
    }
  ]
}

Index sample documents

POST /logs_2022/_bulk
{ "index": { "_id": "1" } }
{ "message": "Error occurred", "level": "ERROR", "timestamp": "2022-01-01T12:00:00Z" }
{ "index": { "_id": "2" } }
{ "message": "Error occurred", "level": "ERROR", "timestamp": "2022-01-01T12:00:00Z" }

POST /logs_2023/_bulk
{ "index": { "_id": "1" } }
{ "message": "Info message", "level": "INFO", "timestamp": "2023-01-01T12:00:01Z" }
{ "index": { "_id": "2" } }
{ "message": "Info message", "level": "INFO", "timestamp": "2023-01-01T12:00:01Z" }

POST /logs_2024/_bulk
{ "index": { "_id": "1" } }
{ "message": "Debug message", "level": "DEBUG", "timestamp": "2024-01-01T12:00:01Z" }
{ "index": { "_id": "2" } }
{ "message": "Debug message", "level": "DEBUG", "timestamp": "2024-01-01T12:00:01Z" }

Create a general search using the logs alias (all log messages should be returned)
```
GET logs/_search
```

Create a search query using the logs alias to search for error and info messages

GET /logs/_search
{
  "query": {
    "terms": {
      "message": [
        "error",
        "info"
      ]
    }
  }
}

Test

Verify the alias was created
```
GET /_alias/logs
```

Confirm 4 documents returned when executing the query from Step 6: 2 from logs_2022 and 2 from logs_2023

{
  ...
    "hits": [
      {
        "_index": "logs_2022",
        "_id": "1",
        "_score": 1,
        "_source": {
          "message": "Error occurred",
          "level": "ERROR",
          "timestamp": "2022-01-01T12:00:00Z"
        }
      },
      {
        "_index": "logs_2022",
        "_id": "2",
        "_score": 1,
        "_source": {
          "message": "Error occurred",
          "level": "ERROR",
          "timestamp": "2022-01-01T12:00:00Z"
        }
      },
      {
        "_index": "logs_2023",
        "_id": "1",
        "_score": 1,
        "_source": {
          "message": "Info message",
          "level": "INFO",
          "timestamp": "2023-01-01T12:00:01Z"
        }
      },
      {
        "_index": "logs_2023",
        "_id": "2",
        "_score": 1,
        "_source": {
          "message": "Info message",
          "level": "INFO",
          "timestamp": "2023-01-01T12:00:01Z"
        }
      }
    ]
  }
}

Considerations

The index must be set up in the proper order for the query using the alias with filter and routing to work:
- create the index
- create the alias using filtering and/or routing
- index the documents
Index aliases with filtering and routing allow you to control which documents are included in the alias based on specific criteria.
In this example, we created an alias that points to three indices with filtering based on the log level and routing to separate indices.

Clean-up (optional)

Delete the aliases

DELETE logs_2022/_alias/logs
DELETE logs_2023/_alias/logs
DELETE logs_2024/_alias/logs

Delete the indices

DELETE logs_2022
DELETE logs_2023
DELETE logs_2024

Documentation

3.5 Task: Define and use a search template

Example 1: Creating Search Templates for Product Search

Requirements

Create an index and populate it with example product documents
Define a search template for querying products using description

Steps

Open the Kibana Console or use a REST Client

Create the index as a side-effect of indexing sample documents

POST /products/_bulk
{ "index": { "_id": "1" } }
{ "name": "Laptop", "description": "A high-performance laptop", "price": 999.99 }
{ "index": { "_id": "2" } }
{ "name": "Smartphone", "description": "A latest model smartphone", "price": 799.99 }
{ "index": { "_id": "3" } }
{ "name": "Tablet", "description": "A new tablet with excellent features", "price": 499.99 }

Define a search template

PUT _scripts/product_search_template
{
  "script": {
    "lang": "mustache",
    "source": {
      "query": {
        "match": {
          "description": "{{query_string}}"
        }
      }
    }
  }
}

Use the search template

GET products/_search/template
{
  "id": "product_search_template",
  "params": {
    "query_string": "laptop"
  }
}

Test

Verify the documents are indexed
```
GET products/_search
```
Verify the template was created
```
GET _scripts/product_search_template
```

Perform a search using the template (results below)

{
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "hits": [
      {
        "_index": "products",
        "_id": "1",
        "_source": {
          "name": "Laptop",
          "description": "A high-performance laptop",
          "price": 999.99
        }
      }
    ]
  }
}

Considerations

Template Flexibility: Using a search template allows for reusable and parameterized queries, reducing the need to write complex queries multiple times.
Performance: Search templates can improve performance by reusing the query logic and reducing the overhead of constructing queries dynamically.
Template Language: Mustache is used as the templating language, providing a simple and powerful way to define dynamic queries.

Clean-up (optional)

Delete the search template
```
DELETE _scripts/product_search_template
```
Delete the index
```
DELETE products
```

Documentation

Example 2: Creating search templates for an e-commerce product catalog with sorting and pagination

Requirements

An Elasticsearch index named products with documents containing the fields name, price, category, description, rating.
Define a search template to search for products based on a user-provided query string, category filter, sort order, and pagination.

Steps

Open the Kibana Console or use a REST client.

Index sample product documents using the /_bulk endpoint:

POST /products/_bulk
{"index":{"_id":1}}
{"name":"Product A", "price":99.99, "category":"Electronics", "description":"High-quality product", "rating":4.2}
{"index":{"_id":2}}
{"name":"Product B", "price":49.99, "category":"Books", "description":"Best-selling novel", "rating":4.5}
{"index":{"_id":3}}
{"name":"Product C", "price":149.99, "category":"Electronics", "description":"Top-rated gadget", "rating":3.8}
{"index":{"_id":4}}
{"name":"Product D", "price":29.99, "category":"Clothing", "description":"Stylish t-shirt", "rating":4.1}

Iteratively create a search query that satisfies the various requirements (query string, category filter, sort order, pagination)

// just search for a product using the name field
GET products/_search
{
  "query": {
    "query_string": {
      "default_field": "name",
      "query": "product"
    }
  }
}

// add the filter which entails also changing the query set-up
GET products/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "default_field": "name",
            "query": "product"
          }
        }
      ],
      "filter": [
        {
          "term": {
            "category": "electronics"
          }
        }
      ]
    }
  }
}

// add sort
GET products/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "default_field": "name",
            "query": "product"
          }
        }
      ],
      "filter": [
        {
          "term": {
            "category": "electronics"
          }
        }
      ]
    }
  },
  "sort": [
    {
      "price": {
        "order": "desc"
      }
    }
  ]
}

// add pagination (such as it is)
GET products/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "default_field": "name",
            "query": "product"
          }
        }
      ],
      "filter": [
        {
          "term": {
            "category": "electronics"
          }
        }
      ]
    }
  },
  "sort": [
    {
      "price": {
        "order": "desc"
      }
    }
  ],
  "size": 1,
  "from": 1
}

Using the above, define the search template:

PUT _scripts/product_search_template
{
  "script": {
    "lang": "mustache",
    "source": {
      "query": {
        "bool": {
          "must": [
            {
              "query_string": {
                "default_field": "name",
                "query": "{{query_string}}"
              }
            }
          ],
          "filter": [
            {
              "term": {
                "category": "{{category}}"
              }
            }
          ]
        }
      },
      "sort": [
        {
          "price": {
            "order": "{{sort_order}}"
          }
        }
      ],
      "from": "{{from}}",
      "size": "{{size}}"
    }
  }
}

Use the _render endpoint to take a look at the formatting of the query

POST _render/template
{
  "id": "product_search_template",
  "params": {
    "query_string": "product",
    "category" : "electronics",
    "sort_order" : "desc",
    "from": 0,
    "size": 1
  }
}

Use the search template with sorting and pagination:

GET products/_search/template
{
  "id": "product_search_template",
  "params": {
    "query_string": "product",
    "category" : "books",
    "sort_order" : "desc",
    "from": 0,
    "size": 1
  }
}

Test

Verify the documents are indexed
```
GET products/_search
```
Verify the template is created
```
GET _scripts/product_search_template
```

Execute a query for product, category of books, sort order of desc, and pagination starting at item 0, with a result size of 1.

{
  ...
    "hits": [
      {
        "_index": "products",
        "_id": "2",
        "_score": null,
        "_source": {
          "name": "Product B",
          "price": 49.99,
          "category": "Books",
          "description": "Best-selling novel",
          "rating": 4.5
        },
        "sort": [
          49.99
        ]
      }
    ]
  }
}

Considerations

The search template includes sorting and pagination parameters (sort_field, sort_order, from, size).
The sort parameter in the template specifies the field and order for sorting the results.
The from and size parameters control the pagination of the results.
The params object in the search template request provides the values for all placeholders in the template.

Clean-up (optional)

Delete the search template
```
DELETE _scripts/product_search_template
```
Delete the index
```
DELETE products
```

Documentation

Example 3: Creating search templates for an e-commerce product catalog with nested queries, sorting, pagination, and aggregations

Requirements

An index named products with documents containing the fields name, price, category, description, rating, tags, specifications, specifications.ram and specifications.storage.
Define a search template to search for products based on:
- a user-provided query string,
- category filter
- tag filter
- sort order
- pagination
include aggregations for category, tags, and price_range

Steps

Open the Kibana Console or use a REST client.

Create the index with two fields (category and tags) of type keyword for use in the aggregation

PUT products
{
  "mappings": {
    "properties": {
      "name": {
        "type": "text"
      },
      "price" : {
        "type": "float"
      },
      "category" : {
        "type": "keyword"
      },
      "description" : {
        "type": "text"
      },
      "rating" : {
        "type": "float"
      },
      "tags" : {
        "type": "keyword"
      },
      "specifications" : {
        "properties": {
          "ram" : {
            "type" : "text"
          },
          "storage" : {
            "type" : "text"
          }
        }
      }
    }
  }
}

Index sample products documents

POST /products/_bulk
{"index":{"_id":1}}
{"name":"Product A", "price":99.99, "category":"Electronics", "description":"High-quality product", "rating":4.2, "tags":["electronics", "gadget"], "specifications":{"ram":"8GB", "storage":"256GB"}}
{"index":{"_id":2}}
{"name":"Product B", "price":49.99, "category":"Books", "description":"Best-selling novel", "rating":4.5, "tags":["book", "fiction"]}
{"index":{"_id":3}}
{"name":"Product C", "price":149.99, "category":"Electronics", "description":"Top-rated gadget", "rating":3.8, "tags":["electronics", "laptop"], "specifications":{"ram":"16GB", "storage":"512GB"}}
{"index":{"_id":4}}
{"name":"Product D", "price":29.99, "category":"Clothing", "description":"Stylish t-shirt", "rating":4.1, "tags":["clothing", "tshirt"]}

Define a search query that satisfies the requirements

GET products/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "default_field": "name",
            "query": "product"
          }
        }
      ],
      "filter": [
        {
          "term": {
            "category": "Electronics"
          }
        },
        {
          "term": {
            "tags": "electronics"
          }
        }
      ]
    }
  },
  "sort": [
    {
      "price": {
        "order": "desc"
      }
    }
  ],
  "size": 1,
  "from": 0,
  "aggs": {
    "category_aggs": {
      "terms": {
        "field": "category"
      }
    },
    "tags_aggs": {
      "terms": {
        "field": "tags"
      }
    },
    "price_range_aggs": {
      "range": {
        "field": "price",
        "ranges": [
          {
            "to": 30
          },
          {
            "from": 30,
            "to": 100
          },
          {
            "from": 100
          }
        ]
      }
    }
  }
}

Use the above query to create the search template

PUT _scripts/products_search_template
{
  "script": {
    "lang": "mustache",
    "source": {
      "query": {
        "bool": {
          "must": [
            {
              "query_string": {
                "default_field": "name",
                "query": "{{query}}"
              }
            }
          ],
          "filter": [
            {
              "term": {
                "category": "{{category}}"
              }
            },
            {
              "term": {
                "tags": "{{tags}}"
              }
            }
          ]
        }
      },
      "sort": [
        {
          "price": {
            "order": "{{sort_order}}"
          }
        }
      ],
      "size": "{{size}}",
      "from": "{{from}}",
      "aggs": {
        "category_aggs": {
          "terms": {
            "field": "category"
          }
        },
        "tags_aggs": {
          "terms": {
            "field": "tags"
          }
        },
        "price_range_aggs": {
          "range": {
            "field": "price",
            "ranges": [
              {
                "to": 30
              },
              {
                "from": 30,
                "to": 100
              },
              {
                "from": 100
              }
            ]
          }
        }
      }
    }
  }
}

Use the search template with sorting, pagination, and aggregations:

GET products/_search/template
{
  "id": "products_search_template",
  "params": {
    "query" : "product",
    "category" : "Electronics",
    "tags" : "electronics",
    "sort_order" : "desc",
    "size" : 2,
    "from" : 0
  }
}

Test

Verify the index is created
```
GET products
```
Verify the documents are indexed
```
GET products/_search
```
Verify the template is created
```
GET _scripts/products_search_template
```

Execute a search using the search template query, and it should return the first 2 documents matching the provided query string (“*“), category filter (”Electronics”), tag filter (“electronics”), sorted by price in descending order with aggregations.

The response should also include aggregations for category, tags, and price.

{
  ...
  "hits": [
    {
      "_index": "products",
      "_id": "3",
      "_score": null,
      "_source": {
        "name": "Product C",
        "price": 149.99,
        "category": "Electronics",
        "description": "Top-rated gadget",
        "rating": 3.8,
        "tags": [
          "electronics",
          "laptop"
        ],
        "specifications": {
          "ram": "16GB",
          "storage": "512GB"
        }
      },
      "sort": [
        149.99
      ]
    },
    {
      "_index": "products",
      "_id": "1",
      "_score": null,
      "_source": {
        "name": "Product A",
        "price": 99.99,
        "category": "Electronics",
        "description": "High-quality product",
        "rating": 4.2,
        "tags": [
          "electronics",
          "gadget"
        ],
        "specifications": {
          "ram": "8GB",
          "storage": "256GB"
        }
      },
      "sort": [
        99.99
      ]
    }
  ]
},
"aggregations": {
  "category_aggs": {
    "doc_count_error_upper_bound": 0,
    "sum_other_doc_count": 0,
    "buckets": [
      {
        "key": "Electronics",
        "doc_count": 2
      }
    ]
  },
  "tags_aggs": {
    "doc_count_error_upper_bound": 0,
    "sum_other_doc_count": 0,
    "buckets": [
      {
        "key": "electronics",
        "doc_count": 2
      },
      {
        "key": "gadget",
        "doc_count": 1
      },
      {
        "key": "laptop",
        "doc_count": 1
      }
    ]
  },
  "price_range_aggs": {
    "buckets": [
      {
        "key": "*-30.0",
        "to": 30,
        "doc_count": 0
      },
      {
        "key": "30.0-100.0",
        "from": 30,
        "to": 100,
        "doc_count": 1
      },
      {
        "key": "100.0-*",
        "from": 100,
        "doc_count": 1
      }
    ]
  }
}
}

Considerations

The search template includes queries, filters, sorting, pagination, and aggregations.
The tags filter uses a terms query to match documents with any of the specified tags.
The aggs section in the template defines the aggregations to be included in the search results.
The params object in the search template request provides the values for all placeholders in the template.

Clean-up (optional)

Delete the search template
```
DELETE _scripts/product_search_template
```
Delete the index
```
DELETE products
```