How to perform basic Elasticsearch operations

Perform basic Elasticsearch 7.x operations.

This blog post describes only basic operations, and it is already too long. There is a lot more to learn, so I strongly suggest studying Elasticsearch Reference.

Create index

Create bookmarks_index index.

$ curl --silent \
       --request PUT \
       --header 'Content-Type: application/json' \
       http://10.0.2.15:9200/bookmarks_index?pretty=true \
       --data-ascii \
'{
  "settings": {
    "index": {
      "number_of_shards": 1,
      "number_of_replicas": 1
    }
  },  
  "mappings": {
    "properties" : {
      "name": {"type": "keyword"},
      "address": {"type": "keyword"},
      "date": {"type": "date"},
      "tag": {"type": "keyword", "store": true}
    }
  }
}'

{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "bookmarks_index"
}

Alter index settings

Alter index settings.

$ curl --silent \
       --request PUT \
       --header 'Content-Type: application/json' \
       http://10.0.2.15:9200/bookmarks_index/_settings?pretty=true \
       --data-ascii \
'{
  "index": {
    "number_of_replicas": "0"
  }
}'

{
  "acknowledged" : true
}

Display index settings.

$ curl --silent http://10.0.2.15:9200/bookmarks_index/_settings?pretty=true

{
  "bookmarks_index" : {
    "settings" : {
      "index" : {
        "creation_date" : "1578215671819",
        "number_of_shards" : "1",
        "number_of_replicas" : "0",
        "uuid" : "linR7BR-Q-2qyD1A6qMKAQ",
        "version" : {
          "created" : "7050199"
        },
        "provided_name" : "bookmarks_index"
      }
    }
  }
}

Alter index mappings

You cannot modify an existing field as it could invalidate data that is already indexed. To perform such an operation, you need to create a new index containing updated mapping and reindex data.

Alter index mappings to add an alias.

$ curl --silent \
       --request PUT \
       --header 'Content-Type: application/json' \
       http://10.0.2.15:9200/bookmarks_index/_mappings?pretty=true \
       --data-ascii \
'{
  "properties": {
    "url": {
      "type": "alias",
      "path": "address"
    }
  }
}'

{
  "acknowledged" : true
}

Display index mappings.

$ curl --silent http://10.0.2.15:9200/bookmarks_index/_mappings?pretty=true
{
  "bookmarks_index" : {
    "mappings" : {
      "properties" : {
        "address" : {
          "type" : "keyword"
        },
        "date" : {
          "type" : "date"
        },
        "name" : {
          "type" : "keyword"
        },
        "tag" : {
          "type" : "keyword",
          "store" : true
        },
        "url" : {
          "type" : "alias",
          "path" : "address"
        }
      }
    }
  }
}

Alter index aliases

Add aliases to an index.

$ curl --silent \
       --request PUT \
       http://10.0.2.15:9200/bookmarks_index/_alias/bookmarks?pretty=true

{
  "acknowledged" : true
}

$ curl --silent \
       --request PUT \
       http://10.0.2.15:9200/bookmarks_index/_alias/mybookmarks?pretty=true

{
  "acknowledged" : true
}

Delete an index alias.

$ curl --silent \
       --request DELETE \
       http://10.0.2.15:9200/bookmarks_index/_alias/mybookmarks?pretty=true

{
  "acknowledged" : true
}

Display index aliases.

$ curl --silent \
       http://10.0.2.15:9200/bookmarks_index/_alias?pretty=true

{
  "bookmarks_index" : {
    "aliases" : {
      "bookmarks" : { }
    }
  }
}

Check if index exists

Determine if a non-existent index exists.

$ curl --silent --head http://10.0.2.15:9200/bookmark/

HTTP/1.1 404 Not Found
content-type: application/json; charset=UTF-8
content-length: 377

Determine if the index exists using its alias.

$ curl --silent --head http://10.0.2.15:9200/bookmarks/

HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 433

Determine if the index exists using its real name.

$ curl --silent --head http://10.0.2.15:9200/bookmarks_index/

HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 433

List indices

List indices.

$ curl --silent "http://10.0.2.15:9200/_cat/indices?v"
health status index           uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   bookmarks_index linR7BR-Q-2qyD1A6qMKAQ   1   0          0            0       283b           283b

$ curl --silent "http://10.0.2.15:9200/_cat/indices?v&h=index,status"
index           status
bookmarks_index open

Create documents

Add some documents and wait for a refresh each time.

$ curl --silent \
       --header 'Content-Type: application/json' \
       --request POST \
       "http://10.0.2.15:9200/bookmarks/_doc?refresh=wait_for&pretty=true" \
       --data-ascii \
'{ 
   "name": "Personal blog", 
   "address": "https://sleeplessbeastie.eu", 
   "tag": ["personal", "blog"],
   "date":"2020-01-05"
}'

{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "V5gUdW8BkI-uNnF5JfU7",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

$ curl --silent \
       --header 'Content-Type: application/json' \
       --request POST \
       "http://10.0.2.15:9200/bookmarks/_doc?refresh=wait_for&pretty=true" \
       --data-ascii \
'{ 
   "name": "Debian", 
   "address": "https://www.debian.org", 
   "tag": ["linux", "debian"],
   "date":"2020-01-05"
}'

{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "WJgYdW8BkI-uNnF5hvVG",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 1,
  "_primary_term" : 1
}

$ curl --silent \
       --header 'Content-Type: application/json' \
       --request POST \
       "http://10.0.2.15:9200/bookmarks/_doc?refresh=wait_for&pretty=true" \
       --data-ascii \
'{ 
   "name": "Debian packages", 
   "address": "https://www.debian.org/distrib/packages", 
   "tag": ["linux", "debian", "packages"],
   "date":"2020-01-05"
}'

{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "WZgZdW8BkI-uNnF5AfWy",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 2,
  "_primary_term" : 1
}

Search documents

Match all documents.

$ curl --silent \
       --header 'Content-Type: application/json' \
       "http://10.0.2.15:9200/bookmarks/_search?pretty=true" \
       --data-ascii \
'{
  "query": { 
    "match_all": {} 
   }
}'

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 3,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "V5gUdW8BkI-uNnF5JfU7",
        "_score" : 1.0,
        "_source" : {
          "name" : "Personal blog",
          "address" : "https://sleeplessbeastie.eu",
          "tag" : [
            "personal",
            "blog"
          ],
          "date" : "2020-01-05"
        }
      },
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "WJgYdW8BkI-uNnF5hvVG",
        "_score" : 1.0,
        "_source" : {
          "name" : "Debian",
          "address" : "https://www.debian.org",
          "tag" : [
            "linux",
            "debian"
          ],
          "date" : "2020-01-05"
        }
      },
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "WZgZdW8BkI-uNnF5AfWy",
        "_score" : 1.0,
        "_source" : {
          "name" : "Debian packages",
          "address" : "https://www.debian.org/distrib/packages",
          "tag" : [
            "linux",
            "debian",
            "packages"
          ],
          "date" : "2020-01-05"
        }
      }
    ]
  }
}

Exact match query.

$ curl --silent \
       -header 'Content-Type: application/json'
       "http://10.0.2.15:9200/bookmarks/_search?pretty=true" \
       --data-ascii \
'{
  "query": { 
    "term": { "url":"https://www.debian.org/distrib/packages"}
  }, 
  "_source": ["name", "address"]
}'

{
  "took" : 9,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 0.9808292,
    "hits" : [
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "WZgZdW8BkI-uNnF5AfWy",
        "_score" : 0.9808292,
        "_source" : {
          "address" : "https://www.debian.org/distrib/packages",
          "name" : "Debian packages"
        }
      }
    ]
  }
}

Wildcard query.

$ curl --silent \
       --header 'Content-Type: application/json'
       "http://10.0.2.15:9200/bookmarks/_search?pretty=true" \
       --data-ascii \
'{
  "query": { 
    "wildcard": { "url":"*debian*"} 
  }, 
  "_source": ["name", "address"]
}'

{
  "took" : 27,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "WJgYdW8BkI-uNnF5hvVG",
        "_score" : 1.0,
        "_source" : {
          "address" : "https://www.debian.org",
          "name" : "Debian"
        }
      },
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "WZgZdW8BkI-uNnF5AfWy",
        "_score" : 1.0,
        "_source" : {
          "address" : "https://www.debian.org/distrib/packages",
          "name" : "Debian packages"
        }
      }
    ]
  }
}

A more useful query that contains multiple conditions and filter context.

$ curl --silent \
       --header 'Content-Type: application/json' \
       "http://10.0.2.15:9200/bookmarks/_search?pretty=true" \
       --data-ascii \
'{
  "query": { 
    "bool": {
      "must": [
        {"wildcard": { "url":"*debian*"}}, 
        {"prefix": { "url":"https://*"}}
      ],
      "filter": [
        { "term": { "tag": "packages"}},
        { "range": { "date": { "gte": "2020-01-05" }}}
      ]  
    }
  } 
}'

{
  "took" : 52,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 2.0,
    "hits" : [
      {
        "_index" : "bookmarks_index",
        "_type" : "_doc",
        "_id" : "WZgZdW8BkI-uNnF5AfWy",
        "_score" : 2.0,
        "_source" : {
          "name" : "Debian packages",
          "address" : "https://www.debian.org/distrib/packages",
          "tag" : [
            "linux",
            "debian",
            "packages"
          ],
          "date" : "2020-01-05"
        }
      }
    ]
  }
}

Update documents

Perform partial updates.

$ curl  --silent \
        --header 'Content-Type: application/json' \
        --request POST \
        http://10.0.2.15:9200/bookmarks/_update/WZgZdW8BkI-uNnF5AfWy?pretty=true \
        --data-ascii \
'{ 
   "doc": {
      "name": "Debian - Packages"
    }
}'

{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "WZgZdW8BkI-uNnF5AfWy",
  "_version" : 2,
  "result" : "updated",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 3,
  "_primary_term" : 1
}

Perform a scripted update.

$ curl  --silent \
        --header 'Content-Type: application/json' \
        --request POST \
        http://10.0.2.15:9200/bookmarks/_update/WZgZdW8BkI-uNnF5AfWy?pretty=true \
        --data-ascii \
'{
    "script" : {
        "source": "if (ctx._source.tag.contains(params.tag)) { ctx._source.tag.remove(ctx._source.tag.indexOf(params.tag)) }",
        "lang": "painless",
        "params" : {
            "tag" : "packages"
        }
    }
}'

{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "WZgZdW8BkI-uNnF5AfWy",
  "_version" : 3,
  "result" : "updated",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 4,
  "_primary_term" : 1
}

Get document

Get a single document.

$ curl --silent http://10.0.2.15:9200/bookmarks_index/_doc/WZgZdW8BkI-uNnF5AfWy?pretty=true

{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "WZgZdW8BkI-uNnF5AfWy",
  "_version" : 3,
  "_seq_no" : 4,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "Debian - Packages",
    "address" : "https://www.debian.org/distrib/packages",
    "tag" : [
      "linux",
      "debian"
    ],
    "date" : "2020-01-05"
  }
}

Get multiple documents.

$ curl --silent \
       --header 'Content-Type: application/json' \
       http://10.0.2.15:9200/bookmarks/_mget?pretty=true \
       --data-ascii \
'{ 
   "ids": [ "WZgZdW8BkI-uNnF5AfWy","WJgYdW8BkI-uNnF5hvVG" ]
}'

{
  "docs" : [
    {
      "_index" : "bookmarks_index",
      "_type" : "_doc",
      "_id" : "WZgZdW8BkI-uNnF5AfWy",
      "_version" : 3,
      "_seq_no" : 4,
      "_primary_term" : 1,
      "found" : true,
      "_source" : {
        "name" : "Debian - Packages",
        "address" : "https://www.debian.org/distrib/packages",
        "tag" : [
          "linux",
          "debian"
        ],
        "date" : "2020-01-05"
      }
    },
    {
      "_index" : "bookmarks_index",
      "_type" : "_doc",
      "_id" : "WJgYdW8BkI-uNnF5hvVG",
      "_version" : 1,
      "_seq_no" : 1,
      "_primary_term" : 1,
      "found" : true,
      "_source" : {
        "name" : "Debian",
        "address" : "https://www.debian.org",
        "tag" : [
          "linux",
          "debian"
        ],
        "date" : "2020-01-05"
      }
    }
  ]
}

Delete document

Delete document.

$ curl --silent \
       --request DELETE \
       "http://10.0.2.15:9200/bookmarks_index/_doc/V5gUdW8BkI-uNnF5JfU7?pretty=true"

{
  "_index" : "bookmarks_index",
  "_type" : "_doc",
  "_id" : "V5gUdW8BkI-uNnF5JfU7",
  "_version" : 2,
  "result" : "deleted",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 5,
  "_primary_term" : 1
}

Notice, the document will become unavailable, but it will not be deleted immediately.

$ curl --silent "http://10.0.2.15:9200/_cat/indices?v"
health status index           uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   bookmarks_index linR7BR-Q-2qyD1A6qMKAQ   1   0          3            1     22.1kb         22.1kb

You really don’t need to, but you can use force merge to only expunge segments containing document deletions.

$ curl --silent \
       --request POST \
      "http://10.0.2.15:9200/bookmarks/_forcemerge?only_expunge_deletes=true&pretty=true"

{
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  }
}

$ curl --silent "http://10.0.2.15:9200/_cat/indices?v"
health status index           uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   bookmarks_index linR7BR-Q-2qyD1A6qMKAQ   1   0          2            0      4.7kb           4.7k

Count documents

Count documents.

$ curl --silent http://10.0.2.15:9200/bookmarks/_count?pretty=true

{
  "count" : 2,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  }
}

Check if document exists

Determine if a non-existing/deleted document exists.

$ curl --silent --head http://10.0.2.15:9200/bookmarks_index/_doc/V5gUdW8BkI-uNnF5JfU7

HTTP/1.1 404 Not Found
content-type: application/json; charset=UTF-8
content-length: 86

Determine if the document exists.

$ curl --silent --head http://10.0.2.15:9200/bookmarks_index/_doc/WJgYdW8BkI-uNnF5hvVG

HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 247

Disable index

Ensure that each shard has only a single segment on disk.

$ curl --silent \
       --request POST \
      "http://10.0.2.15:9200/bookmarks/_forcemerge?max_num_segments=1&pretty=true"

{
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  }
}

Close index to disable read/write operations.

$ curl --silent \
       --request POST \
       http://10.0.2.15:9200/bookmarks/_close?pretty=true

{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "indices" : {
    "bookmarks_index" : {
      "closed" : true
    }
  }
}

You cannot perform read/write operations.

$ curl --silent http://10.0.2.15:9200/bookmarks/_count?pretty=true

{
  "error" : {
    "root_cause" : [
      {
        "type" : "index_closed_exception",
        "reason" : "closed",
        "index_uuid" : "linR7BR-Q-2qyD1A6qMKAQ",
        "index" : "bookmarks_index"
      }
    ],
    "type" : "index_closed_exception",
    "reason" : "closed",
    "index_uuid" : "linR7BR-Q-2qyD1A6qMKAQ",
    "index" : "bookmarks_index"
  },
  "status" : 400
}

Open index to re-enable read/write operations.

$ curl --silent \
       --request POST \
       http://10.0.2.15:9200/bookmarks/_open?pretty=true

{
  "acknowledged" : true,
  "shards_acknowledged" : true
}

$ curl --silent http://10.0.2.15:9200/bookmarks/_count?pretty=true

{
  "count" : 2,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  }
}

Delete index

Do not use an alias to delete a specific index.

$ curl --silent \
       --request DELETE \
       http://10.0.2.15:9200/bookmarks?pretty=true

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "The provided expression [bookmarks] matches an alias, specify the corresponding concrete indices instead."
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "The provided expression [bookmarks] matches an alias, specify the corresponding concrete indices instead."
  },
  "status" : 400
}

Delete index using its real name.

$ curl --silent \
       --request DELETE \
       http://10.0.2.15:9200/bookmarks?pretty=true

{
  "acknowledged" : true
}

Create index#

Alter index settings#

Alter index mappings#

Alter index aliases#

Check if index exists#

List indices#

Create documents#

Search documents#

Update documents#

Get document#

Delete document#

Count documents#

Check if document exists#

Disable index#

Delete index#