Elasticsearch
Page last updated:
Overview
Elasticsearch is a powerful open-source search engine for full text search.
Integrating the Service With Your App
After the creation of the service and the service key, you’ll find all required credentials in this service key, as shown here:
{
"host": "https://d15c5caa5a3147ed9b444e4694162b3f.elasticsearch.lyra-836.appcloud.swisscom.com",
"kibana_system_username": "username",
"kibana_system_password": "password",
"logstash_system_username": "username",
"logstash_system_password": "password",
"kibana_read_only_username": "username",
"kibana_read_only_password": "password",
"kibana_dashboard_only_username": "username",
"kibana_dashboard_only_password": "password",
"full_access_username": "username",
"full_access_password": "password"
}
User Privileges
Each service key or binding to the elasticsearch-service comes with multiple users with different privileges.
kibana_read_only_username
: a user which has read access to Kibana. This user should be used when giving access to your Kibana.kibana_dashboard_only_username
: a user which has thekibana_dashboard_only_role
assigned and can only access dashboards in Kibana.full_access_username
: a user which has read and write access to all indices. This user is optimal for integration with your application.kibana_system_username
: a system user for the Kibana application. This is a system user and you should not use it (e.g. for logging in to Kibana).logstash_system_username
: a system user for the Logstash application. This is a system user and you should not use it.
Versions
For the Elasticsearch service multiple versions are available. Currently available versions:
- 7.16.1
Deprecated versions:
- 6.8.*
- 7.8.*
To create a cluster with a specific version you must provide the version (from the list above) as a parameter. E.g.:
cf create-service elasticsearch xxsmall <my-elasticsearch> -c '{"version":"X.Y.Z"}'
Or you can update any existing service by using the update-service command. E.g.:
cf update-service <service-name> -c '{"version":"X.Y.Z"}'
Note: when using a custom Elasticsearch version you must specify the version on all cf update-service
calls.
Otherwise, we will use the default version.
Version Upgrade
Upgrading a Service Instance requires cf CLI v6.46.0+ and CAPI release 1.83.0+
Elasticsearch supports upgrading service instances to newer versions. Upgrades to specific versions of Elasticsearch are described below.
To upgrade your service instances, do the following:
Confirm that an upgrade is available by running
cf services
and reviewing theupgrade available
column:$ cf services Getting services in org acceptance / space dev as admin... name service plan bound apps last operation broker upgrade available mydb elasticsearch small create succeeded ece-broker yes otherdb elasticsearch medium create succeeded ece-broker no
Upgrade the service instance using the
--upgrade
flag:$ cf update-service mydb --upgrade You are about to update mydb. Warning: This operation may be long running and will block further operations on the service until complete. Really update service mydb? [yN]: y OK
We recommend creating a backup before evey upgrade, to prevent possible data loss. The preferred method is using backman.
Upgrade to Specific Version
The upgrade to the specific versions is done through the CF CLI by running the update service command.
cf update-service <service-name> -c '{"version":"X.Y.Z"}'
We recommend creating a backup before evey elasticsearch upgrade, to prevent possible data loss. The preferred method is using backman.
Upgrade to Elasticsearch version 7.x
Before you can upgrade to the latest major release of Elasticsearch (version 7) you must check for any incompatibilities of your existing indices. We recommend first upgrading to the latest minor release of Elasticsearch version 6 (currently version 6.8.2) and then upgrade to version 7.x. Currently the upgrade assistant is not available.
Plan Upgrade
Elasticsearch service can be upgraded to larger plans (downgrading to smaller plans is not possible).
The upgrade to a larger plan is done through the CF CLI by running the update service command.
cf update-service <service-name> -p <new-plan-name>
Note: if you use a different Elasticsearch version than the default you must specify that version as well. Otherwise, we will use the default version.
cf update-service <service-name> -p <new-plan-name> -c '{"version":"X.Y.Z"}'
X-Pack
Our Elasticsearch service comes with X-Pack preinstalled and activated. The X-Pack license fee is included in the price.
Kibana
We have deprecated the Kibana buildpack with the release of cflinuxfs3 (Ubuntu 18).
You can find a guide on how to set up Kibana with Docker here: Kibana Docker
Index Lifecycle Management
To clean up some old data and free up space in your elasticsearch service, you have to set up index lifecycle policies. This is necessary, because otherwise your instance would get overloaded over time and stop working properly.
The corresponding guide can be found here: Index Lifecycle Management
Logstash
We have deprecated the Logstash buildpack with the release of cflinuxfs3 (Ubuntu 18).
You can find a guide on how to set up Logstash with Docker here: Logstash Docker
Curator
Current versions of ElasticSearch (6.7+) offer Index Lifecycle Management as alternative to Curator.
For older versions, use cf-curator-cron as workaround to periodically clean up your Indices.
High Availability
The Elasticsearch cluster consists of 3 nodes in 3 different availability zones. By default, each index in Elasticsearch has 5 primary shards and 1 replica allocated to it. If you don’t want to use the default settings, make sure to configure the number of shards & replicas accordingly in the index templates. Swisscom cannot guarantee the availability of your data unless you configure at least one replica.
Backups
If you want to create backups there is an app available that allows for automatic scheduled backups: backman - the appcloud backup manager. Any Elasticsearch service instance bound to this app will be automatically backed up, can be downloaded and also restored on-demand.
Further reading
Official ElasticSearch Documentation
View the source for this page in GitHub