First steps

Once you have followed the steps in Installation for the operator and its dependencies, you will now go through the steps to set up and connect to an OpenSearch instance.

User configuration

Let’s begin by defining the users with passwords and their corresponding mappings to back-end roles within a Secret:

---
apiVersion: v1
kind: Secret
metadata:
  name: initial-opensearch-security-config
stringData:
  internal_users.yml: |
    ---
    _meta:
      type: internalusers
      config_version: 2
    admin:
      hash: $2y$10$xRtHZFJ9QhG9GcYhRpAGpufCZYsk//nxsuel5URh0GWEBgmiI4Q/e
      reserved: true
      backend_roles:
      - admin
      description: OpenSearch admin user
    kibanaserver:
      hash: $2y$10$vPgQ/6ilKDM5utawBqxoR.7euhVQ0qeGl8mPTeKhmFT475WUDrfQS
      reserved: true
      description: OpenSearch Dashboards user
  roles_mapping.yml: |
    ---
    _meta:
      type: rolesmapping
      config_version: 2
    all_access:
      reserved: false
      backend_roles:
      - admin
    kibana_server:
      reserved: true
      users:
      - kibanaserver

The passwords in internal_users.yml are hashes using the bcrypt algorithm. Such a hash can be e.g. generated with htpasswd:

$ htpasswd -nbBC 10 admin AJVFsGJBbpT6mChn
admin:$2y$10$xRtHZFJ9QhG9GcYhRpAGpufCZYsk//nxsuel5URh0GWEBgmiI4Q/e

$ htpasswd -nbBC 10 kibanaserver E4kENuEmkqH3jyHC
kibanaserver:$2y$10$vPgQ/6ilKDM5utawBqxoR.7euhVQ0qeGl8mPTeKhmFT475WUDrfQS

Now apply the Secret:

kubectl apply -f initial-opensearch-security-config.yaml

OpenSearch cluster definition

OpenSearch nodes must be created as a custom resource; Create a file called opensearch.yaml:

---
apiVersion: opensearch.stackable.tech/v1alpha1
kind: OpenSearchCluster
metadata:
  name: simple-opensearch
spec:
  image:
    productVersion: 3.4.0
  clusterConfig:
    security:
      settings:
        config:
          managedBy: API
          content:
            value:
              _meta:
                type: config
                config_version: 2
              config:
                dynamic:
                  authc:
                    basic_internal_auth_domain:
                      description: Authenticate via HTTP Basic against internal users database
                      http_enabled: true
                      transport_enabled: true
                      order: 1
                      http_authenticator:
                        type: basic
                        challenge: true
                      authentication_backend:
                        type: intern
                  authz: {}
        internalUsers:
          managedBy: API
          content:
            valueFrom:
              secretKeyRef:
                name: initial-opensearch-security-config
                key: internal_users.yml
        rolesMapping:
          managedBy: API
          content:
            valueFrom:
              secretKeyRef:
                name: initial-opensearch-security-config
                key: roles_mapping.yml
  nodes:
    roleConfig:
      discoveryServiceListenerClass: external-unstable
    roleGroups:
      default:
        replicas: 3
    configOverrides:
      opensearch.yml:
        plugins.security.restapi.roles_enabled: all_access

And apply it:

kubectl apply -f opensearch.yaml

metadata.name contains the name of the OpenSearch cluster.

You need to wait for the OpenSearch nodes to finish deploying. You can do so with this command:

kubectl rollout status --watch statefulset/simple-opensearch-nodes-default --timeout 600s

Using the REST API

You can use the REST API as follows:

export CREDENTIALS=admin:AJVFsGJBbpT6mChn

OPENSEARCH_HOST=$(
    kubectl get configmap simple-opensearch \
        --output=jsonpath='{.data.OPENSEARCH_HOSTS}'
)

curl \
    --insecure \
    --user $CREDENTIALS \
    --request PUT \
    --json '{"name": "Stackable"}' \
    "$OPENSEARCH_HOST/sample_index/_doc/1"

# Formatted output:
# {
#   "_index": "sample_index",
#   "_id": "1",
#   "_version": 1,
#   "result": "created",
#   "_shards": {
#     "total": 2,
#     "successful": 1,
#     "failed": 0
#   },
#   "_seq_no": 0,
#   "_primary_term": 1
# }


curl \
    --insecure \
    --user $CREDENTIALS \
    --request GET \
    "$OPENSEARCH_HOST/sample_index/_doc/1"

# Formatted output:
# {
#   "_index": "sample_index",
#   "_id": "1",
#   "_version": 1,
#   "_seq_no": 0,
#   "_primary_term": 1,
#   "found": true,
#   "_source": {
#     "name": "Stackable"
#   }
# }

Great! Now you can create your own indexes, populate them with data and search for it.

Using OpenSearch Dashboards

OpenSearch Dashboards is currently not managed by the operator, but the Stackable Data Platform provides a supported OCI image. First, create the Helm values file opensearch-dashboards-values.yaml:

---
image:
  repository: oci.stackable.tech/sdp/opensearch-dashboards
  tag: 3.4.0-stackable0.0.0-dev
serviceAccount:
  create: false
  name: simple-opensearch-serviceaccount
config:
  opensearch_dashboards.yml:
    server:
      ssl:
        enabled: true
        certificate: /stackable/opensearch-dashboards/config/tls/tls.crt
        key: /stackable/opensearch-dashboards/config/tls/tls.key
    opensearch:
      username: kibanaserver
      ssl:
        verificationMode: full
        certificateAuthorities:
        - /stackable/opensearch-dashboards/config/tls/ca.crt
    opensearch_security:
      cookie:
        secure: true
# See https://github.com/opensearch-project/helm-charts/blob/main/charts/opensearch-dashboards/templates/deployment.yaml#L122
opensearchHosts: ""
extraEnvs:
- name: OPENSEARCH_HOSTS
  valueFrom:
    configMapKeyRef:
      name: simple-opensearch
      key: OPENSEARCH_HOSTS
- name: OPENSEARCH_PASSWORD
  valueFrom:
    secretKeyRef:
      name: opensearch-credentials
      key: kibanaserver
extraVolumes:
- name: tls
  ephemeral:
    volumeClaimTemplate:
      metadata:
        annotations:
          secrets.stackable.tech/class: tls
          secrets.stackable.tech/scope: service=opensearch-dashboards
      spec:
        storageClassName: secrets.stackable.tech
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: "1"
extraVolumeMounts:
- mountPath: /stackable/opensearch-dashboards/config/tls
  name: tls
- mountPath: /stackable/opensearch-dashboards/config/opensearch_dashboards.yml
  name: config
  subPath: opensearch_dashboards.yml
podSecurityContext:
  fsGroup: 1000

Then, create the Secret with the credentials for the kibanaserver user and deploy the OpenSearch Dashboards Helm chart:

kubectl create secret generic opensearch-credentials \
    --from-literal kibanaserver=E4kENuEmkqH3jyHC

helm install opensearch-dashboards opensearch-dashboards \
    --repo https://opensearch-project.github.io/helm-charts \
    --version 3.4.0 \
    --values opensearch-dashboards-values.yaml \
    --wait

Forward the OpenSearch Dashboards port 5601 to localhost, open https://localhost:5601 in your browser and log in with username admin and password AJVFsGJBbpT6mChn:

kubectl port-forward services/opensearch-dashboards 5601 > /dev/null 2>&1 &

Browser showing OpenSearch Dashboards after logging in

What’s next

Check the Usage guide to find out more about configuring your OpenSearch instance or have a look at the OpenSearch documentation to ingest.