Deploy a Kubernetes cluster on Amazon Elastic Kubernetes Service for performing highly scalable, highly available gallery searches.

Overview

Certain customer use cases require the ROC SDK to be one or all of:

Highly available for performing roc_gallery operations.
Highly scalable for performing multiple operations concurrently.
Easily upgraded to future SDK versions, with minimal downtime.
Easily rolled back to previous roc_gallery states.
Managed and/or administered in a cloud native environment.

In such cases, customers are encouraged to set up and manage their own Kubernetes cluster in a hosted Kubernetes environment such as Amazon EKS. While other hosted Kubernetes offerings such as Google Kubernetes Engine or Azure Kubernetes Service are available, these instructions are specific to deploying the ROC SDK on Amazon EKS.

Deployment Architecture

ROC Enterprise Search is deployed as a collection of roc-serve pods orchestrated by Kubernetes. Each pod either connects to or hosts a roc_gallery. All communication between pods is managed by a RabbitMQ Server to maintain data consistency across the cluster. The RabbitMQ Server is exposed to client applications as a Kubernetes Service via an Elastic Load Balancer. Each roc-serve pod is backed by a Kubernetes Persistent Volume in case of unexpected failures in the underlying hardware. This persistent volume contains all necessary information for a pod to continue handling requests correctly in case of a restart. The cluster provisions one additional persistent volume, snapshot storage which is accessible to all pods in the cluster. This volume provides snapshots of roc_gallery state, which are needed for handling Rollbacks and Scaleout. Additional roc-serve pods are deployed to the cluster to provide an HTTP interface. Client applications which prefer to communicate via HTTP can send requests to the URL exposed by the Kubernetes Service as http-endpoint. These requests will be converted to the AMQP protocol and forwarded to RabbitMQ.

Enterprise Search Deployment Architecture

Deployment Interface

All messages sent to and received from the cluster contain a messsage body that is a Protobuf serialized ROC SDK API function. See Web API for information about how to encode and decode Protobuf messages as well as an example showing how to communicate with the cluster via HTTP. Client applications must communicate with the deployment cluster via the RabbitMQ Server using AMQP and have two options for doing so:

1.) AMQP Client Interface

Client applications send and receive AMQP messages via the RabbitMQ Server URL exposed by the Kubernetes Service.

Enroll a template to AMQP interface using the ROC Command Line Interface
$ roc-enroll template.t amqp://<username>:<password>@<load_balancer_domain_name> --license ROC.lic

2.) HTTP Client Interface

Client applications POST messages to the HTTP URL exposed by the Kubernetes Service.

Enroll a template to HTTP interface using the ROC Command Line Interface
- $ roc-enroll template.t http://<load_balancer_domain_name>:80/ --license ROC.lic

Setup

Deployment Tools

The following third party tools are required in order to deploy the cluster:

eksctl for provisioning a cluster on Amazon EKS.
kubectl for administering the cluster.
krew for installing kubectl plugins.
docker for managing roc-serve containers to be run in the EKS cluster.

Deployment Files

claim.yaml contains the definition to create an EBS -backed Persistent Volume Claim.
storageclass.yaml contains the definition to register a Kubernetes Storage Class necessary to provision EBS volumes in Amazon EKS.
rabbitmq-cluster-deployment.yaml contains the definition to deploy a RabbitMQ Server using the RabbitMQ Cluster Operator for Kubernetes. The server is deployed behind an Elastic Load Balancer.
eks-roc-search-stateful-set.yaml contains the definition to deploy the roc-serve worker pods as a Kubernetes Stateful Set as well as the definition to deploy all persistent volumes.
eks-roc-search-http-deployment.yaml contains the definition to deploy the roc-serve HTTP interface pods so that client applications have the option to communicate with the cluster via HTTP.
eks-roc-search-service.yaml contains the definition to expose the HTTP interface to the outside world.

Setup Instructions

1.) Create a cluster on Amazon EKS.: $ eksctl create cluster --node-type t2.medium \
--zones=us-east-1a,us-east-1b
This command will take several minutes and result in a randomly generated cluster name (e.g. "fabulous-party-1654608598"), though you can specify a cluster name if you prefer. The rest of this document refers to the cluster as <cluster_name>.

2.) Create a namespace for the worker pods.: $ kubectl create namespace eks-roc-search-app
This namespace is used to simplify administering the cluster, but is optional. The configuration yaml files expect to operate in the eks-roc-search-app namespace. However, if operating in a separate namespace is not desired, the yaml files can be modified to remove the namespace, and this step will no longer need to be run.

3.) Deploy RabbitMQ to the cluster.

3a.) Install the RabbitMQ cluster operator.: $ kubectl krew install rabbitmq

3b.) Create a RabbitMQ Server for the cluster.: $ kubectl apply -f rabbitmq-cluster-deployment.yaml

3c.) Retrieve RabbitMQ Server login credentials.

$ new_username="$(kubectl -n eks-roc-search-app get secret roc-serve-rabbitmq-default-user -o jsonpath='{.data.username}' | base64 --decode)"
$ new_password="$(kubectl -n eks-roc-search-app get secret roc-serve-rabbitmq-default-user -o jsonpath='{.data.password}' | base64 --decode)"
$ new_ClusterIP="$(kubectl -n eks-roc-search-app get service roc-serve-rabbitmq -o jsonpath='{.spec.clusterIP}')"

Modify eks-roc-search-stateful-set.yaml:

Replace <username> in eks-roc-search-stateful-set.yaml with ${new_username}.
Replace <password> in eks-roc-search-stateful-set.yaml with ${new_password}.
Replace <rabbitmq_ip_address> in eks-roc-search-stateful-set.yaml with ${new_ClusterIP}.

4.) Create a Persistent Volume Claim.: $ kubectl apply -f claim.yaml

5.) Create a Storage Class.: $ kubectl apply -f storageclass.yaml

6.) Install Container Storage Interface (CSI) driver.

6a.) Get OpenID Connect (OIDC) Provider.: $ eksctl utils associate-iam-oidc-provider --cluster=${cluster_name} \
--approve

6b.) Create a CSI Driver role.

$ eksctl create addon --name aws-ebs-csi-driver \
  --cluster ${cluster_name} \
  --service-account-role-arn arn:aws:iam::${account_id}:role/AmazonEKS_EBS_CSI_DriverRole \
  --force

${cluster_name} is the cluster name created in Step 1.
${account_id} is the account ID linked to the IAM account used for setting up the cluster.

7.) Pull roc-serve container image.: Prerequisite: Contact suppo.nosp@m.rt@r.nosp@m.ankon.nosp@m.e.io to request an Account ID and Secret Access Key from Rank One Computing. The Account ID and Secret Access Key will be used to pull container images from a private Amazon Elastic Container Registry (ECR), created for your organization by Rank One Computing.

7a.) Pull the container image from the ${company_name} ECR Repository.: $ docker pull 576992771359.dkr.ecr.us-east-1.amazonaws.com/roc-search-${company_name}:latest

7b.) Push the image to a location accessible by Kubernetes, such as ECR or Docker Hub.: // Example push command:
$ docker push example-repo.amazonaws.com/repo-name:latest

7c.) Update eks-roc-search-stateful-set.yaml with the correct container repository.

Replace <company_name> in eks-roc-search-stateful-set.yaml with your company name.
Verify the "image:" URI in eks-roc-search-stateful-set.yaml matches the URI for the location you have stored the roc-serve container image.

7d.) Update eks-roc-search-http-deployment.yaml.

Replace <company_name> in eks-roc-search-http-deployment.yaml with your company name.
Verify the "image:" URI in eks-roc-search-http-deployment.yaml matches the URI for the location you have stored the roc-serve container image.
(See the values for username, password, and clusterIP from step 3.)
Replace <username> in eks-roc-http-deployment.yaml with ${new_username}.
Replace <password> in eks-roc-search-http-deployment.yaml with ${new_password}.
Replace <rabbitmq_ip_address> in eks-roc-search-http-deployment.yaml with ${new_ClusterIP}.

8.) Deploy the roc-serve stateful set.: $ kubectl apply -f eks-roc-search-stateful-set.yaml

9.) Deploy the roc-serve HTTP deployment.: $ kubectl apply -f eks-roc-search-http-deployment.yaml

10.) Expose the cluster HTTP interface.: $ kubectl apply -f eks-roc-search-service.yaml

Workflows

Scaleout

Scale up from three to five roc-serve worker pods.: $ kubectl -n eks-roc-search-app scale statefulset roc-serve-stateful-set --replicas=5

Scale down from five to three roc-serve worker pods.: $ kubectl -n eks-roc-search-app scale statefulset roc-serve-stateful-set --replicas=3

Upgrades

Monitoring

Snapshots

<This area is under construction.> Snapshots of gallery state are useful to perform rollbacks, manage upgrades, and accelerate the process of scaling out new worker pods.

Rollbacks