Installation and setup for Kubernetes

Before getting started, Delphix recommends engaging your Kubernetes Admin for deployment guidance on available Kubernetes deployment platforms (AKS, EKS, etc.). Configurations like node sizing and persistent volume settings will need to be determined.

Hardware requirements

The hardware requirements for Data Control Tower (DCT) on Kubernetes are listed below. In addition to these requirements, inbound port 443 must be open for API clients, and outbound port 443 to engines. This is the minimum total resource request for the Kubernetes deployment of DCT. Individual service-level resource requests are contained in the values.yaml file and can be overridden during deployment.

CPU: 4-Core
CPU architecture: x86_64
Memory: 16GB
Storage: 50GB
Port: 443

The recommended minimum 50 GB of storage is shared across the Kubernetes cluster (i.e. hosts). All pods and/or services use this storage for mounted volumes and other utilities, including image storage.

In a single node cluster, if shared volumes are not externalized, the host requires the full 50 GB of storage. If the persistent volume is mounted externally, the host requires 37 GB of storage, since the default storage required by the database (10 GB), gateway (1 GB), masking (1 GB), and virtualization-app (1 GB) draws from the external storage. The default storage configuration for the database, gateway, masking, and virtualization-app can be modified in values.yaml.

Many users may have default container settings as part of their Kubernetes or OpenShift infrastructure management. It is paramount to compare those default settings with the recommended minimum performance specifications (please engage your container infrastructure team to verify). If those default setting are lower, please update them to the minimum or higher.

For users who need to have limits set, you can start with the following sample configuration.

CODE

dataBookmarks:
  resources:
    requests:
      memory: "256Mi"
      cpu: "100m"
    limits:
      memory: "512Mi"
      cpu: "200m"
dataLibrary:
  resources:
    requests:
      memory: "256Mi"
      cpu: "100m"
    limits:
      memory: "512Mi"
      cpu: "200m"
database:
  resources:
    requests:
      memory: "256Mi"
      cpu: "200m"
    limits:
      memory: "1024Mi"
      cpu: "1100m"
gateway:
  resources:
    requests:
      memory: "512Mi"
      cpu: "400m"
    limits:
      memory: "1024Mi"
      cpu: "400m"
graphql:
  resources:
    requests:
      memory: "256Mi"
      cpu: "100m"
    limits:
      memory: "512Mi"
      cpu: "200m"
jobs:
  resources:
    requests:
      memory: "256Mi"
      cpu: "200m"
    limits:
      memory: "512Mi"
      cpu: "400m"
jobsCleaner:
  resources:
    requests:
      memory: "256Mi"
      cpu: "200m"
    limits:
      memory: "512Mi"
      cpu: "400m"
masking:
  resources:
    requests:
      memory: "512Mi"
      cpu: "500m"
    limits:
      memory: "1024Mi"
      cpu: "500m"
virtualizationApp:
  resources:
    requests:
      memory: "512Mi"
      cpu: "600m"
    limits:
      memory: "1024Mi"
      cpu: "600m"

Kubernetes overview

Data Control Tower can be deployed in a matter of minutes, once a Kubernetes cluster has been identified and deployment details have been aligned with your Kubernetes administrator. The installation consists of three components:

Kubernetes cluster: The identified infrastructure to which DCT will be deployed.
HELM: This deploys DCT as a Kubernetes application by referencing HELM charts (.yaml files) that make up the DCT install, either by an external Helm repository (https://dlpx-helm-dct.s3.amazonaws.com, this is the quickest path to installing DCT, as it largely automated) or via local install (this is accomplished by downloading the Helm charts directly via the DCT .tar file on download.delphix.com).
kubectl: Is a command line tool that enables administrative communication with the deployed pods (most useful post-deployment or after an upgrade).

Installation requirements (Kubernetes)

DCT requires a running Kubernetes cluster; This could be an on-premises cluster, Azure AKS or AWS EKS cluster. DCT also requires a kubectl command line tool to interact with Kubernetes cluster and HELM for deployment on to the cluster.

Requirement

DCT recommended version

Comments

Kubernetes Cluster

1.25 or above

HELM

3.9.0 or above

Install HELM as the package manager using the HELM installation article.

kubectl

1.25.0 or above

HELM will internally refer to the kubeconfig file to connect to the Kubernetes cluster. The default kubeconfig file is present at location: ~/.kube/config

If the kubeconfig file needs to be overridden while running HELM commands, set the KUBECONFIG environment variable to the location of the kubeconfig file.

To install kubectl follow the instructions at https://kubernetes.io/docs/tasks/tools/ .

Installing DCT

Add the DCT HELM repo with the following, which will link the DCT HELM repo to the local client HELM repo:

CODE

helm repo add dct-services https://dlpx-helm-dct.s3.amazonaws.com

Update the added repos with the following, which will update the local HELM charts metadata (index.yaml).

CODE

helm repo update

Pull the helm charts with the following, which is used to download the .tgz file.

CODE

helm pull dct-services/delphix-dct --version x.0.0

The downloaded file is then extracted using the following command (where x.0.0 should be changed to the version of DCT being installed):

CODE

tar -xvf delphix-dct-x.0.0.tgz

Update the following properties in values.yaml which is present in the extracted folder delphix-dct.

To generate the bootstrap APIKey, set apiKeyCreate: true.
Provide image credentials to pull images from docker registry.
- username: <retrieved from download.delphix.com>
- password: <retrieved from download.delphix.com>

For getting the image credentials, visit the Delphix DCT Download page and login with your customer login credentials. Once logged in, select the DCT Helm Repository link and accept the Terms and Conditions. Use password from the popup screen, as shown below.

OFFLINE MODE

Offline mode is supported for scenarios including:

Clusters that are air-gapped and cannot access the Delphix Docker registry.
Instances where Delphix Docker images must be pushed to an organization's internal registry for security scans or to comply with other security policies.

For these scenarios, a jumpbox machine with access to both the Delphix Docker registry or download site and the organization's internal registry is required. This jumpbox must also meet the installation prerequisites.

1. Get the Docker images

The two ways to get the Delphix Docker images are:

Pull the images from Delphix Docker registry.
Download and load the images from tarball.

2. Pull the images from Delphix Docker registry

Login into the jumpbox machine.

Pull the Docker images from the Delphix Docker registry.

CODE

$ docker pull dct.download.delphix.com/delphix-dct:nginx-x.0.0
$ docker pull dct.download.delphix.com/delphix-dct:app-x.0.0
$ docker pull dct.download.delphix.com/delphix-dct:data-bookmarks-x.0.0
$ docker pull dct.download.delphix.com/delphix-dct:delphix-data-library-x.0.0
$ docker pull dct.download.delphix.com/delphix-dct:graphql-x.0.0
$ docker pull dct.download.delphix.com/delphix-dct:ui-x.0.0
$ docker pull dct.download.delphix.com/delphix-dct:jobs-x.0.0
$ docker pull dct.download.delphix.com/delphix-dct:postgres-x.0.0
$ docker pull dct.download.delphix.com/delphix-dct:virtualization-x.0.0
$ docker pull dct.download.delphix.com/delphix-dct:masking-x.0.0
$ docker pull dct.download.delphix.com/delphix-dct:virtualization-app-x.0.0

Re-tag all above Docker images. Below is a sample re-tag example for one image:

CODE

$ docker tag dct.download.delphix.com/delphix-dct:nginx-x.0.0 <local registry url>:nginx-x.0.0

Note: Re-tag all remaining DCT images in the same format.

Push re-tagged images into the internal registry. Below is a ample push example for one image:
CODE
```
$ docker push <local registry url>:nginx-x.0.0
```

3. Download and load the images from tarball

If you do not have direct access to Delphix HELM and Docker repositories, download the HELM charts and Docker images packages as delphix-dct-x.0.0.tar.gz, which is uploaded on the download site.

Login into the jumpbox machine.
Download the HELM chart and Docker images packages as delphix-dct-x.0.0.tar.gz, which is uploaded on the download site.
Extract the downloaded .tar.gz file to get the HELM chart and Docker images.

Load extracted Docker images using following:

CODE

for image in *tar; do sudo docker load --input $image; done

Re-tag all above Docker images. Sample re-tag example for one image:

CODE

$ docker tag registry.delphix.com/delphix-dct:nginx-x.0.0 <local registry url>:nginx-x.0.0

Note: Re-tag all remaining DCT images in the same format.

Push re-tagged images into internal registry. Sample push example for one image:
CODE
```
$ docker push <local registry url>:nginx-x.0.0
```

4. Deploy

Update values.yaml for registry name and image credentials:
1. registry: <local registry url>
2. username: <local registry username>
3. password: <local registry password>
Install.
1. Follow the instructions below.

In version 8.0.0 and above, instead of username/password, an option is available to use credentials from a pre-existing Kubernetes Secret. To do so, instead of providing a username/password, users must create the Kubernetes Secret in the same namespace as the one used for DCT, and reference the registryKey as follows:

NONE

imageCredentials:
  # registry to pull docker images from.
  registry: dct.download.delphix.com/delphix-dct
  # username to login to docker registry. Do not set if registryKey is set.
  username:
  # password to login to docker registry. Do not set if registryKey is set.
  password:
  # Name of an existing docker registry key to use to pull images.
  registryKey: <insert-secret-name-here>

After updating the values.yaml, install it using the following command:

NONE

helm install dct-services delphix-dct

delphix-dct is the name of the folder which was extracted in the previous step. dct-services is the chart name which is given for this deployment.

In the above directory structure, the values.yaml file contains the properties which are configured above. If you want to configure additional properties in values.yaml, create a values.yaml using the sample below. Deploy DCT using following command to use the custom values.yaml:

helm install -f <path to edited values.yaml> dct-services <directory path of the extracted chart>

Sample values.yaml file

A sample values.yaml file can be downloaded below.

values.yaml

Once deployment is complete, check the status of the deployment using the following command (where X.0.0 should be changed to the version of DCT being installed):

NONE

helm list
NAME              NAMESPACE    REVISION    UPDATED                                 STATUS      CHART                    APP VERSION
dct-services      default      1           2023-01-10 19:33:41.713202 -0900        deployed    delphix-dct-x.0.0        x.0.0

Assuming an ingress controller configuration on the Kubernetes cluster is present, when accessing DCT after the deployment, the ingress controller rule needs to be added for proxy service, along with port 443 (if SSL is enabled) and port 80 (if SSL is disabled).