Pyramid on AWS Kubernetes

The following guide provides a broad outline of the steps to deploy Pyramid as a Kubernetes cluster using Docker containers on Amazon's Elastic Kubernetes Service or EKS. Although this guide is written for AWS, Pyramid's containers, and core K8s templates, can be equally deployed into environments on Azure, GCP, Oracle and Alibaba. It can also work equally well on frameworks like RedHat OpenShift or Google Anthos. Its also possible to construct and execute your own mechanics for deploying Pyramid containers.

The guide below is NOT meant to be an exhaustive specification of the steps for setting up EKS.

Pyramid can be installed through a 'native' (or direct) installation to Windows or Linux host machines (in any venue: on-premise, or cloud). The Kubernetes (K8s) deployment option, however, offers a super convenient mechanism to easily deploy a cluster of Pyramid services that take advantage of cloud computing. More specifically, it provides a key mechanism to an elastic analytic solution computing using Pyramid - where the resources powering the solution can be easily ramped up or down based on demand.

Ingredients

Before you begin deploying Pyramid into K8s, you should have the following items ready to go for the steps in this guide:

  • An account on AWS, with administrative rights to create and deploy an RDS database and EKS cluster.
  • Details for the AWS programmatic access (AWS access Key ID, AWS Secret Access Key)
  • Details of your AWS S3 buckets (AWS access Key ID, AWS Secret Access Key)
  • Access to Pyramid's Kubernetes Configurator tool in the online customer portal.
  • Optionally, it is useful to have a Pyramid license key ready. The key needs to be an "Enterprise Edition" key (trial or otherwise).

If you know what you are doing, the entire process takes around 30 minutes to configure AWS (including waiting time); 10-15 minutes to setup utilities; and another 10 minutes to build a Kubernetes template and to deploy and launch Pyramid.

The guide is written for admins working off a Windows PC. They can easily be adapted to Linux.

Deployment Steps

1. AWS Prep

For AWS prep steps, login into AWS Management Console.

A. Setup RDS

Pyramid requires a persistent data repository. Although you can use any database accessible to the cluster (hosted on AWS or elsewhere), the simplest is to use a PostgreSQL RDS database on AWS itself.

  1. Open RDS.
  2. Create a database (either PostgreSQL, MS SQL Server or Oracle).
    1. Provide details like instance identifier, master username and password.
    2. For instance size, use 4 vCPU's and 8GB memory machine as a start. You may have to increase this for much larger deployments.
    3. Ensure Manage master credentials AWS Secret Manager is disabled.
    4. Important: Under "Additional Connectivity Configuration" you might have to set the database to "Public Access" so the database is accessible from your EKS cluster.

After triggering the creation, it might take around five minutes before the database is provisioned and ready for use. Continue to the next steps while you wait.

B. Setup EKS Cluster

Next, without waiting for the database to provision, go back to the management console to add EKS.

  1. Open EKS.
  2. Create the EKS cluster by providing a name:
    1. Set the K8s version (1.26 was used at the time of writing).
    2. In the next screen, choose a VPC and subnets and set Security Groups.
    3. Set end point access to Public if you want to manage the cluster from outside AWS.
  3. Choose Next, and finally Create.
  4. Once the cluster is created, configure each subnet that is defined on the cluster, and add a tag to allow the load balancer to assign an external IP address.
  5. The tag should be in the following format:

    Key:

    kubernetes.io/cluster/

    Value:

    shared
Add Compute Nodes

After the cluster is provisioned (around 5 minutes), we need to add "nodes". These are effectively the 'equipment' behind the EKS cluster.

  1. Inside the cluster settings, choose "Compute" :
    1. Add a node group.
    2. Give the group a name.
    3. Set the IAM role.
    4. Click next, and then choose the type of machine for the node:
      1. Ensure the machine has the horsepower to drive the cluster. You should not choose anything less than 12 cores and 16 Gb of memory.
      2. You can use two or more nodes to create a group cluster of resources.
        • Specify the number of nodes to create
    5. In Specify networking, enable the flag "configure remote access to nodes".
    6. Specify the EC2 key pair.
  2. Create the node group and nodes.

The nodes will also take five minutes to provision.

2. Kubernetes Utilities Setup

The following steps can be executed while you're waiting for the various elements in AWS to be provisioned. However, to finish them, the EKS cluster must be ready.

A. AWS "CLI" Setup

Download and install the AWS command line interface (cli) tool.

  1. For Windows, download and install the cli: https://awscli.amazonaws.com/AWSCLIV2.msi
  2. Note: You can do the same using the built in AWS CloudShell

  3. Once installed, open a command line session with administrative privileges. Type:
     "aws configure"
    1. Supply your AWS access Key ID
    2. Next your AWS Secret Access Key
    3. Supply the region (make sure it matches the region of the EKS deployment)
    4. Set output to JSON
    5. Hit ENTER

Once the EKS cluster is ready, you need to run the following command in the command line session. Be sure to swap out the region and the cluster name you set above in the EKS stages (1B).

aws eks --region region update-kubeconfig --name cluster_name

B. KubeCTL Setup

Download the "KubeCTL" utility.

  1. For Windows download and store in a folder: https://storage.googleapis.com/kubernetes-release/release/v1.19.0/bin/windows/amd64/kubectl.exe

    Note: You can use the KubeCTL that is built-in the AWS CloudShell

    • In that folder, right + SHIFT click and open a command line session from the folder itself.
  2. Once the EKS cluster is ready, and you have finished the AWS CLI steps above, you can continue with the following steps. At the prompt type:
     kubectl cluster-info
    • If successful, it will return the details of our AWS cluster and print to screen the configuration of the EKS cluster.
  3. Keep the command line session open for the next stage.

3. Pyramid K8s Configurator

After all the above steps have been completed, we are ready to define the Pyramid cluster and deploy it. (More details on this form area available here)

  1. Login to Pyramid's Customer Portal from the main website.
  2. From the dashboard, choose Kubernetes Setup.
  3.  In the form:
    1. Choose which version of Pyramid to deploy.
    2. Supply the initial size of your nodes in the K8s cluster in EKS (memory and CPU)
    3. Provide the internal web port number used for connecting to the service below.
    4. Set the number of initial pods to deploy when the cluster is initialized. (Two routers are recommended).
    5. Decide which services will automatically scale and the maximum number of pods to scale up to.
  4. Once done, click the button to generate the YAML instruction file. Save the file to a destination on your workstation for the next step.

    Note: To upload the YAML file using the AWS CloudShell, click actions > upload file.

If you subsequently decide to change the configuration of your cluster, simply reopen the tool to generate a new YAML, then repeat the cluster deployment below.

4. Cluster Deployment

Note: Your pods should have internet access for functional operations during the installation process. If you need to install without internet access, please contact support for more assistance.

Using the KubeCTL command line session (from above), execute the following command using the YAML file from step 3 (above):

kubectl apply -f pathtoYamlfile

Once executed, the Pyramid cluster will be created on the EKS. In the command line window, a bunch of details on the deployment will be printed to screen. This usually takes around 1-3 minutes.

A. Pod creation

To see the pods lighting up type:

kubectl get pods -n pyramid

Repeat the above commands to see the pod creation stages. Once all the pods are "running", the cluster is fully operational and ready for initialization (see below).

Warning: If you see some pods are not being created successfully, it usually points to a lack of resources in the cluster. You may need to enlarge the cluster or reduce the pod resource allocations. It also possible, in a multi-node deployment, that Kubernetes does not make the most intelligent allocation of pods to each node.

B. Service Address

To see the Pyramid service and retrieve the public web address for the cluster, type:

kubectl get svc -n pyramid

Grab the public web address so you can continue with initialization below.

Note: If the external IP is "pending," please add a tag to your subnet.

The tag should contain the following:

Key:

kubernetes.io/cluster/

Value:

shared

After you add the tag, remove the svc, and re-add it again by applying the YAML:

kubectl delete svc pyramid -n pyramid

5. System Initialization

The following steps need to be completed the first time a cluster is deployed. If the cluster is upgraded or changed, the initialization is not required again.

Go to a browser and put in the service web address found in the steps above (4.B.). You may need to append the port assignment too (3.3.3 above):

http://webUrlAddress.aws.com:port

If the services are up, you will be prompted to fill in the initialization form. Details on the form itself can be found in here. Details on an unattended deployment of these details via YAML can be found here.

  1. Supply the details for the RDS database you created earlier.
  2. Supply the details for the AWS S3 Bucket storage.
  3. Supply the username and password of the initial system user.
  4. If you have a Pyramid license file, upload and attach it to the form.
  5. Click Run Setup.

After about 2-3 minutes, the system will be initialized, and you will be bounced directly into the application.