Scripting Environments

The Scripting Environments manager allows admins to create and govern multiple virtual environments for Python and managed environments for R. The environments are used to stage and run Python and R scripting in Model, Discover and Formulate. Environments can be added, removed, and edited.

Why Use Environments

Versioning

Both Python and R are scripting languages. Neither has inherent versioning switches and both use fixed version 3rd party packages. As such, it is extremely complicated to create and run different Python or R programs from the same setup, which each program may use a different version of the language engines AND different versions of the underlying 3rd party packages.

Pyramid solves this issue by crating and maintaining multiple environments:

  • Python - multiple virtual environments can be created and deployed to the one or more AI servers. Different versions of Python CAN coexist on the same host, so its possible to deploy multiple coexisting versions.
  • R - a managed environment can be created and deployed to each AI server (or duplicated across multiple servers). Different versions of R CANNOT coexist on the same host, so while there can be multiple environment definitions of R in Pyramid, each one can only be deployed to any given host at any one time.

Scale Out and Load Balancing

Each environment can be hosted on multiple AI servers so the mechanism also effectively provides an load balancing and resource allocation framework for Python and R processing as well.

Environments Manager

The Environments manager lists the current environments:

  • Type: Python or R
  • Status: the status of the environment; green for installed, yellow for installing / downloading packages, and red for unusable.
  • Environment: the name of the environment.
  • Version: the Python or R version.
  • Servers: the list of AI servers on which the environment is hosted.

For each environment administrators can:

  • Edit: opens and edits the given environment in the bottom panel.
  • Hosted Servers: set and edit which AI servers to host the environment on.
  • Reinstall: this option is enabled if the environment is unstable; for instance, if a package failed to download, or a server crashed. In this scenario, the environment may be reinstalled.
  • Delete: deletes the selected environment.

The "Pyramid Python Environment" and "Pyramid R Environment" are created during installation and may not be edited or deleted. However, they can be moved to operate on a different AI server(s).

Add a Virtual Environment

To add an environment click "Add Environment" or "Add Existing Environment" buttons. The Environment Settings panel is opened by adding an environment or editing an existing environment

Environment Settings

  • Secure: if the environment is going to be enabled for certain tenants only, check the box and choose which tenants will have access to the environment.
  • Name: provide a name for the environment.
  • Type: select either R or Python
  • Version: select the appropriate version for the given type.
  • Description: add a description if required.
  • Server Coverage :This setting is not visible unless the automatic "Percentage" scaling model is chosen. If provided you must set the percent "coverage" that this environment will have on all AI servers in the cluster. Setting to 100%, means the environment is deployed to all machines. 50% means its deployed to half the servers etc.

Packages

  • Add Package:
    • Name: the name of the package you want to download.
    • Version: enter the Python version required. If this field is left blank, the latest version will be installed.
    • Package Manager: download package from Pip or Conda.

Package Listing

The Environment Settings panel lists all the packages downloaded to the given environment, along with the package version, package manager, and any addition information.

From here you can update a package version or remove or download a package.

  • Actions: edit (the name, version, or package manager) or delete a package.
  • Packages: package name.
  • Version: package version.
  • Package Manager: whether the package was downloaded from Pip or Conda.
  • User Defined: if yes, the package was imported by a user; if no, the it's a dependency package or a system package. For instance, if the user imported numpy, the numpy package is user defined; the setup tools package is a dependency package; DateTime is a system package.
  • Info: indicates any additional information about the package.

Hosted Servers

This tab is visible if the manual scaling model is chosen. Then, admins must manually allocate each environment to each AI server in the cluster. Scripting environments may be installed on multiple servers.

By default, the environment will be hosted on the tenant's default AI server.

Finish

Once you've selected the required packages, click the Add button to complete installation