LLM Manager

The LLM Manager allows admins to create and govern LLM Providers within Pyramid. The providers are used to generate images, scripts, excel formulas, translations, and to choose colors using AI. Providers can be added, removed, and edited from this page.

You should create or edit the LLM details in this page to reflect those details associated with the actual service or deployment that is already set up in advance. For more information about selecting the LLM that is most appropriate to your needs and setting it up ready for use, see Integrating LLMs with Pyramid.

LLM Manager

  • Open the LLM Manager from the AI and DSML section of the Admin Console.

This page contains the details of all current LLM providers at its top, and when you either click Add LLM Provider (yellow arrow) or select a provider or click Edit in the top-list (purple arrow) the Edit LLM panel opens at its bottom.

LLM Providers list

The LLM Provider Manager lists the details for the providers:

  • Name: The name of the instance; typically matches the provider name.
  • Type: The name of the vendor.
  • Endpoint: The endpoint defined for the LLM provider.

For each of the current providers administrators can click:

  • Edit: To open and edit the given provider details in the bottom panel.
  • Delete: To delete the selected provider.
  • Copy: To create a copy of the selected provider. You will be prompted to provide a new name for your newly created (copied) provider.
  • Audit Trail: To display an audit trail for the selected provider in a pop-up window including operation, performed by, date, and details.

To create a new provider, click Add LLM Provider above the LLM Providers list and specify its details in the Edit LLM panel.

Add and Edit LLM providers

Common settings

  • Vendor: Select the required vendor from the drop-down list.
  • Name: Provide a name for the LLM provider.

OpenAI

If OpenAI is selected, then enter the following details:

  • API Key: Enter your API key, which can be found in the Account Settings of your OpenAI account.

Azure AI

If Azure AI is selected, then enter the following details: 

  • API Key: Enter your API key, which can be found in your Azure portal in the Endpoint and Keys pane of the Resource Management section.
  • Endpoint: Enter your Azure Endpoint URL, which can be found in your Azure portal in the Endpoint and Keys pane of the Resource Management section.
  • Deployment: Enter the name of the relevant Azure Deployment Environment.

Google Gemini

If Google Gemini is selected, then enter the following details: 

  • Region: Select the region providing the service from the drop-down.
  • Model Id: Enter the model ID for the provider.
  • Project Id: Enter the project ID for the provider.
  • Client Id: Enter the client ID for the provider.
  • Client Secret: Enter the client secret for the provider.
  • Click Sign-In with Google. The OAuth refresh code field is populated in response to this action.

Mistral

If Mistral is selected, then enter the following details:

  • API Key: Enter your API key.

IBM Watsonx

If IBM Watsonx is selected, then enter the following details: 

  • Region: Select the region providing the IBM Watsonx service from the drop-down.
  • Project Id: Enter the project ID for the provider.
  • Model ID: Enter the model ID for the provider.

WatsonX needs to be licensed separately. Click here for more or contact sales.

Test and save your settings

  • Click Test to test the LLM provider details in the Edit LLM panel. Note: This test will fail if the main Generative AI and Natural Language Querying option is not enabled at the top of the AI Settings panel.
  • Click Save or Apply to save your changes. Save is offered when you are creating a new LLM provider, and Apply when you are editing details of an existing LLM Provider.