File Sources

To connect to a file data source, add the required node to the canvas. From the Properties panel, select the way you want to connect to the file: upload the file, connect to a shared file, or connect to a URL.

Supported File Sources

Pyramid Content

  • There are two possible Pyramid "file" sources:
    • Discoveries built using Discover.
    • Tabulations built using Tabulate.
  • Connecting to Pyramid Content allows you to copy reports (discoveries or tabulations) that were built and saved in Pyramid and use them to build a new data model.
  • Click here to learn more about using a discovery as a data source.
  • Click here to learn more about using a tabulation as a data source.

Access

  • Supports local file upload, or pointer to a shared file on a network drive.
  • Each sheet within the given Access file is converted into a table.
  • Each column within the sheet is converted into a column within the corresponding table.
  • Click here to learn more about Access data sources.

Excel

  • Supports local file upload, or pointer to a shared file on a network drive, or URL address.
  • Each sheet within the given Excel file is converted into a table.
  • Each column within the sheet is converted into a column within the corresponding table.
  • Click here to learn more about Excel data sources.

JSON

  • Supports local file upload, or pointer to a shared file on a network drive, or URL address.
  • Each JSON file is converted into a single table.

 

  • Click here to learn more about JSON data sources.

Multi Files

  • Supports Text, JSON, XML, and Excel files.
  • Supports both local file upload and pointer to a shared file on a network drive.
  • To connect to a shared file, enter the extensions of the files to upload.
  • Each extension must be preceded by an asterisk, and multiple extensions must be delimited by a coma and then a space. For example: *.txt, *.csv.
  • Each file must have the same structure (columns) and file type (such as Text, JSON, or XML).

  • Note: this data source is not available in the Community Edition.

  • Click here to learn more about Multi Files data sources.

Google Sheets

  • Provide the URL link to the Google Sheet.
  • It's important that the sheet's sharing preferences are set to Anyone with the link to allow Pyramid to access it.
  • Click here to learn more about Google Sheets data sources.

Parquet

  • Supports local file upload, a pointer to a shared file on a network drive, or URL address.
  • Each Parquet file is converted into a single table.
  • Click here to learn more about Parquet data sources.

SAS Data Set

  • Supports local file upload, or pointer to a shared .sas7bdat file on a network drive, or URL address.
  • Click here to learn more about SAS Data Set data sources.

Text

  • Supports local file upload, a pointer to a shared file on a network drive.
  • Supports both Fixed Width Column files, and Delimited files.
  • Click here to learn more about Text file data sources.

XML

  • Supports local file upload, or pointer to a shared file on a network drive, or URL address.
  • Each XML file is converted into a single table.
  • Click here to learn more about XML data sources.

AI Data

  • Supports local file upload, a pointer to a shared file on a network drive, or URL address.
  • Each local file is supplied with a prompt. The LLM uses the prompt to extract the data from the file for use as source data.
  • Click here to learn more about AI Data sources.

Related information

Connection Type

For all the File data sources, you will need to establish a connection to the file.

Where there is no uploaded file, the Upload File fields are shown in the Properties panel. Start by selecting the method you want to use to provide the file.

Uploaded File

To upload a file, either drag and drop the file from its folder location onto the Properties panel or click Upload File and then find and select the file from its folder location.

Shared File

Provide a pointer to a shared file, rather than uploading the file into Pyramid. When the model is processed, the file is read and included in the model. The file path must include the file name and its extension.

  • Expression: If you want to build a dynamic file path to connect to your data source, you will need to select this checkbox and then use the PQL Editor to build your custom expression.

Note: The shared file must use a UNC file path format to the shared directory and file on the network. Pyramid's host machine must have access to the shared file location. If the Pyramid services are running on different machines, the Runtime and Task Servers must be given access to the shared file location.

URL

Provide a direct URL to a file hosted online or in a web-accessible location. The data will be downloaded from the URL when the model is processed. To connect to a URL, enter the URL and its required authentication.

  • Expression: If you want to build a dynamic URL to connect to your data source, you will need to select the Expression checkbox and then use the PQL Editor to build your custom expression.
  • Authentication:
    • None: Select None if no authorization is required.
    • Basic Authentication: Select if basic authentication is required.
    • Custom Header: Select if custom header authorization is required.
  • Custom HTTP Request Fields: Use this table to supply any custom headers you want to add to your HTTP request to send to the server.

Table Selection

From the Table Selection panel, select the tables that should be added to the Data Flow.