Sources

This is where you'll select your data source.

Click here for information about the available data sources.

If your data source is a file, it can be up uploaded (using the drag and drop function or the upload button), or it can be a shared file path (UNC path), or a URL (the data will be downloaded from the URL when the ETL is run).

Select a Data Source

Drag the relevant data source onto the canvas, and in the Properties panel:

• Select the server and database you want to query (blue highlight) - you can select the server from the drop down, or click the Add Server button to add a server.
  • Click the Refresh icon to refresh the server selection
  • Once the server has been selected, choose the database - click the Refresh icon to refresh the database selection
  • Alternatively, select Direct Query Data Source

Table Selection

• Select the tables you want to include in the data model - use the filter window to find tables quickly (green highlight)
  • Refresh the table selection - if tables have been added or removed from the datasource, click the refresh button (red highlight), so that the changes will be reflected in the table selection
• Add the designated tables as select nodes, or as a multi-select node (orange highlight below).
  • Select Nodes: a select node will be added for each selected table. Each table may then be previewed, and columns can be removed from given tables.
  • Multi-Select Nodes: one multi-select node will be added, containing all the selected tables
• A description may be added (yellow highlight)

When select nodes are added, click on a node to open its Preview panel.

Column Selection

Click on a select column node to open its Properties panel, where you can:

• Rename the table, under Resulting Table Name
• Replace the selected table node with a different table that wasn't previously selected (blue highlight)
• Select the columns you want to include, or change the column types, from the Column Selection drop down menu (green highlight)
• Refresh the columns in the selected table to update changes in the data source (indicated below in red)
• Add a description (orange highlight)

To remove a table from the data flow:

Select Node: select the relevant node and click Delete. For more information, see Data Sources.

Multi-Select Node: click the node and under Select Multiple in the Properties panel, deselect the relevant nodes.

Data Source Authentication

If you add a server, and the data source added is based on Windows Authentication, you may need to supply a set of credentials.

This often occurs when the authentication method for the data source is set to "End-user" Windows Authentication. Under this model, the user running the queries is used to connect (via Kerberos token) to the data source. However, when running a batch task (offline), the user's token is not available - as such a set of their credentials is required for connectivity by the Pyramid engine.

Note: This may not be required if the website authentication is via Basic or Forms.

Node Selection

From the data flow, you can left click on a node to select it, or left click and drag to move it. To select multiple node, drag select from the flow canvas, and include all nodes you want to select.

The selected nodes will be highlighted (red box below), and can then be deleted (green arrow).