Model processing converts the model definition file into a live, usable data model. To generate an actual database, data model, or machine learning model, this definition file must be executed. To produce or materialize a database, data model, or machine learning model, the definition file must be run. Once the model definition file is materialized, the live data model that is produced can be queried from the other Pyramid modules.
Model processing can also be scheduled on a regular basis to ensure the model is updated with the most recent data.
- Click here to learn how to schedule data model processing.
Processing Options dialog
- Open the model in the Advanced Data Flow and click Execute (Pyramid) from the top left-hand corner.
- Select the Reporting Action you require:
- Select the processing options to apply:
- Override Security: Override the metadata security set from the Admin Console or the Materialized Manager. Disable if metadata security should not be affected by processing the data model. Click here to learn more.
- Process Models Only: Only process the models. Do not process the Data Flow or Master Flow.
- Auto Sync Models: Synchronize the columns in the Data Model with the columns in the Data Flow; this is important where changes were made from the Data Flow.
- Validate Model Relationships: Check that the join definitions between tables are valid. The validation process may impact model processing time, so, although the validation is enabled by default, it can be turned off if needed.
- Once you have selected your options, click Master Flow (Green arrow above) to process the entire master flow and all of its components. If you haven't already saved the model definition file, you will be prompted to save before processing begins.
The Processing Options dialog opens:
Choose which module the new data model should be opened in: Discover, Smart Discover, Smart Publish, or Smart Present. The data model opens in the given reporting tool as soon as processing is complete. If you don't want any reporting tool to open automatically, select None.
Disabled Processing Options
Where there is no underlying model selected, the Processing Options (described above) may be disabled in this dialog:
- If you click Master Flow without making any changes, the database is generated. The Target is populated with content as described in the Data Flow.
- If you want to build a model, you need to close this dialog and click on the Data Model tab in the main view to create the model. When you re-open the dialog, all options will be available and clicking Master Flow will process the entire master flow.
Job Progress
You can view the progress of the job in the Job Spooler, which is opened from the App Tray at the bottom-right (yellow arrow below):
In this example, the Model job is highlighted in blue (above). Under the Actions column of the Model job, you can:
- Open the data model in Discover.
- Open the job's ETL Progress dialog.
- Remove the job from the Spooler.
Reprocess a Model
The model should be "reprocessed" whenever any changes are made to the Master Flow, Data Flow, Data Models, or Security. Changes made to the model definition file will only be seen in the materialized objects if the model is reprocessed.
Models should also be reprocessed to ensure they are kept updated with the latest data. You can reprocess a model manually, by processing from the Processing Options dialog (as described above). Model reprocessing can also be automated via model scheduling.
- Click here to learn about model processing on a schedule
Manage Security
Roles are assigned read and write permissions from the Security panel. These security permissions are saved to the model definition file, and automatically applied to the materialized database and data model once it is processed.
Security for materialized live databases and data models can then be managed in two places:
- Materialized Manager in the CMS: assign roles to servers, materialized live databases, and data models. Here you can assign read or write permissions only - you cannot manage metadata security and overlays.
- Source Manager in Admin: assign roles to servers, databases, and data models. Roles can also be assigned to metadata in each materialized live data model, enabling you to define which roles can and can't see given hierarchies, members, and measures. Assign given roles to hierarchy and member overlays.