etl-client.py

The etl-client.py script allows you to access the Extract, Transform and Load (ETL) scheduler and check the status of the jobs configured. The script also supports triggering jobs manually.

Parameters

The following parameters are available for the etl-client.py script:

-s | --status
Prints the status of all the jobs configured in the ETL service.
-a | --status-all
Prints the status of incremental and historical jobs configured in the ETL service.
-v| --verbose
Chronicles the process of requested operation a bit more.
-r| --run=
Triggers a job manually for a given job.

Data collection (ETL) jobs supported by the ETL service

While some ETL jobs are scheduled to run automatically, some must be triggered manually. The following table lists the available ETL jobs in TeamForge.

Job Category Job Name Description

Historical Data Collection

Historical data collection jobs must be triggered manually. As a best practice, these jobs are run as part of post migration activities. Refer to the "Related Links" for more information.

SCMCommitInitialJob Collects the historical commit data from TeamForge.
TrackerInitialJob Collects the historical data of artifacts from TeamForge.
LoadFlexFields Collects the historical data of custom-defined artifacts from TeamForge.

Incremental Data Collection

Incremental data collection jobs collect data that are added or modified incrementally on an ETL run-to-run basis. These jobs are scheduled to run automatically on a regular basis.

SCMCommitActivityJob Collects the SCM commit data incrementally on an ETL run-to-run basis.
TrackerIncrementalJob Collects the tracker artifacts data incrementally on an ETL run-to-run basis.
UserLoginActivityJob Collects the user logon activity data incrementally on an ETL run-to-run basis.

Imported Data Collection (Simbel)

TeamForge supports bulk data import through Simbel. This job collects Simbel-imported data. This job must be triggered manually post data import into TeamForge.

SimbelImportJob Collects all Simbel-imported data such as the user logon activity, SCM commit and tracker artifacts data.