System Administration

Manage the TeamForge Orchestrate server-side components

System administration introduction

The scope of this guide is limited to key administrative features like starting/stopping services, log locations, etc. Administration and configuration of dependent services (e.g., nginx, MongoDB, RabbitMQ) is not covered; see respective product documentation. It is assumed that as TeamForge Orchestrate administrator you are familiar with basic Linux administration commands and terminology.

Background services

TeamForge Orchestrate installs and starts six background services on the App server. Each service is managed using init scripts, which are installed and started as part of the installation process. Init scripts are all located in /etc/init.d/ on the respective hosts. Once started, each service creates a pid file.

Init script PID file Host Description
collabnet-mongod /var/run/collabnet-mongodb/mongodb.pid DB server Database service
collabnet-nginx /var/run/collabnet-nginx/nginx.pid App server Web and application server
collabnet-rabbitmq-server /var/run/collabnet-rabbitmq/rabbitmq.pid MQ server AMQP message service
collabnet-redis /var/run/collabnet-redis/redis.pid App server Internal job queue service
orchestrate-ctf-jobs /var/run/orchestrate/ctf_job_runner.pid App server Background communication with TeamForge
orchestrate-queue-commit /var/run/orchestrate/scm_queue.pid App server Processes message from commit queue
orchestrate-queue-build /var/run/orchestrate/build_queue.pid App server Processes message from build queue
orchestrate-queue-review /var/run/orchestrate/review_queue.pid App server Processes message from review queue

Starting and stopping services

Use the init scripts to start, stop, restart, or get status on a service individually.
status
Is the process currently running? Does it have a PID (even if it's not running - e.g. in the case of a crash)
start
Starts the process
stop
Stop the process
restart
Stop and then start the process

Example usage:

sudo service orchestrate-queue-commit restart

Tip: While each service can be managed individually, the orchestrate init script will start and stop all App server services: /etc/init.d/orchestrate

Example usage:

sudo service orchestrate start

Service administration passwords

During the installation of TeamForge Orchestrate, if RabbitMQ and MongoDB were installed as part of the installation process then the administrative passwords have been written to disk for reference:

Log files

The following log files are relevant to TeamForge Orchestrate services and may be useful for debugging. Log rotation on all log files is enabled by default.

Orchestrate App server logs:
The server.log is the main log file for TeamForge Orchestrate, detailing the server's page serves and any errors that Orchestrate may encounter. It relates to the Phusion Passenger service, which is started with nginx.
/var/log/orchestrate/
  • build_queue.log
  • ctf_job_runner.log
  • nginx-error.log
  • nginx.log
  • review_queue.log
  • scm_queue.log
  • server.log
/var/log/collabnet-redis/
  • redis.log
MongoDB server log:
  • /var/log/collabnet-mongodb/mongodb.log
RabbitMQ server log:
/var/log/collabnet-rabbitmq/
  • rabbit@localhost.log
  • rabbit@localhost-sasl.log
  • startup_err
  • startup_log

Backup and restore

See Backup and Restore