API

class cotyledon.Service(worker_id)

Base class for a service

This class will be executed in a new child process/worker ServiceWorker of a ServiceManager. It registers signals to manager the reloading and the ending of the process.

Methods run(), terminate() and reload() are optional.

__init__(worker_id) → None

Create a new Service

Parameters:

worker_id (int) – the identifier of this service instance

The identifier of the worker can be used for workload repartition because it’s consistent and always the same.

For example, if the number of workers for this service is 3, one will got 0, the second got 1 and the last got 2. if worker_id 1 died, the new spawned process will got 1 again.

graceful_shutdown_timeout = None

Timeout after which a gracefully shutdown service will exit. zero means endless wait. None means same as ServiceManager that launch the service

name = None

Service name used in the process title and the log messages in additional of the worker_id.

reload() → None

Reloading of the service

This method will be executed when the Service receives a SIGHUP.

If not implemented the process will just end with status 0 and ServiceRunner will start a new fresh process for this service with the same worker_id.

Any exceptions raised by this method will be logged and the worker will exit with status 1.

run() → None

Method representing the service activity

If not implemented the process will just wait to receive an ending signal.

This method is ran into the thread and can block or return as needed

Any exceptions raised by this method will be logged and the worker will exit with status 1.

terminate() → None

Gracefully shutdown the service

This method will be executed when the Service has to shutdown cleanly.

If not implemented the process will just end with status 0.

To customize the exit code, the SystemExit exception can be used.

Any exceptions raised by this method will be logged and the worker will exit with status 1.

class cotyledon.ServiceManager(wait_interval=0.01, graceful_shutdown_timeout=60)

Manage lifetimes of services

ServiceManager acts as a master process that controls the lifetime of children processes and restart them if they die unexpectedly. It also propagate some signals (SIGTERM, SIGALRM, SIGINT and SIGHUP) to them.

Each child process (ServiceWorker) runs an instance of a Service.

An application must create only one ServiceManager class and use ServiceManager.run() as main loop of the application.

Usage:

class MyService(Service):
    def __init__(self, worker_id, myconf):
        super(MyService, self).__init__(worker_id)
        preparing_my_job(myconf)
        self.running = True

    def run(self):
        while self.running:
            do_my_job()

    def terminate(self):
        self.running = False
        gracefully_stop_my_jobs()

    def reload(self):
        restart_my_job()


class MyManager(ServiceManager):
    def __init__(self):
        super(MyManager, self).__init__()
        self.register_hooks(on_reload=self.reload)

        conf = {'foobar': 2}
        self.service_id = self.add(MyService, 5, conf)

    def reload(self):
        self.reconfigure(self.service_id, 10)

MyManager().run()

This will create 5 children processes running the service MyService.

__init__(wait_interval=0.01, graceful_shutdown_timeout=60) → None

Creates the ServiceManager object

Parameters:

wait_interval (float) – time between each new process spawn

add(service, workers=1, args=None, kwargs=None)

Add a new service to the ServiceManager

Parameters:

  • service (callable) – callable that return an instance of Service

  • workers (int) – number of processes/workers for this service

  • args (tuple) – additional positional arguments for this service

  • kwargs (dict) – additional keywoard arguments for this service

Returns:

a service id

Return type:

uuid.uuid4

reconfigure(service_id, workers) → None

Reconfigure a service registered in ServiceManager

Parameters:

  • service_id (uuid.uuid4) – the service id

  • workers (int) – number of processes/workers for this service

Raises:

ValueError

register_hooks(on_terminate=None, on_reload=None, on_new_worker=None, on_dead_worker=None) → None

Register hook methods

This can be callable multiple times to add more hooks, hooks are executed in added order. If a hook raised an exception, next hooks will be not executed.

Parameters:

  • on_terminate (callable()) – method called on SIGTERM

  • on_reload (callable()) – method called on SIGHUP

  • on_new_worker (callable(service_id, worker_id, exit_code)) – method called in the child process when this one is ready

  • on_new_worker – method called when a child died

If window support is planned, hooks callable must support to be pickle.pickle(). See CPython multiprocessing module documentation for more detail.

run() → None

Start and supervise services workers

This method will start and supervise all children processes until the master process asked to shutdown by a SIGTERM.

All spawned processes are part of the same unix process group.

cotyledon.oslo_config_glue.setup(service_manager, conf, reload_method='reload') → None

Load services configuration from oslo config object.

It reads ServiceManager and Service configuration options from an oslo_config.ConfigOpts() object. Also It registers a ServiceManager hook to reload the configuration file on reload in the master process and in all children. And then when each child start or reload, the configuration options are logged if the oslo config option ‘log_options’ is True.

On children, the configuration file is reloaded before the running the application reload method.

Options currently supported on ServiceManager and Service: * graceful_shutdown_timeout

Parameters:

  • service_manager (cotyledon.ServiceManager) – ServiceManager instance

  • conf (oslo_config.ConfigOpts()) – Oslo Config object

  • reload_method (str "reload/mutate") – reload or mutate the config files