class Dpl::Provider

Base class for all concrete providers that ‘dpl` supports.

These are subclasses of ‘Cl::Cmd` which means they are going to be detected by the first argument passed to `dpl [provider]`, instantiated, and run.

Implementors are encouraged to use the provider DSL to declare various features, requirements, and attributes that apply to their provider, to implement any of the following stages (methods) according to their needs and semantics:

* init
* install
* login
* setup
* validate
* prepare
* deploy
* finish

The main logic should sit in the ‘deploy` stage.

If at any time the method ‘error` is called, or any exception raised the deploy process will be halted, and subsequent stages skipped. However, the stage `finish` will run even if previous stages have raised an error, giving the provider the opportunity to potentially clean up stage.

In addition to this the following methods will be called if implemented by the provider:

* run_cmd
* add_key
* remove_key

Like the ‘finish` stage, the method `remove_key` will be called even if previous stages have raised an error.

See the respective method’s documentation for details on these.

The following stages are not meant to be overwritten, but considered internal:

* before_install
* before_setup
* before_prepare
* before_finish

Dependencies declared as required, such as APT, NPM, or Python are going to be installed as part of the ‘before_install` stage .

Cleanup is run as part of the ‘before_prepare` stage if the option `–cleanup` was given. This will use `git stash –all` in order to reset the working directory to the committed state, and cleanup any left over artifacts from the build process. Providers can use the DSL method `keep` in order to declare known artifacts (such as CLI tooling installed to the working directory) that needs to be moved out of the way and restored after the cleanup process. (It is recommended to place such artifacts outside of the build working directory though, for example in `~/.dpl`).

The method ‘run_cmd` is called for each command specified using the `–run` option. By default, these command are going to be run as local shell commands, but providers can choose to overwrite this method in order to run the command on a remote machine.

@see github.com/svenfuchs/cl Cl’s documentation for details on how providers (commands) are declared and run.

Constants

FOLDS

Fold names to display in the build log.

STAGES

Deployment process stages.

In addition to the stages listed here the stage ‘finish` will be run at the end of the process.

Also, the methods ‘add_key` (called before `setup`), `remove_key` (called before `finish`), and `run_cmd` (called after `deploy`) may be of interest to implementors.

Attributes

key_name[R]
repo_name[R]

Public Class Methods

examples() click to toggle source
Calls superclass method
# File lib/dpl/provider.rb, line 88
def examples
  @examples ||= super || Examples.new(self).cmds
end
install_deps(ctx) click to toggle source
# File lib/dpl/provider.rb, line 104
def install_deps(ctx)
  ctx.apts_get(apt) if apt?
  ctx.gems_require(gem) if gem?
  npm.each { |npm| ctx.npm_install *npm } if npm?
  pip.each { |pip| ctx.pip_install *pip } if pip?
end
install_deps?() click to toggle source
# File lib/dpl/provider.rb, line 100
def install_deps?
  apt? || gem? || npm? || pip?
end
move_files(ctx) click to toggle source
# File lib/dpl/provider.rb, line 92
def move_files(ctx)
  ctx.move_files(move) if move.any?
end
new(ctx, *args) click to toggle source
Calls superclass method
# File lib/dpl/provider.rb, line 193
def initialize(ctx, *args)
  @repo_name = ctx.repo_name
  @key_name = ctx.machine_name
  super
end
unmove_files(ctx) click to toggle source
# File lib/dpl/provider.rb, line 96
def unmove_files(ctx)
  ctx.unmove_files(move) if move.any?
end
validate_runtimes(ctx) click to toggle source
# File lib/dpl/provider.rb, line 111
def validate_runtimes(ctx)
  ctx.validate_runtimes(runtimes) if runtimes.any?
end

Public Instance Methods

before_finish() click to toggle source

Finalizes the deployment process.

This will:

  • Call the method ‘remove_key` if implemented by the provider, and if the feature `ssh_key` has been declared as required.

  • Revert the cleanup process, i.e. restore files moved out of the way during ‘cleanup`.

  • Remove the temporary directory ‘~/.dpl`

# File lib/dpl/provider.rb, line 312
def before_finish
  remove_key if needs?(:ssh_key) && respond_to?(:remove_key)
  uncleanup if cleanup?
  unmove_files(ctx)
  remove_dpl_dir
end
before_init() click to toggle source

Initialize the deployment process.

This will:

  • Displays warning messages about the provider’s maturity status, and deprecated options used.

  • Setup a ~/.dpl working directory

  • Move files out of the way that have been declared as such

# File lib/dpl/provider.rb, line 247
def before_init
  warn status.msg if status && status.announce?
  deprecations.each { |(key, msg)| ctx.deprecate_opt(key, msg) }
  setup_dpl_dir
  move_files(ctx)
end
before_install() click to toggle source

Install APT, NPM, and Python dependencies as declared by the provider.

# File lib/dpl/provider.rb, line 255
def before_install
  validate_runtimes(ctx)
  return unless install_deps?
  info :before_install
  install_deps(ctx)
end
before_prepare() click to toggle source

Prepares the deployment by cleaning up the working directory.

@see Provider#cleanup

# File lib/dpl/provider.rb, line 280
def before_prepare
  cleanup if cleanup?
end
before_setup() click to toggle source

Sets the build environment up for the deployment.

This will:

  • Setup a ~/.dpl working directory

  • Create a temporary, per build SSH key, and call ‘add_key` if the feature `ssh_key` has been declared as required.

  • Setup git config (email and user name) if the feature ‘git` has been declared as required.

  • Either set or unset the environment variable ‘GIT_HTTP_USER_AGENT` depending if the feature `git_http_user_agent` has been declared as required.

# File lib/dpl/provider.rb, line 270
def before_setup
  info :before_setup
  setup_ssh_key if needs?(:ssh_key)
  setup_git_config if needs?(:git)
  setup_git_http_user_agent
end
chmod(perm, path) click to toggle source
Calls superclass method
# File lib/dpl/provider.rb, line 625
def chmod(perm, path)
  super(perm, expand(path))
end
cleanup() click to toggle source

Resets the current working directory to the commited state.

Cleanup will use ‘git stash –all` in order to reset the working directory to the committed state, and cleanup any left over artifacts from the build process. Providers can use the DSL method `keep` in order to declare known artifacts (such as CLI tooling installed to the working directory) that needs to be moved out of the way and restored after the cleanup process.

# File lib/dpl/provider.rb, line 327
def cleanup
  info :cleanup
  keep.each { |path| shell "mv ./#{path} ~/#{path}", echo: false, assert: false }
  shell 'git stash --all'
  keep.each { |path| shell "mv ~/#{path} ./#{path}", echo: false, assert: false }
end
compact(hash) click to toggle source

Compacts the given hash by rejecting nil values.

# File lib/dpl/provider.rb, line 596
def compact(hash)
  hash.reject { |_, value| value.nil? }
end
escape(str) click to toggle source

Escapes the given string so it can be safely used in Bash.

# File lib/dpl/provider.rb, line 550
def escape(str)
  Shellwords.escape(str)
end
expand(*args) click to toggle source
# File lib/dpl/provider.rb, line 645
def expand(*args)
  File.expand_path(*args)
end
file?(path) click to toggle source
# File lib/dpl/provider.rb, line 617
def file?(path)
  File.file?(expand(path))
end
finish?() click to toggle source
# File lib/dpl/provider.rb, line 220
def finish?
  stage.size == STAGES.size
end
fold(name, opts = {}) { || ... } click to toggle source

Creates a log fold.

Folds any log output from the given block into a fold with the given name.

# File lib/dpl/provider.rb, line 415
def fold(name, opts = {}, &block)
  return yield unless fold?(name, opts)
  title = FOLDS[name] || "deploy.#{name}"
  ctx.fold(title, &block)
end
fold?(name, opts = {}) click to toggle source

Checks if the given stage needs to be folded.

Depends on the option ‘–fold`, also omits folds for the init and finish stages. Can be overwritten by passing `fold: false`.

Calls superclass method
# File lib/dpl/provider.rb, line 425
def fold?(name, opts = {})
  !opts[:fold].is_a?(FalseClass) && super() && !%i(init).include?(name)
end
mkdir_p(path) click to toggle source
# File lib/dpl/provider.rb, line 621
def mkdir_p(path)
  FileUtils.mkdir_p(expand(path))
end
mv(src, dest) click to toggle source
Calls superclass method
# File lib/dpl/provider.rb, line 629
def mv(src, dest)
  super(expand(src), expand(dest))
end
only(hash, *keys) click to toggle source

Returns a new hash with the given keys selected from the given hash.

# File lib/dpl/provider.rb, line 601
def only(hash, *keys)
  hash.select { |key, _| keys.include?(key) }
end
open(path, *args, &block) click to toggle source
# File lib/dpl/provider.rb, line 637
def open(path, *args, &block)
  File.open(expand(path), *args, &block)
end
opt_for(key, opts = {}) click to toggle source
# File lib/dpl/provider.rb, line 583
def opt_for(key, opts = {})
  case value = send(key)
  when String then "#{opt_key(key, opts)}=#{value.inspect}"
  when Array  then value.map { |value| "#{opt_key(key, opts)}=#{value.inspect}" }
  else opt_key(key, opts)
  end
end
opt_key(key, opts) click to toggle source
# File lib/dpl/provider.rb, line 591
def opt_key(key, opts)
  "#{opts[:prefix] || '--'}#{opts[:dashed] ? key.to_s.gsub('_', '-') : key}"
end
opts_for(keys, opts = {}) click to toggle source

Generate shell option strings to be passed to a shell command.

This generates strings like ‘–key=“value”` for the option keys passed. These keys are supposed to correspond to methods on the provider instance, which will be called in order to determine the option value.

If the returned value is an array then the option will be repeated multiple times. If it is a String then it will be double quoted. Otherwise it is assumed to be a flag that does not have a value.

@option prefix [String] Use this to set a single dash as an option prefix (defaults to two dashes). @option dashed [Boolean] Use this to dasherize the option key (rather than underscore it, defaults to underscore).

# File lib/dpl/provider.rb, line 578
def opts_for(keys, opts = {})
  strs = Array(keys).map { |key| opt_for(key, opts) if send(:"#{key}?") }.compact
  strs.join(' ') if strs.any?
end
quote(str) click to toggle source

Double quotes the given string.

# File lib/dpl/provider.rb, line 555
def quote(str)
  %("#{str.to_s.gsub('"', '\"')}")
end
read(path) click to toggle source
# File lib/dpl/provider.rb, line 641
def read(path)
  File.read(expand(path))
end
remove_dpl_dir() click to toggle source

Remove the internal working directory ‘~/.dpl`.

# File lib/dpl/provider.rb, line 347
def remove_dpl_dir
  rm_rf '~/.dpl'
end
rm_rf(path) click to toggle source
Calls superclass method
# File lib/dpl/provider.rb, line 633
def rm_rf(path)
  super(expand(path))
end
run() click to toggle source

Runs all stages, all commands provided by the user, as well as the final stage ‘finish` (which will be run even if an error has been raised during previous stages).

# File lib/dpl/provider.rb, line 202
def run
  stages = stage.select { |stage| run_stage?(stage) }
  stages.each { |stage| run_stage(stage) }
  run_cmds
rescue Error
  raise
rescue Exception => e
  raise Error.new("#{e.message} (#{e.class})", backtrace: backtrace? ? e.backtrace : nil) unless test?
  raise
ensure
  run_stage(:finish, fold: false) if finish?
end
run_cmd(cmd) click to toggle source
# File lib/dpl/provider.rb, line 299
def run_cmd(cmd)
  cmd.downcase == 'restart' ? restart : shell(cmd)
end
run_cmds() click to toggle source

Runs each command as given by the user using the ‘–run` option.

For a command that matches ‘restart` the method `restart` will be called (which can be overwritten by providers, e.g. in order to restart service instances).

All other commands will be passed to the method ‘run_cmd`. By default this will be run as a shell command locally, but providers can choose to overwrite this method in order to run the command on a remote machine.

# File lib/dpl/provider.rb, line 293
def run_cmds
  Array(opts[:run]).each do |cmd|
    cmd.downcase == 'restart' ? restart : run_cmd(cmd)
  end
end
run_stage(stage, opts = {}) click to toggle source

Runs a single stage.

For each stage the base class has the opportunity to implement a ‘before` stage method, in order to apply default behaviour. Provider implementors are asked to not overwrite these methods.

Any log output from both the before stage and stage method is going to be folded in the resulting build log.

# File lib/dpl/provider.rb, line 232
def run_stage(stage, opts = {})
  fold(stage, opts) do
    send(:"before_#{stage}") if respond_to?(:"before_#{stage}")
    send(stage) if respond_to?(stage)
  end
end
run_stage?(stage) click to toggle source

Whether or not a stage needs to be run

# File lib/dpl/provider.rb, line 216
def run_stage?(stage)
  respond_to?(:"before_#{stage}") || respond_to?(stage)
end
script(name, opts = {}) click to toggle source

Runs a script as a shell command.

Scripts can be stored as separate files (assets) in the directory ‘lib/dpl/assets/`.

This is meant for large shell commands that would be hard to read if embedded in Ruby code. Storing them as separate files helps with proper syntax highlighting etc in editors, and allows to execute them for testing purposes.

Scripts can have interpolation variables. See Dpl::Interpolate for details on interpolating variables.

See Ctx::Bash#shell for details on the options accepted.

# File lib/dpl/provider.rb, line 443
def script(name, opts = {})
  opts[:assert] = name if opts[:assert].is_a?(TrueClass)
  shell(asset(name).read, opts.merge(echo: false))
end
setup_dpl_dir() click to toggle source

Creates the directory ‘~/.dpl` as an internal working directory.

# File lib/dpl/provider.rb, line 340
def setup_dpl_dir
  rm_rf '~/.dpl'
  mkdir_p '~/.dpl'
  chmod 0700, '~/.dpl'
end
setup_git_config() click to toggle source

Setup git config

This adds the current user’s name and email address (as user@localhost) to the git config.

# File lib/dpl/provider.rb, line 368
def setup_git_config
  shell "git config user.email >/dev/null 2>/dev/null || git config user.email `whoami`@localhost", echo: false, assert: false
  shell "git config user.name  >/dev/null 2>/dev/null || git config user.name  `whoami`", echo: false, assert: false
end
setup_git_http_user_agent() click to toggle source

Sets or unsets the environment variable ‘GIT_HTTP_USER_AGENT`.

# File lib/dpl/provider.rb, line 389
def setup_git_http_user_agent
  return ENV.delete('GIT_HTTP_USER_AGENT') unless needs?(:git_http_user_agent)
  info :setup_git_ua
  ENV['GIT_HTTP_USER_AGENT'] = user_agent(git: `git --version`[/[\d\.]+/])
end
setup_git_ssh(key) click to toggle source

Sets up ‘git-ssh` and the GIT_SSH env var

# File lib/dpl/provider.rb, line 374
def setup_git_ssh(key)
  info :setup_git_ssh
  path, conf = '~/.dpl/git-ssh', asset(:dpl, :git_ssh).read % expand(key)
  open(path, 'w+') { |file| file.write(conf) }
  chmod(0740, path)
  ENV['GIT_SSH'] = expand(path)
end
setup_ssh_key() click to toggle source

Creates an SSH key, and sets up git-ssh if needed.

This will:

  • Create a temporary, per build SSH key.

  • Setup a ‘git-ssh` executable to use that key.

  • Call the method ‘add_key` if implemented by the provider.

# File lib/dpl/provider.rb, line 358
def setup_ssh_key
  ssh_keygen(key_name, '~/.dpl/id_rsa')
  setup_git_ssh('~/.dpl/id_rsa')
  add_key('~/.dpl/id_rsa.pub') if respond_to?(:add_key)
end
shell(cmd, *args) click to toggle source

Runs a single shell command.

Shell commands can have interpolation variables. See Dpl::Interpolate for details on interpolating variables.

See Ctx::Bash#shell for details on the options accepted.

# File lib/dpl/provider.rb, line 454
def shell(cmd, *args)
  opts = args.last.is_a?(Hash) ? args.pop : {}
  cmd = Cmd.new(self, cmd, opts)
  ctx.shell(cmd)
end
sq(str) click to toggle source

Outdents the given string.

@see Dpl::Squiggle

# File lib/dpl/provider.rb, line 562
def sq(str)
  self.class.sq(str)
end
ssh_keygen(key, path) click to toggle source

Generates an SSH key.

# File lib/dpl/provider.rb, line 383
def ssh_keygen(key, path)
  info :ssh_keygen
  ctx.ssh_keygen(key, expand(path))
end
symbolize(obj) click to toggle source

Deep symbolizes the given hash’s keys

# File lib/dpl/provider.rb, line 606
def symbolize(obj)
  case obj
  when Hash
    obj.map { |key, obj| [key.to_sym, symbolize(obj)] }.to_h
  when Array
    obj.map { |obj| symbolize(obj) }
  else
    obj
  end
end
try_ssh_access(host, port) click to toggle source

Tries to connect to the given SSH host and port.

# File lib/dpl/provider.rb, line 406
def try_ssh_access(host, port)
  info :ssh_try_connect
  shell "#{ENV['GIT_SSH']} #{host} -p #{port} 2>&1 | grep -c 'PTY allocation request failed' > /dev/null", echo: false, assert: false
end
uncleanup() click to toggle source

Restore files that have been cleaned up.

# File lib/dpl/provider.rb, line 335
def uncleanup
  shell 'git stash pop', assert: false
end
wait_for_ssh_access(host, port) click to toggle source

Waits for SSH access on the given host and port.

This will try to connect to the given SSH host and port, and keep retrying 30 times, waiting a second inbetween retries.

# File lib/dpl/provider.rb, line 399
def wait_for_ssh_access(host, port)
  info :ssh_remote_host, host, port
  1.upto(20) { try_ssh_access(host, port) && break || sleep(3) }
  success? ? info(:ssh_connected) : error(:ssh_failed)
end