class Dpl::Provider
Base class for all concrete providers that ‘dpl` supports.
These are subclasses of ‘Cl::Cmd` which means they are going to be detected by the first argument passed to `dpl [provider]`, instantiated, and run.
Implementors are encouraged to use the provider DSL to declare various features, requirements, and attributes that apply to their provider, to implement any of the following stages (methods) according to their needs and semantics:
* init * install * login * setup * validate * prepare * deploy * finish
The main logic should sit in the ‘deploy` stage.
If at any time the method ‘error` is called, or any exception raised the deploy process will be halted, and subsequent stages skipped. However, the stage `finish` will run even if previous stages have raised an error, giving the provider the opportunity to potentially clean up stage.
In addition to this the following methods will be called if implemented by the provider:
* run_cmd * add_key * remove_key
Like the ‘finish` stage, the method `remove_key` will be called even if previous stages have raised an error.
See the respective method’s documentation for details on these.
The following stages are not meant to be overwritten, but considered internal:
* before_install * before_setup * before_prepare * before_finish
Dependencies declared as required, such as APT, NPM, or Python are going to be installed as part of the ‘before_install` stage .
Cleanup is run as part of the ‘before_prepare` stage if the option `–cleanup` was given. This will use `git stash –all` in order to reset the working directory to the committed state, and cleanup any left over artifacts from the build process. Providers
can use the DSL method `keep` in order to declare known artifacts (such as CLI tooling installed to the working directory) that needs to be moved out of the way and restored after the cleanup process. (It is recommended to place such artifacts outside of the build working directory though, for example in `~/.dpl`).
The method ‘run_cmd` is called for each command specified using the `–run` option. By default, these command are going to be run as local shell commands, but providers can choose to overwrite this method in order to run the command on a remote machine.
@see github.com/svenfuchs/cl Cl’s documentation for details on how providers (commands) are declared and run.
Constants
- FOLDS
-
Fold names to display in the build log.
- STAGES
-
Deployment process stages.
In addition to the stages listed here the stage ‘finish` will be run at the end of the process.
Also, the methods ‘add_key` (called before `setup`), `remove_key` (called before `finish`), and `run_cmd` (called after `deploy`) may be of interest to implementors.
Attributes
Public Class Methods
Source
# File lib/dpl/provider.rb, line 88 def examples @examples ||= super || Examples.new(self).cmds end
Source
# File lib/dpl/provider.rb, line 104 def install_deps(ctx) ctx.apts_get(apt) if apt? ctx.gems_require(gem) if gem? npm.each { |npm| ctx.npm_install *npm } if npm? pip.each { |pip| ctx.pip_install *pip } if pip? end
Source
# File lib/dpl/provider.rb, line 100 def install_deps? apt? || gem? || npm? || pip? end
Source
# File lib/dpl/provider.rb, line 92 def move_files(ctx) ctx.move_files(move) if move.any? end
Source
# File lib/dpl/provider.rb, line 193 def initialize(ctx, *args) @repo_name = ctx.repo_name @key_name = ctx.machine_name super end
Source
# File lib/dpl/provider.rb, line 96 def unmove_files(ctx) ctx.unmove_files(move) if move.any? end
Source
# File lib/dpl/provider.rb, line 111 def validate_runtimes(ctx) ctx.validate_runtimes(runtimes) if runtimes.any? end
Public Instance Methods
Source
# File lib/dpl/provider.rb, line 312 def before_finish remove_key if needs?(:ssh_key) && respond_to?(:remove_key) uncleanup if cleanup? unmove_files(ctx) remove_dpl_dir end
Finalizes the deployment process.
This will:
-
Call the method ‘remove_key` if implemented by the provider, and if the feature `ssh_key` has been declared as required.
-
Revert the cleanup process, i.e. restore files moved out of the way during ‘cleanup`.
-
Remove the temporary directory ‘~/.dpl`
Source
# File lib/dpl/provider.rb, line 247 def before_init warn status.msg if status && status.announce? deprecations.each { |(key, msg)| ctx.deprecate_opt(key, msg) } setup_dpl_dir move_files(ctx) end
Initialize the deployment process.
This will:
-
Displays warning messages about the provider’s maturity status, and deprecated options used.
-
Setup a ~/.dpl working directory
-
Move files out of the way that have been declared as such
Source
# File lib/dpl/provider.rb, line 255 def before_install validate_runtimes(ctx) return unless install_deps? info :before_install install_deps(ctx) end
Install APT, NPM, and Python dependencies as declared by the provider.
Source
# File lib/dpl/provider.rb, line 280 def before_prepare cleanup if cleanup? end
Prepares the deployment by cleaning up the working directory.
@see Provider#cleanup
Source
# File lib/dpl/provider.rb, line 270 def before_setup info :before_setup setup_ssh_key if needs?(:ssh_key) setup_git_config if needs?(:git) setup_git_http_user_agent end
Sets the build environment up for the deployment.
This will:
-
Setup a ~/.dpl working directory
-
Create a temporary, per build SSH key, and call ‘add_key` if the feature `ssh_key` has been declared as required.
-
Setup git config (email and user name) if the feature ‘git` has been declared as required.
-
Either set or unset the environment variable ‘GIT_HTTP_USER_AGENT` depending if the feature `git_http_user_agent` has been declared as required.
Source
# File lib/dpl/provider.rb, line 625 def chmod(perm, path) super(perm, expand(path)) end
Source
# File lib/dpl/provider.rb, line 327 def cleanup info :cleanup keep.each { |path| shell "mv ./#{path} ~/#{path}", echo: false, assert: false } shell 'git stash --all' keep.each { |path| shell "mv ~/#{path} ./#{path}", echo: false, assert: false } end
Resets the current working directory to the commited state.
Cleanup will use ‘git stash –all` in order to reset the working directory to the committed state, and cleanup any left over artifacts from the build process. Providers
can use the DSL method `keep` in order to declare known artifacts (such as CLI tooling installed to the working directory) that needs to be moved out of the way and restored after the cleanup process.
Source
# File lib/dpl/provider.rb, line 596 def compact(hash) hash.reject { |_, value| value.nil? } end
Compacts the given hash by rejecting nil values.
Source
# File lib/dpl/provider.rb, line 550 def escape(str) Shellwords.escape(str) end
Escapes the given string so it can be safely used in Bash.
Source
# File lib/dpl/provider.rb, line 645 def expand(*args) File.expand_path(*args) end
Source
# File lib/dpl/provider.rb, line 617 def file?(path) File.file?(expand(path)) end
Source
# File lib/dpl/provider.rb, line 415 def fold(name, opts = {}, &block) return yield unless fold?(name, opts) title = FOLDS[name] || "deploy.#{name}" ctx.fold(title, &block) end
Creates a log fold.
Folds any log output from the given block into a fold with the given name.
Source
# File lib/dpl/provider.rb, line 425 def fold?(name, opts = {}) !opts[:fold].is_a?(FalseClass) && super() && !%i(init).include?(name) end
Checks if the given stage needs to be folded.
Depends on the option ‘–fold`, also omits folds for the init and finish stages. Can be overwritten by passing `fold: false`.
Source
# File lib/dpl/provider.rb, line 621 def mkdir_p(path) FileUtils.mkdir_p(expand(path)) end
Source
# File lib/dpl/provider.rb, line 629 def mv(src, dest) super(expand(src), expand(dest)) end
Source
# File lib/dpl/provider.rb, line 601 def only(hash, *keys) hash.select { |key, _| keys.include?(key) } end
Returns a new hash with the given keys selected from the given hash.
Source
# File lib/dpl/provider.rb, line 637 def open(path, *args, &block) File.open(expand(path), *args, &block) end
Source
# File lib/dpl/provider.rb, line 583 def opt_for(key, opts = {}) case value = send(key) when String then "#{opt_key(key, opts)}=#{value.inspect}" when Array then value.map { |value| "#{opt_key(key, opts)}=#{value.inspect}" } else opt_key(key, opts) end end
Source
# File lib/dpl/provider.rb, line 591 def opt_key(key, opts) "#{opts[:prefix] || '--'}#{opts[:dashed] ? key.to_s.gsub('_', '-') : key}" end
Source
# File lib/dpl/provider.rb, line 578 def opts_for(keys, opts = {}) strs = Array(keys).map { |key| opt_for(key, opts) if send(:"#{key}?") }.compact strs.join(' ') if strs.any? end
Generate shell option strings to be passed to a shell command.
This generates strings like ‘–key=“value”` for the option keys passed. These keys are supposed to correspond to methods on the provider instance, which will be called in order to determine the option value.
If the returned value is an array then the option will be repeated multiple times. If it is a String
then it will be double quoted. Otherwise it is assumed to be a flag that does not have a value.
@option prefix [String] Use this to set a single dash as an option prefix (defaults to two dashes). @option dashed [Boolean] Use this to dasherize the option key (rather than underscore it, defaults to underscore).
Source
# File lib/dpl/provider.rb, line 555 def quote(str) %("#{str.to_s.gsub('"', '\"')}") end
Double quotes the given string.
Source
# File lib/dpl/provider.rb, line 347 def remove_dpl_dir rm_rf '~/.dpl' end
Remove the internal working directory ‘~/.dpl`.
Source
# File lib/dpl/provider.rb, line 633 def rm_rf(path) super(expand(path)) end
Source
# File lib/dpl/provider.rb, line 202 def run stages = stage.select { |stage| run_stage?(stage) } stages.each { |stage| run_stage(stage) } run_cmds rescue Error raise rescue Exception => e raise Error.new("#{e.message} (#{e.class})", backtrace: backtrace? ? e.backtrace : nil) unless test? raise ensure run_stage(:finish, fold: false) if finish? end
Runs all stages, all commands provided by the user, as well as the final stage ‘finish` (which will be run even if an error has been raised during previous stages).
Source
# File lib/dpl/provider.rb, line 299 def run_cmd(cmd) cmd.downcase == 'restart' ? restart : shell(cmd) end
Source
# File lib/dpl/provider.rb, line 293 def run_cmds Array(opts[:run]).each do |cmd| cmd.downcase == 'restart' ? restart : run_cmd(cmd) end end
Runs each command as given by the user using the ‘–run` option.
For a command that matches ‘restart` the method `restart` will be called (which can be overwritten by providers, e.g. in order to restart service instances).
All other commands will be passed to the method ‘run_cmd`. By default this will be run as a shell command locally, but providers can choose to overwrite this method in order to run the command on a remote machine.
Source
# File lib/dpl/provider.rb, line 232 def run_stage(stage, opts = {}) fold(stage, opts) do send(:"before_#{stage}") if respond_to?(:"before_#{stage}") send(stage) if respond_to?(stage) end end
Runs a single stage.
For each stage the base class has the opportunity to implement a ‘before` stage method, in order to apply default behaviour. Provider
implementors are asked to not overwrite these methods.
Any log output from both the before stage and stage method is going to be folded in the resulting build log.
Source
# File lib/dpl/provider.rb, line 216 def run_stage?(stage) respond_to?(:"before_#{stage}") || respond_to?(stage) end
Whether or not a stage needs to be run
Source
# File lib/dpl/provider.rb, line 443 def script(name, opts = {}) opts[:assert] = name if opts[:assert].is_a?(TrueClass) shell(asset(name).read, opts.merge(echo: false)) end
Runs a script as a shell command.
Scripts can be stored as separate files (assets) in the directory ‘lib/dpl/assets/`.
This is meant for large shell commands that would be hard to read if embedded in Ruby code. Storing them as separate files helps with proper syntax highlighting etc in editors, and allows to execute them for testing purposes.
Scripts can have interpolation variables. See Dpl::Interpolate
for details on interpolating variables.
See Ctx::Bash#shell
for details on the options accepted.
Source
# File lib/dpl/provider.rb, line 340 def setup_dpl_dir rm_rf '~/.dpl' mkdir_p '~/.dpl' chmod 0700, '~/.dpl' end
Creates the directory ‘~/.dpl` as an internal working directory.
Source
# File lib/dpl/provider.rb, line 368 def setup_git_config shell "git config user.email >/dev/null 2>/dev/null || git config user.email `whoami`@localhost", echo: false, assert: false shell "git config user.name >/dev/null 2>/dev/null || git config user.name `whoami`", echo: false, assert: false end
Setup git config
This adds the current user’s name and email address (as user@localhost) to the git config.
Source
# File lib/dpl/provider.rb, line 389 def setup_git_http_user_agent return ENV.delete('GIT_HTTP_USER_AGENT') unless needs?(:git_http_user_agent) info :setup_git_ua ENV['GIT_HTTP_USER_AGENT'] = user_agent(git: `git --version`[/[\d\.]+/]) end
Sets or unsets the environment variable ‘GIT_HTTP_USER_AGENT`.
Source
# File lib/dpl/provider.rb, line 374 def setup_git_ssh(key) info :setup_git_ssh path, conf = '~/.dpl/git-ssh', asset(:dpl, :git_ssh).read % expand(key) open(path, 'w+') { |file| file.write(conf) } chmod(0740, path) ENV['GIT_SSH'] = expand(path) end
Sets up ‘git-ssh` and the GIT_SSH env var
Source
# File lib/dpl/provider.rb, line 358 def setup_ssh_key ssh_keygen(key_name, '~/.dpl/id_rsa') setup_git_ssh('~/.dpl/id_rsa') add_key('~/.dpl/id_rsa.pub') if respond_to?(:add_key) end
Creates an SSH key, and sets up git-ssh if needed.
This will:
-
Create a temporary, per build SSH key.
-
Setup a ‘git-ssh` executable to use that key.
-
Call the method ‘add_key` if implemented by the provider.
Source
# File lib/dpl/provider.rb, line 454 def shell(cmd, *args) opts = args.last.is_a?(Hash) ? args.pop : {} cmd = Cmd.new(self, cmd, opts) ctx.shell(cmd) end
Runs a single shell command.
Shell commands can have interpolation variables. See Dpl::Interpolate
for details on interpolating variables.
See Ctx::Bash#shell
for details on the options accepted.
Source
# File lib/dpl/provider.rb, line 562 def sq(str) self.class.sq(str) end
Outdents the given string.
@see Dpl::Squiggle
Source
# File lib/dpl/provider.rb, line 383 def ssh_keygen(key, path) info :ssh_keygen ctx.ssh_keygen(key, expand(path)) end
Generates an SSH key.
Source
# File lib/dpl/provider.rb, line 606 def symbolize(obj) case obj when Hash obj.map { |key, obj| [key.to_sym, symbolize(obj)] }.to_h when Array obj.map { |obj| symbolize(obj) } else obj end end
Deep symbolizes the given hash’s keys
Source
# File lib/dpl/provider.rb, line 406 def try_ssh_access(host, port) info :ssh_try_connect shell "#{ENV['GIT_SSH']} #{host} -p #{port} 2>&1 | grep -c 'PTY allocation request failed' > /dev/null", echo: false, assert: false end
Tries to connect to the given SSH host and port.
Source
# File lib/dpl/provider.rb, line 335 def uncleanup shell 'git stash pop', assert: false end
Restore files that have been cleaned up.
Source
# File lib/dpl/provider.rb, line 399 def wait_for_ssh_access(host, port) info :ssh_remote_host, host, port 1.upto(20) { try_ssh_access(host, port) && break || sleep(3) } success? ? info(:ssh_connected) : error(:ssh_failed) end
Waits for SSH access on the given host and port.
This will try to connect to the given SSH host and port, and keep retrying 30 times, waiting a second inbetween retries.