class Storeband

this class manage the storage of a job directory to S3

Public Instance Methods

compress(dir,name) click to toggle source

Compress a directory

# File lib/manband/store.rb, line 17
def compress(dir,name)
  Zip::ZipFile.open(name, Zip::ZipFile::CREATE)do |zipfile|
    Find.find(dir) do |path|
      Find.prune if File.basename(path)[0] == ?.
      dest = /#{dir}\/(\w.*)/.match(path)
      # Skip files if they exists
      begin
        zipfile.add(dest[1],path) if dest
      rescue Zip::ZipEntryExistsError
      end
    end
  end
end
sends3(name,bucket,access,secret) click to toggle source

Sends the file to the S3 bucket name: file name bucket: destination bucket name access: user credential access secret: user credential secret

# File lib/manband/store.rb, line 71
def sends3(name,bucket,access,secret)
  @@log.debug "connect to s3: "+FlowConfig.s3host+":"+FlowConfig.s3port.to_s+FlowConfig.s3path
  AWS::S3::Base.establish_connection!(
    :access_key_id     => access,
    :secret_access_key => secret,
    :server => FlowConfig.s3host,
    :port => FlowConfig.s3port,
    :path => FlowConfig.s3path
  )
  fbase = File.basename(name)

  AWS::S3::S3Object.store(fbase, open(name), bucket)
  @@log.debug "object stored"
end
store(job,uid,bucket="manband") click to toggle source

Store a job workdir to S3

job: current job uid: user identifier bucket: bucket name to store the zip file @return false in case of failure

# File lib/manband/store.rb, line 37
def store(job,uid,bucket="manband")
  user = User.get(uid)
  if user == nil
    job.update(:store => STORE_ERROR)
    return false
  end
  if user.s3_access.nil? || user.s3_secret.nil?
    job.update(:store => STORE_ERROR)
    return false
  end
  job.update(:store => STORE_RUN)
  zipfile = "manband-"+job.wid.to_s+"-"+job.node+".zip"
  if File.exists?(FlowConfig.workdir+"/"+zipfile)
    File.delete(FlowConfig.workdir+"/"+zipfile)
  end
  begin
    # compress
    @@log.debug "Compress to "+FlowConfig.workdir+"/"+zipfile
    compress(job.workdir, FlowConfig.workdir+"/"+zipfile)
    # Send file
    sends3(FlowConfig.workdir+"/"+zipfile,bucket,user.s3_access,user.s3_secret)
  rescue Exception => e
    @@log.error "An error occured during S3 operation: "+job.id.to_s+" "+e.message
    job.update(:store => STORE_ERROR)
    return false
  end
  job.update(:store => STORE_OVER)
end