class Storeband
this class manage the storage of a job directory to S3
Public Instance Methods
compress(dir,name)
click to toggle source
Compress a directory
# File lib/manband/store.rb, line 17 def compress(dir,name) Zip::ZipFile.open(name, Zip::ZipFile::CREATE)do |zipfile| Find.find(dir) do |path| Find.prune if File.basename(path)[0] == ?. dest = /#{dir}\/(\w.*)/.match(path) # Skip files if they exists begin zipfile.add(dest[1],path) if dest rescue Zip::ZipEntryExistsError end end end end
sends3(name,bucket,access,secret)
click to toggle source
Sends the file to the S3 bucket name: file name bucket: destination bucket name access: user credential access secret: user credential secret
# File lib/manband/store.rb, line 71 def sends3(name,bucket,access,secret) @@log.debug "connect to s3: "+FlowConfig.s3host+":"+FlowConfig.s3port.to_s+FlowConfig.s3path AWS::S3::Base.establish_connection!( :access_key_id => access, :secret_access_key => secret, :server => FlowConfig.s3host, :port => FlowConfig.s3port, :path => FlowConfig.s3path ) fbase = File.basename(name) AWS::S3::S3Object.store(fbase, open(name), bucket) @@log.debug "object stored" end
store(job,uid,bucket="manband")
click to toggle source
Store a job workdir to S3
job: current job uid: user identifier bucket: bucket name to store the zip file @return false in case of failure
# File lib/manband/store.rb, line 37 def store(job,uid,bucket="manband") user = User.get(uid) if user == nil job.update(:store => STORE_ERROR) return false end if user.s3_access.nil? || user.s3_secret.nil? job.update(:store => STORE_ERROR) return false end job.update(:store => STORE_RUN) zipfile = "manband-"+job.wid.to_s+"-"+job.node+".zip" if File.exists?(FlowConfig.workdir+"/"+zipfile) File.delete(FlowConfig.workdir+"/"+zipfile) end begin # compress @@log.debug "Compress to "+FlowConfig.workdir+"/"+zipfile compress(job.workdir, FlowConfig.workdir+"/"+zipfile) # Send file sends3(FlowConfig.workdir+"/"+zipfile,bucket,user.s3_access,user.s3_secret) rescue Exception => e @@log.error "An error occured during S3 operation: "+job.id.to_s+" "+e.message job.update(:store => STORE_ERROR) return false end job.update(:store => STORE_OVER) end