My Question is Can i incorporate some way of uploading this file then doing the rest of this as a background JOB
From the import
function thru the end. so that my server will not time out.
I keep getting timeout errors when i upload a file to be inserted into the database.
My form uploads the file and passes the file into my import function here:
def import(file)
parse_file(file)
end
I have an delayed job on the database creation of the data as you can see below.
def parse_file(file)
env = Rails.env.development?
error = 0
line = 2
posts = parse_csv(file)
posts.each do |post|
record = build_records(post, line)
#@log.info record
unless record
@log.debug "Error on line no #{line} Data: #{post}"
line = line+1
error = error+1
next
end
line = line+1
if env #checks for development
#development
RejectLoads.create(record)
else
#production
RejectLoads.delay(:queue => 'rejects').create(record)
end
end
@log.info "Total Errors: #{error}"
end
parsing here:
def parse_csv(file)
table = CSV.read(file.path, { :headers => true, :col_sep => "\t", :skip_blanks => true, :encoding => 'ISO-8859-1'})
unless table.empty?
header_arry = Array.new
table.headers.each do |h|
# simplest case here
header_arry << h.downcase
#which produces an array of column names called header_arry
end
rows = table.to_a
rows.delete_at(0)
posts = []
rows.each do |row|
#convert to hash using the column names
post = Hash[header_arry.zip(row)]
# do something with the row hash
posts.push post
end
posts
end
end
Aucun commentaire:
Enregistrer un commentaire