samedi 17 décembre 2016

Rails 3: how to check if very large migration is running properly

I want to update all of a column in a table with over 2.2 million rows where the attribute is set to null. There is a Users table and a Posts table. Even though there is a column for num_posts in User, only about 70,000 users have that number populated; otherwise I have to query the db like so:

@num_posts = @user.posts.count

I need to use a migration to update the attributes and I'm not sure whether or not it's running properly. Here is my migration file:

class UpdateNilPostCountInUsers < ActiveRecord::Migration
  def up
    nil_count = User.select(:id).where("num_posts IS NULL")

    nil_count.each do |user|
      user.update_attribute :num_posts, user.posts.count
    end
  end

  def down
  end
end

In my console, I ran a query on the first 10 rows where num_posts was null, and then used puts for each user.posts.count . The total time was 85.3ms for 10 rows, for an avg of 8.53ms. 8.53ms*2.2million rows is about 5.25 hours, and that's without updating any attributes.

How do I know if my migration is running as expected? Is there a way to log to the console %complete? I really don't want to wait 5+ hours to find out it didn't do anything. Much appreciated.

Aucun commentaire:

Enregistrer un commentaire