Make Rails autoloading/reloading follow dynamic includes

Context

I want to add some admin specific code to all models via concerns that are automatically included. I'm using the naming convention MyModel => MyModelAdmin and they're in the standard Rails directory app/models/concerns/my_model_admin.rb. Then I can glob over all of them and do MyModel.include(MyModelAdmin).

Issue

Dynamic includes work fine, but when changing the concern in development Rails doesn't reload it properly. In fact the concern seems to get removed.

Reproduction

app/models/my_model.rb

class MyModel
end

app/models/concerns/my_model_admin.rb

module MyModelAdmin
  extend ActiveSupport::Concern

  def say
    "moo"
  end
end

config/initializers/.rb

MyModel.include(MyModelAdmin)

So far so good, MyModel.new.say == "moo".
But now change say to "baa" and you get NoMethodError undefined method 'say'.

Notes

I tried a number of things that didn't help:

• require_dependency
• Model.class_eval "include ModelAdministration"
• config.autoload_paths
• Using another explicit concern in ApplicationModel with an included hook that includes the specific concern in each model.
• ActiveSupport.on_load only triggered on Base not each model.

Does this mean Rails can only autoload using static scope? I guess Rails sees the concern change, knows the model has it included, reloads the model but the static model definition doesn't have the include so the concern goes missing and stops being tracked. Is there a way to force Rails to track dynamically included modules?