Rails query - Intersection of single attribute of 2 large models

So I have 2 models, ModelOne and ModelTwo, both of which have a my_field attribute. I want to count how many distinct ModelTwo objects there are, such that my_field uniquely exists in ModelTwo and not in ModelOne.

With a small enough dataset, I could write a simple inefficient function and get the data I need, but both ModelOne and ModelTwo have several million objects. So I figure that just getting the intersection count of how many my_field values are shared between the two tables would be enough, since subtracting that count from ModelTwo total count would give me the number I'm looking for.

What's a good, efficient, way to get this intersection count?