Active Record Query don't load all id's

I have an active record query like this one:

House.where.not(id: Person.ill.pluck(:house_id))

When I run explain on this query, I see that all the id's get loaded into the query:

Seq Scan on houses  (cost=0.00..4274.07 rows=1102 width=77)
 Filter: id <> ALL ('{1117,487,842,126,127,459,458,515,332,55,54,10,1697,449,448,1555,13,510,1986,8,9,7,6,5,1865,519,157,513,512,56,103,28,27,97,25,23,385,138,278,92,1,435,196,195,61,363,229,230,238,237,231,160,158,749,748,518,517,174,173,172,395,153,1170,207,206,276,199,198, ....

Is there a more performent method to achieve the same query? Without loading all the ids?