Okay not having a card is a valid reason but
counterbalance is some kind of Lucky game. So it doesnt do a very reliable job for you.
and for the mill cards, those are also very luck based and a very big majority of decks are playing cards like
eternal witness so they dont really feel to bad if you mill of something.
if you dont want to care about the stuff they are doing play
Teferi, Mage of Zhalfir. this makes your opponent only cast in sourery speed and lets be honest you play so much counterspells that the only thing you are maybe worried about is a
counterspell follow up witch Teferi, Mage of Zhalfirdenies to 100%. Play cards who actually do things for you. Random mill cards can help a flooding opponent as well.
imagine the following - you are in a game and its getting close, then you pplay a land for turn and mill off 2 lands from your opponents deck. Your opponent topdecks a card like
Emrakul, the Promised End. This would be a lost game for you just because you milled your opponent. Sure it can go the other way as well but ppl usually construckt their edh decks in a way that they can interact with their lost stuff or just play copies of cards who do similar stuff.