Extremely low reach when using manual F/UF?

So, I have been manually following/unfollowing people from my niche and I’ve noticed something strange

When I don’t follow/unfollow people my reach is around 200-250 people per post (Noticed over a 5 day period)

As soon as I start following/unfollowing people (within limits - maximum 100 a day) the reach reduces drastically to maybe 100-120 people per post

Anyone has noticed this? If yes, what caused this issue and how did you solve it?

Bumping for help. If anyone knows what this is or it is something new that will really help