How to know who to unfollow?

if I followed someone manually and that person doesnt follow back how can I know that?

you can have agent to keep eye on every account which is following back and add them on white list or you can use any android app there are many app for that

You can check the follow back status