A few days ago, instagram blocked the download of Location data (I had to remove the tracker from IGMonitoring: /), now it seems that they are blocking the download of anything.
By entering the web version with DC proxy, it redirects to login.
The problem occurs in analytical tools and there are already several threads about changes in the data structure.
From what I see, it is connected to a proxy, going through mobile everything works fine.
I didn’t notice this when getting data from tags.
Anyone have any other solution than the mobile proxy?
You need to give the IP a “rest” and if you have had a lot of operations, then enter a proxy mechanism. You can’t do much on one IP. I’m just talking about downloading profile data from the address, e.g. https: //in…com/instagram.
In addition, the limit is not fixed, it varies depending on the IP.
I slowed my scraper bot down to opening 1 profile per minute and I have been able to open 200-400 profiles without getting blocked. Also implemented backup scrapping logins when a login gets rate limited. And running on 6 servers to pipeline the process to get the overall scrapping speed back up.
When you are not logged in and get rate limited block, it redirects to login. If you are logged in and get rate limited block, you get an error try again page.