Hi, my scrapers are getting fried (disabled) every day.
Some users here are in my same situation, some aren’t.
Talking with a friend that for long time didn’t have any block on scrapers accounts (with block I mean email confirmation/phone verification/captcha/get help signign in/disabled) we figured out something interesting. He enabled hashtag scraping and some hours/days after that his scrapers started getting fried.
This makes sense to me, because I’m also using scraping from hashtag (also followers of accounts, users that interacted with accounts and geo location).
So it could be that the users that are not using hashtag scraping are not having their scrapers disabled because of this.
I’m going to test some days with hashtag sources disabled to check if it makes difference or not.
This could also be Jarvee problem, because maybe IG changed something on API level when getting users from hashtag (or geo location).
I assume that there are no issues with “followers of accounts” and “users that interacted with accounts” sources as many users are not having scrapers troubles while using those sources.
Anyway in the midtime let’s make a poll to understand better how many of us are having scrapers disabled and which sources we’re using.
Users that are having blocks (disabled or other) on scrapers frequently Select which sources you’re using.
Hashtag sources
Followers of accounts
User that interacted with accounts
Geo location
0voters
Users that are NOT having blocks (disabled or other) on scrapers frequently Select which sources you’re using.
Hashtag sources
Followers of accounts
User that interacted with accounts
Geo location
0voters
Please vote so we can understand better the situation about accounts that are having blocks and if it’s because of this or not.
You should really separate your scraper accounts into groups by endpoints. This way you can find out the “weak spots” you are having and can minimize the exposure to endpoints that cause trouble.
We barely lose any accounts for a long time already, but probably our endpoint usage might look a bit different than yours. Which endpoints cause you the most troubles?
I got into trouble on about 15 accounts doing hashtag scraping.
Also commenting inside hashtags and stuff like that, even on the embedded browser got me in trouble. But on the latter it only happened on 2 or 3 accounts, so it’s not quite enough data.
I have 3 old accounts that I’m using for scrapers and they are running fine, but I bought 10 new accounts, no profile picture, no posts, and no followers, fake accounts and they are always after few days in verification loop…
I set up really low API limits but no help, I believe that we can’t use that fake accounts for scraping, we must bought older profiles, with followers and posts, or resolving verification loop every few days
Also what I saw is that I got verification loop on one profile, I deleted that profile from jarvee, after few days I bring back account from verification loop, wait few days and I tried to put that account again on jarvee to scrape but verification loop was again there…
It’s really pain in the ass
It’s definately related to hashtag scraping. I’ve had the same 100+ scrape accounts do scraping for me for the last year and I’ve had a handful of EV/PV during this time. A week ago I started to scrape hashtags for the first time and by this morning all of them have got the captcha error. The overal level of scraping has not changed for me so I don’t think it’s related to API Calls/errors.
I’m hoping it’s caused by Jarvee so we can get a fix. What I’ve noticed as well is that when you scrape hastags, Jarvee just completly ignores the “skip non-english characters” filter so it looks like there are bugs with hashtag scraping. I’ve spoken to customer support about this but have got nowhere so far.
UPDATE
Until 6 hours ago all my scrapers (+500) was scraping also on hashtag sources and in that configuration I was having 5-20 new captcha/phone verication every 30 minutes.
Now it’s 6 hours straight that I’ve disabled hashtag scraping and 0 scrapers got a block.
So +1 per Jarvee who’s having some troubles with hashtag scraping.
Try tp stop all the scraping operation on those accounts with problems and after day or two start scraping again but don’t use always the same source. Try combining different sources.
With some crapers I’m scraping from Hashtags since 3-4 weeks and yesterday I got the first captcha verifications after 1-2 weeks. But as I’m using different types of sources for almost every tool I cannot say if Hashtags are causing this.
When you’re right Jarvee should either look into that and fix it it’s possible or (what I wanted request anyways) add more types of sources to the Like Comment and Story Viewer Tool. On the follow tool we have a lot of options but on the Story Viewer for example we just have 3 more except the Hashtags source.
I don’t want to have more than 1 tool scraping from followers of target accounts because I get better results when I use different sources for each tool (more exposure) and followers of target accounts can lead to issues in some cases as well.
