Help with scrapers dying

Have you tried increasing your main account to scrapers ratio to 1:10?

1 Like

Thats what I am trying right now. I got 80 Main Accounts and 150 Scrapers but my Scrapers can’t “lift” them. So I am going to increase my Scrapers to at least 1:5 and see how it goes.

@heroeslair which tools are you using on the main account? Maybe you can check which ones is causing the scrapers to do a lot of API calls by turning off all of them and keep only one tool

I have observed that one cycle of follow tool with basic filters causes, on scraper side, 150 API calls on average.

I also observed that 1 scraper can do 500-600 API calls per day before getting EV, PV…

Once you know the cost per cycle of each tool, you can compute how many scrapers you need and do a tradeoff between tools you want to turn on/off and cost of scrapers

2 Likes

Nah. Its only 3:1

Would need thousands of scrapers to do 10:1 and would be very time consuming. My api limits are low and all my accounts do full actions on 3:1.

The ones that usually die, die under 100 api calls. I have very low limits on my scrapers.

I think it was account quality causing issues as a lot were dying on login. Which barely has happened in the past.

You can try to not use the accounts right away you have them do random actions first.
image

1 Like

Wouldn’t that increase the cpu load lots? Having 100s-1000s of scrapers doing random actions? plus the mains

Yeah I’m using that. Doesn’t seem to change much. Losing 20-30 a day with api calls maxed out at 100 day. About 5-10% of my scrapers a day.

Can use the same accounts and proxies and do full actions on api. Ie 150 follows a day. Yet if they are used as scrapers they die within a few days. So, must be the scraping use causing issues.

What are they doing, are they scraping as they go or you prescrape and use them for other scraper actions?

If I have enough scrapers, none die ever.

my settings: i use Full API on childs and 15-20 scrapers per child.

Do not use EB scraping for scrapers, my experience is they die

API only for scraping :slight_smile:

I would assume though you may need 20-25 scrapers per child since you are using EB

I put 50 scrapers per 4G , and I also put 5 of my child accounts on the same proxies. There is no issue here

1 Like

How many child accounts do you have.

Youd need 5000+ scrapers for only 250 account?
Seems very expensive. $1750us on scrapers for one jv licence. I have 3.

It’s probably not that many actually. I use 2,700 for every 200 child accounts.

$1750 ?? My scrapers cost me $0.08 each

For that price i isn’t too bad. Ive been trying ones from the marketplace which are more expensive but die just as fast.

Your 4g must be cheap to run that many scrapers on it?

I run a similar set-up and make my own scrapers for about $0.015 a piece. There’s virtually no reason to use 4g to automate them as they’re doomed to perish after 48 - 72 hours anyways. Keep your slaves on 4g but scrapers may as well just be sitting on DCs or at most, residential proxies.

1 Like

I use residential, DC didn’t work well for me.

I wasn’t getting any scrapers deaths for months until just this past week. Must be something new with IG

Oh yeah, there’s been a flurry of changes since the 17th. Surprised it only started hitting you just this last week.

@GreyHatNik How are you able to create them for so cheap? Only $0.015 a piece

I have a partnership with a virtual sms provider

Which type of proxies do you use??