Guide "How to make scrapers Immortal"

[instascalp]

By the way, I’m from Russia and I don’t know much English. I use a translator, so I hope everything will be clear)

So hello everyone. I have been working in automation for more than 6 years. Now many guys have a lot of questions about the fact that the scrapers burn very quickly. They make about 500 + api requests and get to the checkpoint.

In fact, few people know that scrapers can be improved and made so that they can safely make from 10,000 to 20,000 api requests. So I have 3-4 scrapers tied to 12-15 accounts. And the accounts perform the entire limit of actions per day.

As you can see in the screenshots, it’s only 13: 00 in the afternoon. But the accounts have already completed most of the actions. They start work at 04: 30 in the morning and finish by 00: 00 at night.

So, how to pump paper clips?

1. Register the scraper via Jarvee to the mail without using the phone. That is, register only by email

2. Take a phone number for rent on SMS activation services. And link 1 phone number to 2 accounts. The main thing here is to find the correct prefixes. I am from Russia and use numbers with the prefixes +7 925, and +7 926. According to my tests, these are the best phone numbers. If you take the others, then in 99% you get to the check point of the 24-hour account verification.

3. After registering your account, add it to the scrapers in jarvee and start working. This is where the fun begins. After a certain number of api requests on these accounts, you will start receiving checkpoints again, the first one will be a captcha. After you pass it, you will be asked for your phone number again. Your task is to take numbers with good prefixes again. If there is a bad number, you will kill the account. BUT if you pass the checkpoint with a good phone number, the account will continue to work and this time it will make more api requests. (After confirmation, you need to delete the phone number in your profile so that you can enter a new one at the next request)

This way you pump the number of api requests in scrapers. The whole difficulty lies in the fact that when you pass the checkpoint on the phone on the profile for example 5-6 times, you can take a bad number and the account will die. Therefore, accounts that already make more api requests than the previous ones need to be saved. For example, I do this:

When I take the phone number. The first account where I use room is usually fresh account which only registered if the room is good and I didn’t checkpoint for 24 hours, then I take that number and use for the old profile (in 99.9% checkpoint is passed, though sometimes there is a chance to catch the secondary addition of a 24 hour test account)

Bottom line: It’s pretty simple. I think that by the end of the month I will have only 6-10 scrapers, which will make from 15 to 20 000 api requests, and will be able to process 35-45 main accounts

[ossi]

Thank you for sharing this guide! Which email provider do you use for account creation?

@instascalp

I am from Russia and use numbers with the prefixes +7 925, and +7 926.

It’s difficult to get such numbers for people who don’t live in Russia. Do you know any alternative phone number providers?

[ydm]

Do you use Full Api for your scrapers?

[instascalp]

With phone numbers, you just need to test them. If you see a pattern that the number works well, then remember the prefix. I use different mail most often rambler.ru with active imap

[instascalp]

Yes, I don’t use EB. This reduces the speed of work.

[ydm]

Thanks for sharing your settings

What kind of proxies do you use for them?

[instascalp]

@ydm

акие прокси вы используете для них

I use 4G proxies

[blackgodlrider_now]

does your proxies rotate if yes after how many minutes?

[instascalp]

Yes, 60 minutes

[denis1]

Super valuable for many users that struggle with their scrapers since a few months. No B.S and straight to the point

[Micky]

Now this thread definately deserves more than only 5 likes, get at it and start appreciating the time this gentleman has put in making this thread for you. He could have chosen to keep it for himself. Press that damn like button y’all! That’s how things used to work in here.

@instascalp Thanks for sharing your golden nugget buddy. Incredible to realise how much only a decent number and the right prefix can influence the overall performance of a scraper.

Can you tell if your scrapers fall after 15-20000 api calls? And when they fall and you create new scrapers, do you keep using the same proxies? And how often do you rotate IP on your proxies? I have a slight feeling that tells me, that your proxies are better than average 4G proxies. They perform better.
Other than that, your settings are on point.

[instascalp]

No) my 4g proxies are regular. Rotation every 60 minutes. I don’t throw away scrapers, it’s just that after 15,000 requests they crash for a check point, and I keep confirming them by pumping them up and even harder)) as a result, they will stop asking for checkpoints

[Winds]

Super useful!! Thank you so much for the information. Where do you find these kind of numbers? In PVA services like sms-activate.ru?

[instascalp]

Yes, that’s right. I use sms activate)

[Micky]

Excellent. Thanks for clearifying the fact.

[instascalp]

Please. Guys, ask questions, I’ll answer everyone

[Enguerran]

Hi mate, thanks for the post.

So it costs you around $10 a month to rent one number that you can use to verify around 12 scrapers a month, are my math correct or I’m missing something?

[instascalp]

In different ways. It took me only $6 to create 5 scrapers that process 15 basic accounts and make 15,000 requests each. Now I spend only $1 in 2 weeks on them on support)

[Grammer0]

@instascalp

Take a phone number for rent on SMS activation services. And link 1 phone number to 2 accounts. The main thing here is to find the correct prefixes. I am from Russia and use numbers with the prefixes +7 925, and +7 926. According to my tests, these are the best phone numbers. If you take the others, then in 99% you get to the check point of the 24-hour account verification.

Good prefixes means Russian ones only? Or it can be done with other prefixes as well? By the way, your 4g ip is Russian as well? Because if it is, I could test this with other prefixes but same country 4g IP.

@instascalp

The whole difficulty lies in the fact that when you pass the checkpoint on the phone on the profile for example 5-6 times, you can take a bad number and the account will die. Therefore, accounts that already make more api requests than the previous ones need to be saved. For example, I do this:

When I take the phone number. The first account where I use room is usually fresh account which only registered if the room is good and I didn’t checkpoint for 24 hours, then I take that number and use for the old profile (in 99.9% checkpoint is passed, though sometimes there is a chance to catch the secondary addition of a 24 hour test account)

I haven’t understood this, if you can re explain or someone who understood to explain it again would really appreciate it.

[instascalp]

My 4G proxies are Russian. But it doesn’t matter.I use my Russian proxies to work with clients from the USA, China, etc. This is unimportant. You can take any numbers, you just need to test and find the prefixes that work for the numbers of your country)