Currently I ran into issues of getting no more results found with some code 100 errors as well. I’m still not sure what the exact reason for that is but not using my own name list of 2000 German names anymore has helped.
After doing that I’m reaching daily limits without problems but the quality of followers is going down of course. That’s why I want to find the best way to include or exclude certain users while at the same time making it as scraping friendly as possible to still reach the daily actions limits.
From a logical perspective I would say it’s better to use a blacklist with the most common names of the countries I want to avoid to target. In my case I’m targeting mostly German users but Europe/US will be fine as well.
Here’s the big question:
Excluding users [Blacklist]: More users will meet the filters
Including users [Namelist]: Less users will meet the filters
What is more scraping friendly (faster)?
Maybe some of you who made a lot of tests with “blacklists” and “namelists” can point me in the right direction, if adding more names to the namelists or making a blacklist will be faster in scraping.
Using the namelist method, the quality is higher for sure but in this case I want to find a balance between quality and daily action limits.
I had a problem with my scrapers taking forever after each round (20 minutes before it aborted) and it was because I had a filter selected that not many users met. Once I took that filter off, my scraper moved way quicker and with less problems.
So Im thinking Excluding users [Blacklist]: More users will meet the filters is the way to go.
Thanks for your input! @BigJ
It makes sense and I will try it out.
One question someone maybe can answer here.
To decrease the scraping time and increase the daily actions, I want to put more names into my namelists. But I’m thinking about if it’s really a good thing to make the namelist bigger. On the one side more users would meet the filters/names but wouldnt it take longer for MP to check every user then?
And as far as I understood, with the name filtering its searching for the keyword in name/fullname + bio right? Because when that’s the case (bio as well) we could also add keywords like country, city, emoji of the flag of the nation (unicode format) or other common words people of the target country/language use in their name/username/bio.
If someone has done that already (adding common words of the target country/language, like city, job, hobby) I would love to hear some insights about if it has improved the scraping speed or made it slower.