Hello guys, I’ve been working with a programmer the last few days on a python script that scrapes quotes from topics at brainyquotes.com. With it, you can scrape by topic and word count. Considering the installation takes some setup I will walk you through everything; step-by-step…
The script runs on google chrome inside selenium, everything is built on Python. This tutorial is for windows only. If you are using another OS you can still do so, you’ll just have to find a tutorial on how to install the chrome webdriver for your system… Other than that it is the same instructions.
lets begin…
You’ll need to download:
- Google chrome. If you don’t know how to find or install chrome, you should stop reading and take a basic Comp TIA class… Seriously, if you don’t understand these instructions, you have much more to learn if you want to make a living in marketing.
I digress…
Extract these below once downloaded:
-
Python 3 (Choose the latest release)
https://www.python.org/downloads/ -
My Brainy Quotes Scraper & Google Chrome Webdriver
https://drive.google.com/file/d/1F-rs1hAUoeqZHfq17s_JFMy0gmXg6GBE/view?usp=sharing
Once those are downloaded and extracted please follow the instructions carefully…
-
Go to this video and follow the instructions to install google chrome webdriver.
https://www.youtube.com/watch?v=dz59GsdvUF8 -
Once you install the webdriver and verify it works Install Python and tick the box create link to path.
-
After installation run a command prompt window then type:
-
python --version
-
you should see Python 3.x.x
-
If no you didn’t click the check mark to add a path…
-
If so uninstall by double clicking the installation file again and hitting uninistall
-
Then go to add & remove programs and delete python launcher
-
reinstall python and make sure to tick the add to path box.
-
check your version by typing:
python --version
-
you should see Python 3.x.x
-
now type inside the cmd:
pip install selenium
-
if it asks you to upgrade pip enter the command it displays to upgrade it.
-
now type:
pip install beautifulsoup4
-
Now open a new terminal and go to the directory you installed python, the default is:
C:\Users\Admin\AppData\Roaming\Python\Python37 -
Once inside drag and drop the BrainyScraper.py file into the Python37 folder.
-
Create a new notepad file named quotes.txt inside the Python37 folder.
-
Now inside the cmd prompt type: “python (drag the BrainyScraper.py file ontop of the cmd prompt)”
example: python C:\Users\Admin\AppData\Roaming\Python\Python37\BrainyQuotes.py
this will have python search the directories for the file.
-
hit enter and enter the topic you want from brainyquotes. Lets choose: motivation
-
enter your word count. Lets enter 16.
-
Wait while the chrome opens an automation browser. Allow python through the firewall.
-
This may take very long. Some topics can take an hour or so.
-
Once the scraper is done and the chrome browser closes you will see the list quotes displayed in the command prompt along with the div classes.
-
Now check your quotes.txt file we created earlier and all of the quotes should be there.
Congratulations you now have a fully automated quotes scraper with proper niche and word length.
Stay tuned, as I will be posting my bulk quotes automation, which is an updated spin off of [Update] How to Create Unique & Beautiful 1000+++ Content Just in 5 Minutes
As you know the sizing can be wonky at times but I will share with you how to size the images and quotes perfectly every time. Jam the and I will get the bulk quote photos thread posted ASAP.