Self hosted Proxies breakdown and Tutorial


#1

I have spent the last week working on self-hosted proxies. I was working off two separate VPS boxes with approximately the following specifications.

  • Ubuntu 16.04
  • 1mb Ram
  • 1 core
  • 10gb storage
  • 1Gbps
  • (cost $9 a year)

I initially started with TinyProxy. It is a very lightweight and easy to install. It can be installed and running in just a few minutes. However, I found some pretty serious problems. First of all, TinyProxy does not have Username/Password support. That means you are required to whitelist you home IP and any other you might access from or leave it open. If you are always working from the same IP this is not necessarily an issue. However, if you travel, use a VPN network, etc… this would be a pretty big inconvenience.

The second and even larger problem is I could not find a way to configure the proxy as Anonymous or Elite (meaning a simple test could identify it) that does not mean it cant be done. I tried several proxy-testers and they both identified the proxy. Proxies have a lot of various uses, however, I assume for everyone here the primary concern is to obfuscate the originating IP and to do this without being identified as doing so.

I ended going with Squid3 (3.5). It is a much larger and more diverse application than the previously mentioned program. It does allow for IP whitelisting as well as password authorization, it does require Apache to be installed for this though. I was able to test against several proxy-testers and the only failure point was that the IPs were allocated to data centers. This is something that is pretty hard to get around.

Installation

Update your APT repository and install the software we will need

sudo apt-get update
sudo apt-get install squid3
sudo apt-get install apache2-utils

apache2-utils is required for htpasswd which we will use as a flat file password store to secure the proxy.

Setup the password store

sudo touch /etc/squid/passwords
sudo chmod 777 /etc/squid/passwords
sudo htpasswd -c /etc/squid/passwords USERNAME
[prompt]

In the lines above, replace USERNAME with the username you want on your proxy. When the line is executed you will be prompted to enter a password for the user.

Configure the Squid Proxy

Move the default bloated squid configuration file.

sudo mv /etc/squid/squid.conf /etc/squid/squid.conf.original

Now create a new squid configuration file

sudo vi /etc/squid/squid.conf

This is the configuration file I use however there are a lot of possible variations.

#Define allowable Networks or IPs.
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
#You can enter your home IP here if you would like to whitelist
acl home dst 192.0.0.0/8

#For christ sake do not use the default port
http_port 8881
dns_v4_first on 
cache deny all
forwarded_for delete

acl ip1 myip 111.222.111.222
tcp_outgoing_address 111.222.111.222 ip1


#Define allowable Ports
acl Safe_ports port 80          # http
acl Safe_ports port 443         # https
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 777         # multiling http

#Allow the ports and networks we want, then deny everyone else.
http_access allow manager localhost
#http_access deny manager
#http_access deny !Safe_ports
#http_access allow localhost
#http_access allow home
cache deny all

#Password authentication
auth_param basic program /usr/lib/squid3/basic_ncsa_auth /etc/squid/passwords
auth_param basic realm proxy
acl authenticated proxy_auth REQUIRED
http_access allow authenticated
http_access deny all

#Rules for modifying User Agent, If you are using a program that already does this you can comment this out
#See http://www.useragentstring.com for more examples.
header_replace Accept */*
header_replace Accept-Encoding *
header_replace Accept-Language en-us
header_replace User-Agent Mozilla/6.0 (Windows NT 6.2; WOW64; rv:16.0.1) Gecko/20121011 Firefox/16.0.1

#Rules to anonymize http headers
request_header_access Allow allow all
request_header_access Authorization allow all
request_header_access WWW-Authenticate allow all
request_header_access Proxy-Authorization allow all
request_header_access Proxy-Authenticate allow all
request_header_access Content-Encoding allow all
request_header_access Content-Length allow all
request_header_access Content-Type allow all
request_header_access Date allow all
request_header_access Expires allow all
request_header_access Host allow all
request_header_access If-Modified-Since allow all
request_header_access Last-Modified allow all
request_header_access Location allow all
request_header_access Pragma allow all
request_header_access Accept allow all
request_header_access Accept-Charset allow all
request_header_access Accept-Encoding allow all
request_header_access Accept-Language allow all
request_header_access Content-Language allow all
request_header_access Mime-Version allow all
request_header_access Retry-After allow all
request_header_access Title allow all
request_header_access Connection allow all
request_header_access Proxy-Connection allow all
request_header_access All deny all


#Do not show client IP address
forwarded_for off

Restart the squid proxy

service squid restart

Note the service could also be called squid3. It may take a while for the proxy to restart. If you prefer, you can reload squid configurations with the command

squid -k reconfigure

Check that it is working

service squid status

Go into a web browser and try using your proxy. Here are some sites that you can test your proxies on.

http://amibehindaproxy.com/
http://www.proxyblind.org/tut.shtml
https://www.whatismyip.com/proxy-check/?iref=home

References:


http://www.dataparadis.net/osp/gnu-linux-server/proxy-server/high-anonymous-elite-proxy-with-squid3/

There are a lot more, I trust that you can use google.

If you spot any bugs or can point out any way to improve this please let me know.


#2

Really helpful for starters, good job! +1

2 little comments:

  • this config ‘style’ is only for a one user setup
  • the default squid installation limits the amount of ports to 128.

@maniacmonk force replacing the user agent might not be so good for Instagram related tools :slight_smile:

This might be enough for ‘protecting’ of the origin address and fingerprint:

via off
request_header_access X-Forwarded-For deny all
request_header_access From deny all
request_header_access Referer deny all
request_header_access User-Agent deny all
request_header_access Authorization allow all
request_header_access Proxy-Authorization allow all
request_header_access Cache-Control allow all
request_header_access Content-Length allow all
request_header_access Content-Type allow all
request_header_access Date allow all
request_header_access Host allow all
request_header_access If-Modified-Since allow all
request_header_access Pragma allow all
request_header_access Accept allow all
request_header_access Accept-Charset allow all
request_header_access Accept-Encoding allow all
request_header_access Accept-Language allow all
request_header_access Connection allow all
request_header_access All deny all

#3

Is this for 1 proxy or multiple?


#4

This is for a single proxy. It can be easily modified for multiple IPs. You should be able to do this with something like this.

acl ip1 myip 192.102.51.2 
acl ip2 myip 192.102.51.3

tcp_outgoing_address 192.102.51.2 ip1
tcp_outgoing_address 192.102.51.3 ip2
tcp_outgoing_address 192.102.51.1 # default

Proxies Instagram Account
#5

what if i want username/password? still my ip needed? i have dynamic ip. how would the setup look than?

ip1=serversip1
ip2=serversip2

myip=really my home ip?


#6

@Leni_In_Love if you have pass Auth set up you do not need any IP Auth.


#7

How many proxies can i create with one server?


#8

how does the passauth config look like?


#9

@SpawneR based on the following articles the max number of IPs that you can run on a single Squid server is 128.

http://squid-web-proxy-cache.1019090.n4.nabble.com/squid-with-multiple-ips-is-listenting-to-some-ips-with-port-and-not-all-of-ips-td4668784.html


#10

and how do i set user/pass up?


#11

@Leni_In_Love


#12

but the config still wants my ip than? (mine is dynamic)


#13

No it should not. You can just comment out this line.

#acl home dst 192.0.0.0/8


#14

@HenryCooper are you familiar with using squid with IPv6?


#15

With IPV6, IPV4, Modems, Phones and Servers :smiley:

We use Squid3 only on the master nodes tho. Exit nodes are being run with 3proxy.


#16

Do you know if it possible to utilize IPv6 for IG?


#17

Of course it is.


#18

Can you help me understand what the config differences look like with squid.config and IPv6?


#19

Just use the IPV6 Address in the config file, no more magic needed (assuming that you have proper entries in your interface file)


#20

Do you mind sharing the equivalent of these settings for 3proxies ?