Traffic Filter Plugin

[Peter]

Limitations
The TRAFFIC FILTER's country blocking function is based on a database table, which converts an IP address to a country, in some cases to EU (Europe) and AP (Asia-Pacific). As you already might know, IP address allocations are not permanent. They change from time to time. There will always be some inaccuracy, small amounts of error.

I want to give another example to demonstrate what the country blocking filter can and cannot do.

Let's say that you want to block country 'X' completely. Perhaps you don't want the country X' government (or a commercial entity) to have access to your site. If you block country 'X' using the TRAFFIC FILTER or any similar IP address based solution, the country 'X' users still may be able to access your site. They can do it either through a proxy, an anonymous proxy, or by having a computer (or server) in a country which is not on your blacklist.

Jamit intends to periodically update the TRAFFIC FILTER with enhancements, as well as the IP-to-country database.

IP Blocking Resources
The TRAFFIC FILTER is suitable (and easily configurable) for country blocking (based on IP address). Another solution for country blocking is .htaccess file, but you need to know the IP address ranges or blocks. I have found this website which will give you IP ranges and CIDR blocks of various countries in a format ready to use in your .htaccess file. Of course, I have no idea how accurate these IP ranges are....

http://www.countryipblocks.net/country-blocks/select-formats/

However, I do not recommend using the IP ranges from the above site in the TRAFFIC FILTER, because the TRAFFIC FILTER already has its own database of IP addresses.

Be careful when you use the .htaccess file for country blocking, because some countries have too many IP blocks and your .htaccess can easily become hundreds of kB or even few MB in size and (I wonder if) this could slow down your server. This is why the TRAFFIC FILTER is a good solution for country blocking.

[promotionbox.de]

Code:

AGENT/^$/@DEF ......... empty
AGENT/^\ +$/@DEF ....... 1 or more white spaces (only white spaces)
AGENT/^\.+$/@DEF ........ 1 or more dots (only dots)
AGENT/^Mozilla$/@DEF ....... string is only 'Mozilla' (definitely spoofed user-agent)
AGENT/^Mozzila$/@DEF ...... string is only 'Mozzila' and obviously misspelled (I had such request on my site!)
AGENT/^Mozilla/?[0-9]{1,}\.[0-9]{1,}$/@DEF ..... this is NOT a human visitor
AGENT/^Mozilla/?[0-9]{1,}\.[0-9]{1,}\ ?\(compatible;?\)$/@DEF ...... this is NOT a human visitor
AGENT/^[a-zA-Z0-9]{1,}$/@DEF ..... alphanumeric string, such as '7yT2gB1kcWiP2'
AGENT/^[a-zA-Z\ \.]{1,}$/@DEF ........ alphabetical string with optional spaces or dots, such as 'Morfeus strikes again.' (I had such requests on my site!)
AGENT/^[0-9\ ]{1,}$/@DEF ....... numeric string with optional spaces, such as '8346456 383 38 5494'
... and most important for last ....
AGENT/(\'|\"|\`)/@DEF ..... blocks MySQL injection attacks

Hi! Where do I put the code above? In the field of "Redirects", "Whitelisted IP Addresses (Optional)" or "Whitelisted User-Agents (Optional)". Thanks for your help. Does this code works with German hompages too?

[Peter]

Hi Promotionbox,

That code (you asked about) goes in the control panel of the TRAFFIC FILTER, namely the section 1.1 (Redirects).

Ignore the comments, so just enter:

Code:

AGENT/(\"|\'|\`)/@DEF

... and each entry on new line. Do not put in my comments (those things with ........).

Sure, it will work for German homepages too. Wirklich!

[promotionbox.de]

You're ok Peter!
Thanks for your quick assistance. I'll try it and will give you a feedback. And it's nice to read a German word 'Wirklich'. So I've some homefeeling  . If your German is more don't hesitate to write me in German. I would appreciate that.

Best greetings from Germany to you!
-Thorsten

[Peter]

Thorsten,

Sehr toll! Das ist alles was ich wollte auf Deutsch schreiben. I am not German, but rather a mix of everything. But I get the 'home feeling" just reading your messages.

Greeting from Hongkong to you and all Jamit customers!

Peter

[promotionbox.de]

So macht das Forum richtig Spaß - klasse!!!!
Thank you all for the nice support the last months you gave me. I love you.......Michael Jackson would say.

Stay the way you are.

-Thorsten

[Peter]

Today, the real version 1.0 of the TRAFFIC FILTER was released with some small improvements.

If you are UPGRADING from the BETA version, and you already have some blocking/redirection settings saved, this is what I recommend that you do:
-- Go to the admin panel
-- Go to the control panel for the TRAFFIC FILTER plugin
-- Using Ctrl c and Ctrl c keyboard keys copy and paste the settings to a text editor.
-- Install the version 1.0 plugin
-- Copy the settings from your text editor to the plugin's control panel and save

And you are done!

[screen_mates]

Would you mind posting those malicious IP ranges here?

Keep up the good work...

Thanks!


> I block over 200 IP ranges (could easily be millions of IP addresses), over 20 countries, many user-agent strings.

[screen_mates]

How about a plugin to validate if other installed plugins are up to date or outdated? With an option to apply updates/upgrades?

Thanks!

[screen_mates]

How about a plugin to validate if other installed plugins are up to date or outdated? With an option to apply updates/upgrades?

Thanks!

We need "comments" columns to record reasons why some IP's or expressions were added to the Redirects, Whitelist, etc.

[Peter]

Okay, here is a list of some BAD IP ranges and other redirect rules for the Traffic Filter. Please use it at your own risk. Jamit makes no claim or warranty regarding the accuracy of these IPs. You should consider that you may not want to block all of these IP ranges, but I do block them.

Code:

AGENT/^$/@DEF...empty
AGENT/^\.+$/@DEF...dots
AGENT/^[a-z0-9]{1,}$/[email protected] without whitespace
AGENT#^(Mozilla|Mozilla/[0-9]{1,}\.[0-9]{1,})$#[email protected] Mozilla
AGENT/^[a-z\ \.]{1,}$/[email protected] only
AGENT/^[0-9\ \.]{1,}$/@DEF...numeric only
AGENT/Mozzila/[email protected]
AGENT#(\'|\`|\*|\?|>|<|script|eval|base64_decode)#[email protected] injection
AGENT#MSIE\ [0-9]{1,1}\.[0-9]{1,1};\ MSIE\ [0-9]{1,1}\.[0-9]{1,1}#@DEF...2 browsers
[email protected] d'ivoire
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
65.213.208.128/[email protected], Cyveillance Inc.
65.222.176.96/[email protected], Cyveillance Inc.
65.222.185.72/[email protected], Cyveillance Inc.
151.173.0.0/[email protected], Cyveillance Inc.
65.46.48.192/[email protected], Bluecoat Systems Inc.
65.160.238.176/[email protected], Bluecoat Systems Inc.
204.246.128.0/[email protected], Bluecoat Systems Inc.
208.115.138.0/[email protected], Bluecoat Systems Inc.
217.169.46.96/[email protected], Bluecoat Systems Inc.
66.194.6.0/[email protected], Websense Inc.
208.80.192.0/[email protected], Websense Inc.
204.15.64.0/[email protected], Websense Inc.
208.17.184.0/[email protected], Verisign
69.36.144.0/[email protected], Verisign
206.169.110.0/[email protected], Secure Computing
64.124.14.0/[email protected], Markmonitor
82.80.248.0/[email protected] - Bezeqint-Hosting
62.0.8.0/[email protected]
206.28.72.0/[email protected] images
200.31.42.0/[email protected], VULCO S.A.
213.246.51.0/[email protected], Ikoula Hosting
213.246.52.0/[email protected], Ikoula Hosting
209.120.218.128/[email protected], Technology Universe
83.172.144.0/[email protected], hacker attack
149.226.0.0/[email protected], BSH Bosch und Siemens Hausgeraete GmbH
216.120.128.0/[email protected], Trivalent Group Inc.
216.120.192.0/[email protected], Trivalent Group Inc.
74.52.0.0/[email protected], Houston, Texas, The Planet Internet Services
213.183.192.0/[email protected], Intares, Hamburg
216.38.192.0/[email protected], Denver, Colorado, ViaWest
94.76.219.16/[email protected], BlueConnex Ltd.
208.91.8.0/[email protected], Texas, PRONSS
128.104.0.0/[email protected], University of Wisconsin-Madison, computer lab
206.51.224.0/[email protected], Tampa, Florida, NOC4Hosts Inc.
64.62.128.0/[email protected], Fremont, California, Hurricane Electric
65.19.128.0/[email protected], Fremont, California, Hurricane Electric
208.88.120.0/[email protected], Biznesshosting Inc.
131.107.0.0/[email protected], Microsoft, secret robot
69.71.208.0/[email protected], MoveClicks LLC, Sitedossier.com
209.167.50.16/[email protected], SevenTwentyFour Incorporated
206.183.1.0/[email protected] Search
189.104/[email protected], Tele Norte, HACKER
66.90.64.0/[email protected], FDC Servers
67.159.0.0/[email protected], FDC Servers
208.53.128.0/[email protected], FDC Servers
74.63.64.0/[email protected], FDC Servers
72.232.0.0/[email protected], Layered Technologies
72.233.0.0/[email protected], Layered Technologies
64.92.160.0/[email protected], Layered Technologies
69.58.176.0/[email protected], Verisign
67.215.224.0/[email protected], Secured Private Network
64.246.160.0/[email protected], Whois, Compass Communications, Inc.
66.231.176.0/[email protected] Online Systems, Inc.
72.249.0.0/[email protected] and USA, Colo4Dallas LP, Visvo Bot
72.249.128.0/[email protected] and USA, Colo4Dallas LP, Visvo Bot
66.34.0.0/[email protected], Texas, CI Host, Keyword Spy
208.99.192.0/[email protected], Seattle, Swift Ventures
208.94.240.0/[email protected], Aarons.net, Joe's Data Center
208.43.0.0/[email protected], Softlayer Technologies
74.86.0.0/[email protected], Softlayer Technologies
38.*@DEF...USA, PSI (same as 38.0.0.0/8)
208.115.96.0/[email protected], Topshoppingcart.com, Wowcrack.com
66.232.96.0/[email protected], NOC4Hosts Inc., Hivelocity Inc.
65.98.0.0/[email protected], New Jersey, Fortress ITX
64.69.32.0/[email protected], Los Angeles, CoreExpress
208.138.176.0/[email protected], Dow Jones & Company, Savvis
208.138.192.0/[email protected], Dow Jones & Company, Savvis
208.139.0.0/[email protected], Dow Jones & Company, Savvis
208.140.0.0/[email protected], Dow Jones & Company, Savvis
208.144.0.0/[email protected], Dow Jones & Company, Savvis
208.152.0.0/[email protected], Dow Jones & Company, Savvis
208.156.0.0/[email protected], Dow Jones & Company, Savvis
208.157.0.0/[email protected], Dow Jones & Company, Savvis
208.157.128.0/[email protected], Dow Jones & Company, Savvis
71.48.0.0/[email protected], Embarq Corporation
69.84.192.0/[email protected], Arrival Communications
207.241.224.0/[email protected], San Francisco, Internet Archive
158.237.0.0/[email protected] Military, Quantico, Virginia
138.162.0.0/[email protected] Navy Network Information Center, Pensacola, Florida
74.207.224.0/[email protected], Linode, proxies
199.85.208.0/[email protected], Hostventures, proxies
216.104.0.0/[email protected], Cupertino, TrendMicro.com
65.49.0.0/[email protected], Fremont, California, Hurricane Electric
157.63.0.0/[email protected], University of Tokyo
157.64.0.0/[email protected], University of Tokyo
157.80.0.0/[email protected], University of Tokyo
157.82.0.0/[email protected], University of Tokyo
[email protected], Los Angeles, Vrtservers Inc.
65.23.128.0/[email protected], Datarealm Internet Services
[email protected], NG Marketing
202.151.[email protected], Guam, 624 North Marine Corp Drive
77.91.224.*@DEF...RUSSIA, Web Alta search engine
217.20.138.*@DEF...HUNGARY, Interware hosting company
[email protected]
194.153.113.*@DEF...GERMANY, Cobion AG
[email protected], Concepts ICT BV
[email protected]
[email protected] suspicious activity
[email protected], Rambler Telecom
[email protected], Munax AB
192.114.64.0-192.11[email protected], Bezeq International, previously Trendline
92.52.8[email protected], GoViral IP Space, forged User-Agent
82.161.231.*@DEF...NETHERLANDS, Demon NL co-location customer
94.102.49.*@DEF...NETHERLANDS, Ecatel LTD
212.98.166.0-212.98[email protected], Business Network JV, suspicious sniffing
[email protected], I'VE GOT FANG INC
85.17.49.*@DEF...NETHERLANDS, Leaseweb
84.244.189.*@DEF...NETHERLANDS, I3D Interactive
[email protected], We Dare BV
194.165.42.*@DEF...NETHERLANDS, NASHIRNET-SA
[email protected], Netdirekt
[email protected], Netdirekt
[email protected], Netdirekt
85.214.16.0-85.214.139.[email protected], ServerKompetenz, Strato Rechenzentrum, Berlin
[email protected], KEYWEB
[email protected], KEYWEB
[email protected], KEYWEB
[email protected]
67[email protected], NetSource Communications, Inc
[email protected], construktiv GmbH
[email protected], LESALAB
205.205.208.*@DEF...CANADA, SureFire Commerce Inc.
[email protected], Novgorod Datacom
[email protected], Cityscape Wireless Internet
[email protected], NOA Technology INC
209.85.32.*@DEF...USA, EV1 Houston
213.236.208.*@DEF...Norway, Opera Software ASA
67.55.64.0-6[email protected], Webair Internet Development, Swish robot
69.5[email protected] Creek Telephone Company, FAKE
[email protected], Limit Group Ltd.
[email protected], Access IT
[email protected], Webazilla B.V.
[email protected], Digiweb robot
150.70.84.*@DEF...JAPAN, sniffing
[email protected], hacking attempts
[email protected], uses FORGED Cuill user-agent
[email protected], Lerkins, probing
203.27.226.0-20[email protected], Melbourne Information Technologies
202.94.128.0-202.94.159.[email protected], Sony Corporation (accessing folder /admin/bin/)
83.133.[email protected], Bigfinder.de, Great New Media
[email protected], TheNewPush, LLC
[email protected] REPUBLIC, SELECT-SYSTEMS
[email protected], NAVER robot
62.1[email protected], Aruba SPA Dedicated Servers
209.133.94.*@DEF...USA, Attributor Corp.
[email protected], iWeb Dedicated
72.2[email protected], Playstar Music Corporation
69.65.0.0/[email protected], Arliongton Heights, GigeNET
212.117.160.0/[email protected], LUXEMBOURG, King Servers
212.227.*@DEF...GERMANY, 1&1 Internet AG
65.79.128.0/[email protected], CT, Greenwich, Lamont Digital Systems, Inc.
[email protected], SC MetroNetwork SRL
[email protected], Verso QTS
195.1[email protected], Yellow Register Online AB
209.17.186.*@DEF...CANADA, Esecure Data
216.220.208.0-21[email protected], Palo Alto, Danger.com (Microsoft Corp.)
194.[email protected], Ignatius Ziekenhuis
212.112.229.*@DEF...GERMANY, IPX Server GmbH
212.247.*@DEF...SWEDEN, SwipeNet, spamming
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
193.238.228.0-[email protected], UK, CareerJet, employment database
93.190.138.*@DEF...NETHERLANDS, WorldStream
[email protected], Fingan, Ltd.
193.200.150.*@DEF...GERMANY, SEYCHELES, Anonymouse proxy
[email protected], Netvision, proxy
[email protected], AltusHost
80.93.208.224-80.93.208.255@DEF...TURKEY, Instanbul, TECHNET BILISIM ve ILETISIM HIZMETLERI
61.139.105.128-61.139.10[email protected], Sichuan, Zigong Sciences Informations Academy
[email protected], KDDI Corporation, proxy
[email protected], Softlayer Technologies
207.62.*@DEF...USA, California State University
[email protected], Paris, Dedibox SAS
140.109.0.0-140.13[email protected], National Sun Yat Sen University, hacking
[email protected], Shanghai, Unicom
216.168.32.0/[email protected], digital.forest, Inc
70.84.0.0/[email protected], The Planet
[email protected], Proxyfire
64.182.*@DEF...USA, TX, Bedford, C I Host
[email protected], proxy amenworld.com
[email protected], Linode proxy, host
62.0.18.*@DEF...ISRAEL, Netvision, proxy
216.55.182.*@DEF...USA, Abacus host, bootnetworks.com proxy
[email protected], The Planet
77.68.38.*@DEF...UK, Fast Hosts LTD
93.174.93.*@DEF...NETHERLANDS, Ecatel LTD
[email protected], Dow Jones Telerate
[email protected], XO Communications
[email protected], PaeTec Communications, Inc.
201.17/[email protected], NET Servicos de Comunicadho S.A.
[email protected], Hetzner Online AG
[email protected], BIS Ltd.
76.73.0.0/[email protected], FDC Servers
2[email protected], Red Rocks Data Center, LLC
[email protected], 3t Systems, Inc.
[email protected], Netdirekt
208[email protected], Abovenet Communications, Inc
[email protected], GANDI DEDICATED SERVERS
67[email protected], Wildblue Communications, Inc.
9[email protected], Wildblue Communications, Inc.
[email protected], Dinahosting S.L.
[email protected], Big Pipe Inc.
64[email protected], Ritter Communications, Inc.
174.142.*@DEF...CANADA, iWeb Technologies Inc.
67.228.*@DEF...USA, SoftLayer Technologies Inc.
205.[email protected], City of Jacksonville, Florida
216.2[email protected], BroadRiver Communication Corp.
189.32/[email protected], NET Servicos de Comunicadho S.A.
[email protected], ENTANET International Ltd
[email protected], OVH Dedicated Servers
[email protected], Colostore.com
209.190.0.0/[email protected], Columbus Network Access Point, Inc.
[email protected], Leaseweb
[email protected], Dreamshow Partnership
[email protected], Backslash AG
21[email protected], Rango de IPs HOSTINGLMI
66.6[email protected], FirstDigital Communications, LLC
206.135.*@DEF...USA, MegaPath Networks Inc.
194.72.238.*@DEF...UK, Netcraft Limited
[email protected], Fedem AS, Trondheim
[email protected], Optic Fusion
[email protected], The Planet
[email protected], Texas, VRT Servers
188.40.0.0/[email protected], Hetzner Online AG
[email protected], Charter Communications
[email protected], MSTAR.net LLC
72.1[email protected], GloboTech Communications GTCOMM
75.125.*@DEF...USA, The Planet
[email protected], Ecatel LTD
[email protected], Hivelocity Ventures Corp.
9[email protected], GleSYS Customer servers

(The above code updated on 2009-10-15, 11:29 GMT.) Please update your Traffic Filter to version 1.2. Please see the post dated 16 Oct 2009 for the latest configuration settings for Traffic Filter v 1.2.

The above code includes my notes (comments), because you need to know what each IP range is. If you are using the TRAFFIC FILTER version 1.1 and higher, you can keep the comments in place. If you are using an older version of the plugin, you need to remove the comments before entering this code into the plugin's blacklist. (The comments start with 3 dots ...).

REMEMBER, IP address allocations change from time to time. It is recommended that in the future, you review all those blocked IP ranges against a WHOIS record. Speaking of which, I found this excellent WHOIS website: http://cqcounter.com/whois/

[promotionbox.de]

Today, the real version 1.0 of the TRAFFIC FILTER was released with some small improvements.

Hi, where to find the version 1.0? On your homepage is still version 0.9 beta. Thanks!

-Thorsten

[screen_mates]

The proposed plugin to update/upgrade other plugins should have options to update plugins by trusted vendors (Jamit ofcourse). All other vendors should be optional and not to be included in the default updates/upgrades.

[Peter]

Please note that in the post above (CONFIGURATION example), I have updated the extensive list.

Here is what has changed:

• Added user-agent rules to block rogue robots (those that are badly forged)
• Added some countries
• Added a few more IP ranges (immediately following the Optic Fusion record - [email protected]).
• CORRECTED error. Wrong record was [email protected], Ecatel LTD. Correctly should be [email protected], Ecatel LTD.

Remember,if you are using version less than 1.1, the plugin settings require syntax such as [email protected]
so you will have to delete the 3 dots and the comment. Better that you upgrade the plugin to version 1.1.

If you are using version 1.1 of the TRAFFIC FILTER plugin, you can keep the comments in place.

Again, please use the IP ranges at your own risk. You may want to review each entry for correctness. Thank you.

[Peter]

The version 1.1 has just been released, and it includes fixes of very minor bugs and some enhancements, namely:

Now you can add a memo (comment) on each line of the blacklist.

Use of the memo (comment) is optional. If you want to use the memo, it must start with 3 dots (...) immediately following the blacklist rule. Example:

[email protected]://www.domain.com...United States users

DOWNLOAD the version 1.1 from: http://www.jamit.com/plugins.htm

UPDATING FROM EARLIER VERSIONS
 
If you are already using an earlier version of the TRAFFIC FILTER, this is the best way to update:

Do not uninstall the existing plugin! Just overwrite all the files in the folder /TrafficFilter/ with the new files that are supplied in the download package.

EXAMPLE OF CONFIGURATION (IP ranges, countries, user-agent strings)

This thread, several lines up, also has an example of blocking configuration. Please scroll up and look it up. I just updated the IP ranges today and corrected some serious errors in the example list.

USE WITH EARLIER VERSIONS OF JOB BOARD

The TRAFFIC FILTER requires JB version 3.6.0+, however, it can be used with earlier versions, if you follow the simple procedure outlined in the README file. It only requires that you add one line of code into your /include/functions.php file. (If you don't, the filter just won't work.)