Traffic Filter Plugin

[Peter]

I have installed the TRAFFIC FILTER on my Job Board site yesterday, and in 24 hours, I have blocked 45 events! Some of them were robots from Russia, trying to sign up as an employer! Definitely many attempts to hack in. The Traffic Filter made it harder to break in.

Here is one setting for the Traffic Filter, which helps in preventing code injection and MySQL injection:

Code:

AGENT#(\'|\`|\*|\?|>|<|script|eval|base64_decode)#[email protected] injection, code injection

(The above example is already included in the complete example of configuration few posts up in this thread.)

[Peter]

The version 1.2 has just been released, and it includes fixes of very minor bugs and some enhancements, namely:

Now you can add a memo (comment) on each line of the blacklist.
The plugin has a simple function to validate regular expressions.

Use of the memo (comment) is optional. If you want to use the memo, it must start with 3 dots (...) immediately following the blacklist rule. A whole line is treated as a comment if it starts with #. Examples:

Code:

########################
# COUNTRIES
US@http://www.domain.com...United States users
########################

DOWNLOAD the version 1.2 from: http://www.jamit.com/plugins/TrafficFilter.zip


UPDATING FROM EARLIER VERSIONS
 
If you are already using an earlier version of the TRAFFIC FILTER, this is the best way to update:

Do not uninstall the existing plugin! Just overwrite all the files in the folder /TrafficFilter/ with the new files that are supplied in the download package.

EXAMPLE OF CONFIGURATION (IP ranges, countries, user-agent strings)

Code:

#
# user agents
#
AGENT/^$/@DEF...empty
AGENT/^\.+$/@DEF...dots
AGENT/^[a-z0-9]{1,}$/[email protected] without whitespace
AGENT#^(Mozilla|Mozilla/[0-9]{1,}\.[0-9]{1,})$#[email protected] Mozilla
AGENT/^[a-z\ \.]{1,}$/[email protected] only
AGENT/^[0-9\ \.]{1,}$/@DEF...numeric only
AGENT/Mozzila/[email protected]
AGENT#(\'|\`|\*|\?|>|<|script|eval|base64_decode)#[email protected] injection
AGENT#MSIE\ [0-9]{1,1}\.[0-9]{1,1};\ MSIE\ [0-9]{1,1}\.[0-9]{1,1}#@DEF...2 browsers
#
# countries
#
[email protected] d'ivoire
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
#
# IP ranges
#
65.213.208.128/[email protected], Cyveillance Inc.
65.222.176.96/[email protected], Cyveillance Inc.
65.222.185.72/[email protected], Cyveillance Inc.
151.173.0.0/[email protected], Cyveillance Inc.
65.46.48.192/[email protected], Bluecoat Systems Inc.
65.160.238.176/[email protected], Bluecoat Systems Inc.
204.246.128.0/[email protected], Bluecoat Systems Inc.
208.115.138.0/[email protected], Bluecoat Systems Inc.
217.169.46.96/[email protected], Bluecoat Systems Inc.
66.194.6.0/[email protected], Websense Inc.
208.80.192.0/[email protected], Websense Inc.
204.15.64.0/[email protected], Websense Inc.
208.17.184.0/[email protected], Verisign
69.36.144.0/[email protected], Verisign
206.169.110.0/[email protected], Secure Computing
64.124.14.0/[email protected], Markmonitor
82.80.248.0/[email protected] - Bezeqint-Hosting
62.0.8.0/[email protected]
206.28.72.0/[email protected] images
200.31.42.0/[email protected], VULCO S.A.
213.246.51.0/[email protected], Ikoula Hosting
213.246.52.0/[email protected], Ikoula Hosting
209.120.218.128/[email protected], Technology Universe
83.172.144.0/[email protected], hacker attack
149.226.0.0/[email protected], BSH Bosch und Siemens Hausgeraete GmbH
216.120.128.0/[email protected], Trivalent Group Inc.
216.120.192.0/[email protected], Trivalent Group Inc.
74.52.0.0/[email protected], Houston, Texas, The Planet Internet Services
213.183.192.0/[email protected], Intares, Hamburg
216.38.192.0/[email protected], Denver, Colorado, ViaWest
94.76.219.16/[email protected], BlueConnex Ltd.
208.91.8.0/[email protected], Texas, PRONSS
128.104.0.0/[email protected], University of Wisconsin-Madison, computer lab
206.51.224.0/[email protected], Tampa, Florida, NOC4Hosts Inc.
64.62.128.0/[email protected], Fremont, California, Hurricane Electric
65.19.128.0/[email protected], Fremont, California, Hurricane Electric
208.88.120.0/[email protected], Biznesshosting Inc.
131.107.0.0/[email protected], Microsoft, secret robot
69.71.208.0/[email protected], MoveClicks LLC, Sitedossier.com
209.167.50.16/[email protected], SevenTwentyFour Incorporated
206.183.1.0/[email protected] Search
189.104/[email protected], Tele Norte, HACKER
66.90.64.0/[email protected], FDC Servers
67.159.0.0/[email protected], FDC Servers
208.53.128.0/[email protected], FDC Servers
74.63.64.0/[email protected], FDC Servers
72.232.0.0/[email protected], Layered Technologies
72.233.0.0/[email protected], Layered Technologies
64.92.160.0/[email protected], Layered Technologies
69.58.176.0/[email protected], Verisign
67.215.224.0/[email protected], Secured Private Network
64.246.160.0/[email protected], Whois, Compass Communications, Inc.
66.231.176.0/[email protected] Online Systems, Inc.
72.249.0.0/[email protected] and USA, Colo4Dallas LP, Visvo Bot
72.249.128.0/[email protected] and USA, Colo4Dallas LP, Visvo Bot
66.34.0.0/[email protected], Texas, CI Host, Keyword Spy
208.99.192.0/[email protected], Seattle, Swift Ventures
208.94.240.0/[email protected], Aarons.net, Joe's Data Center
208.43.0.0/[email protected], Softlayer Technologies
74.86.0.0/[email protected], Softlayer Technologies
38.*@DEF...USA, PSI (same as 38.0.0.0/8)
208.115.96.0/[email protected], Topshoppingcart.com, Wowcrack.com
66.232.96.0/[email protected], NOC4Hosts Inc., Hivelocity Inc.
65.98.0.0/[email protected], New Jersey, Fortress ITX
64.69.32.0/[email protected], Los Angeles, CoreExpress
208.138.176.0/[email protected], Dow Jones & Company, Savvis
208.138.192.0/[email protected], Dow Jones & Company, Savvis
208.139.0.0/[email protected], Dow Jones & Company, Savvis
208.140.0.0/[email protected], Dow Jones & Company, Savvis
208.144.0.0/[email protected], Dow Jones & Company, Savvis
208.152.0.0/[email protected], Dow Jones & Company, Savvis
208.156.0.0/[email protected], Dow Jones & Company, Savvis
208.157.0.0/[email protected], Dow Jones & Company, Savvis
208.157.128.0/[email protected], Dow Jones & Company, Savvis
71.48.0.0/[email protected], Embarq Corporation
69.84.192.0/[email protected], Arrival Communications
207.241.224.0/[email protected], San Francisco, Internet Archive
158.237.0.0/[email protected] Military, Quantico, Virginia
138.162.0.0/[email protected] Navy Network Information Center, Pensacola, Florida
74.207.224.0/[email protected], Linode, proxies
199.85.208.0/[email protected], Hostventures, proxies
216.104.0.0/[email protected], Cupertino, TrendMicro.com
65.49.0.0/[email protected], Fremont, California, Hurricane Electric
157.63.0.0/[email protected], University of Tokyo
157.64.0.0/[email protected], University of Tokyo
157.80.0.0/[email protected], University of Tokyo
157.82.0.0/[email protected], University of Tokyo
[email protected], Los Angeles, Vrtservers Inc.
65.23.128.0/[email protected], Datarealm Internet Services
[email protected], NG Marketing
202.151.[email protected], Guam, 624 North Marine Corp Drive
77.91.224.*@DEF...RUSSIA, Web Alta search engine
217.20.138.*@DEF...HUNGARY, Interware hosting company
[email protected]
194.153.113.*@DEF...GERMANY, Cobion AG
[email protected], Concepts ICT BV
[email protected]
[email protected] suspicious activity
[email protected], Rambler Telecom
[email protected], Munax AB
192.114.64.0-192.11[email protected], Bezeq International, previously Trendline
92.52.8[email protected], GoViral IP Space, forged User-Agent
82.161.231.*@DEF...NETHERLANDS, Demon NL co-location customer
94.102.49.*@DEF...NETHERLANDS, Ecatel LTD
212.98.166.0-212.98[email protected], Business Network JV, suspicious sniffing
[email protected], I'VE GOT FANG INC
85.17.49.*@DEF...NETHERLANDS, Leaseweb
84.244.189.*@DEF...NETHERLANDS, I3D Interactive
[email protected], We Dare BV
194.165.42.*@DEF...NETHERLANDS, NASHIRNET-SA
[email protected], Netdirekt
[email protected], Netdirekt
[email protected], Netdirekt
85.214.16.0-85.214.139.[email protected], ServerKompetenz, Strato Rechenzentrum, Berlin
[email protected], KEYWEB
[email protected], KEYWEB
[email protected], KEYWEB
[email protected]
67[email protected], NetSource Communications, Inc
[email protected], construktiv GmbH
[email protected], LESALAB
205.205.208.*@DEF...CANADA, SureFire Commerce Inc.
[email protected], Novgorod Datacom
[email protected], Cityscape Wireless Internet
[email protected], NOA Technology INC
209.85.32.*@DEF...USA, EV1 Houston
213.236.208.*@DEF...Norway, Opera Software ASA
67.55.64.0-6[email protected], Webair Internet Development, Swish robot
69.5[email protected] Creek Telephone Company, FAKE
[email protected], Limit Group Ltd.
[email protected], Access IT
[email protected], Webazilla B.V.
[email protected], Digiweb robot
150.70.84.*@DEF...JAPAN, sniffing
[email protected], hacking attempts
[email protected], uses FORGED Cuill user-agent
[email protected], Lerkins, probing
203.27.226.0-20[email protected], Melbourne Information Technologies
202.94.128.0-202.94.159.[email protected], Sony Corporation (accessing folder /admin/bin/)
83.133.[email protected], Bigfinder.de, Great New Media
[email protected], TheNewPush, LLC
[email protected] REPUBLIC, SELECT-SYSTEMS
[email protected], NAVER robot
62.1[email protected], Aruba SPA Dedicated Servers
209.133.94.*@DEF...USA, Attributor Corp.
[email protected], iWeb Dedicated
72.2[email protected], Playstar Music Corporation
69.65.0.0/[email protected], Arliongton Heights, GigeNET
212.117.160.0/[email protected], LUXEMBOURG, King Servers
212.227.*@DEF...GERMANY, 1&1 Internet AG
65.79.128.0/[email protected], CT, Greenwich, Lamont Digital Systems, Inc.
[email protected], SC MetroNetwork SRL
[email protected], Verso QTS
195.1[email protected], Yellow Register Online AB
209.17.186.*@DEF...CANADA, Esecure Data
216.220.208.0-21[email protected], Palo Alto, Danger.com (Microsoft Corp.)
194.[email protected], Ignatius Ziekenhuis
212.112.229.*@DEF...GERMANY, IPX Server GmbH
212.247.*@DEF...SWEDEN, SwipeNet, spamming
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
193.238.228.0-[email protected], UK, CareerJet, employment database
93.190.138.*@DEF...NETHERLANDS, WorldStream
[email protected], Fingan, Ltd.
193.200.150.*@DEF...GERMANY, SEYCHELES, Anonymouse proxy
[email protected], Netvision, proxy
[email protected], AltusHost
80.93.208.224-80.93.208.255@DEF...TURKEY, Instanbul, TECHNET BILISIM ve ILETISIM HIZMETLERI
61.139.105.128-61.139.10[email protected], Sichuan, Zigong Sciences Informations Academy
[email protected], KDDI Corporation, proxy
[email protected], Softlayer Technologies
207.62.*@DEF...USA, California State University
[email protected], Paris, Dedibox SAS
140.109.0.0-140.13[email protected], National Sun Yat Sen University, hacking
[email protected], Shanghai, Unicom
216.168.32.0/[email protected], digital.forest, Inc
70.84.0.0/[email protected], The Planet
[email protected], Proxyfire
64.182.*@DEF...USA, TX, Bedford, C I Host
[email protected], proxy amenworld.com
[email protected], Linode proxy, host
62.0.18.*@DEF...ISRAEL, Netvision, proxy
216.55.182.*@DEF...USA, Abacus host, bootnetworks.com proxy
[email protected], The Planet
77.68.38.*@DEF...UK, Fast Hosts LTD
93.174.93.*@DEF...NETHERLANDS, Ecatel LTD
[email protected], Dow Jones Telerate
[email protected], XO Communications
[email protected], PaeTec Communications, Inc.
201.17/[email protected], NET Servicos de Comunicadho S.A.
[email protected], Hetzner Online AG
[email protected], BIS Ltd.
76.73.0.0/[email protected], FDC Servers
2[email protected], Red Rocks Data Center, LLC
[email protected], 3t Systems, Inc.
[email protected], Netdirekt
208[email protected], Abovenet Communications, Inc
[email protected], GANDI DEDICATED SERVERS
67[email protected], Wildblue Communications, Inc.
9[email protected], Wildblue Communications, Inc.
[email protected], Dinahosting S.L.
[email protected], Big Pipe Inc.
64[email protected], Ritter Communications, Inc.
174.142.*@DEF...CANADA, iWeb Technologies Inc.
67.228.*@DEF...USA, SoftLayer Technologies Inc.
205.[email protected], City of Jacksonville, Florida
216.2[email protected], BroadRiver Communication Corp.
189.32/[email protected], NET Servicos de Comunicadho S.A.
[email protected], ENTANET International Ltd
[email protected], OVH Dedicated Servers
[email protected], Colostore.com
209.190.0.0/[email protected], Columbus Network Access Point, Inc.
[email protected], Leaseweb
[email protected], Dreamshow Partnership
[email protected], Backslash AG
21[email protected], Rango de IPs HOSTINGLMI
66.6[email protected], FirstDigital Communications, LLC
206.135.*@DEF...USA, MegaPath Networks Inc.
194.72.238.*@DEF...UK, Netcraft Limited
[email protected], Fedem AS, Trondheim
[email protected], Optic Fusion
[email protected], The Planet
[email protected], Texas, VRT Servers
188.40.0.0/[email protected], Hetzner Online AG
[email protected], Charter Communications
[email protected], MSTAR.net LLC
72.1[email protected], GloboTech Communications GTCOMM
75.125.*@DEF...USA, The Planet
[email protected], Ecatel LTD
[email protected], Hivelocity Ventures Corp.
[email protected], GleSYS Customer serve

USE WITH EARLIER VERSIONS OF JOB BOARD

The TRAFFIC FILTER requires JB version 3.6.0+, however, it can be used with earlier versions, if you follow the simple procedure outlined in the README file. It only requires that you add one line of code into your /include/functions.php file. (If you don't, the filter just won't work.)

[wclang]

This is great. Thanks for the hard work.

Question:

I setup my rules to follow your example (except I removed the USA ip's you blocked since I want users from USA to access my site) the question I have does anyone know of a proxy server that we can use from other contries to test if we get redirected/blocked?

Thanks,

-wclang

[Peter]

Thanks for your praise.

For your information, most of those USA-based IP addresses belong to servers that harvest data. Personally, I don't block USA users, but I am not interested in my website's data being resold, nor I am interested in these robots to put excess load on my site.

To find a free proxy, just google it! http://www.google.com.hk/#hl=en&source=hp&q=free+proxy&btnG=Google+Search&meta=&aq=f&oq=free+proxy&fp=9bb0c63c4ba974ec

There are MANY sites which provide lists of proxies. One that I found and has some free and well functioning proxies is http://www.proxy4free.com/page1.html .

On this site, I found a TRULY ANONYMOUS proxy in Myanmar, 203.81.81.37, port 80. This one is impossible to detect.

[wclang]

Oh wow, I didn't know that was why you had them in the list. Thanks for informing me; I will re-add those ip's so they are blocked as well.

Thanks for the proxy links.. I sorta forgot to look on Google.. lol..

[Peter]

It is good to be selective -- my "example" list of blocked IP's may possibly contain some errors. (I hope not!) And yes, you may not want to block all those IP ranges that I block.

You may need to double check the IP addresses from time to time. This is a good Whois service site: http://cqcounter.com/whois/

HOW TO VERIFY/CHECK WHETHER THE TRAFFIC FILTER IS WORKING

The plugin records every single redirection (blocking) event into a datatbase table. The table's name is jb_log_redirects. You can access this table using phpMyAdmin and see which requests were redirected. The table has other valuable information, such as the reason why (the redirect rule and condition).

You may want to look at the table jb_log_redirects periodically and see if your redirect rules work correctly, if you have any errors in your redirect rules.

Additionally, the database has another table called jb_log_redir_aggr, which is only the daily aggregate numbers of all redirects. Still, the most valuable table is jb_log_redirects.

[wclang]

Also, as a last resort or first attempt you can use your host provider to block ip ranges. For instance I use a hosting service that uses cpanels for easier setup and such. I noticed one of the options is security and within there is a link for "IP Deny Manager - This feature will allow you to block a range of IP addresses to prevent them from accessing your site. You can also enter a fully qualified domain name, and the IP Deny Manager will attempt to resolve it to an IP address for you."




Just thought I would share another option with everyone.

Hope it helps...

-wclang

[steve]

The TRAFFIC FILTER requires JB version 3.6.0+, however, it can be used with earlier versions, if you follow the simple procedure outlined in the README file. It only requires that you add one line of code into your /include/functions.php file. (If you don't, the filter just won't work.)

should that read ...version 3.5.0+?

I've looked and the most recent version of Jamit is 3.5.3 

[mshanley]

I like the traffic filter concept, i will probably install it as well. But - I have been in need for a good IP filter for some time without using / overloading the web server.
I was playing around with PeerGuardian and a couple other tools.. but they seemed a little buggy and the new version locks up once i add all the countries..

Anyway.. I'm int he USA and I don't want / need anyone on my web server or my e-mail server..  so  I found a new tool.. after the trial i bought it.. it's under $50.. and works great (much better then peer guardian ever did)

Take a look.. runs as it's own apllication - has a lot of cool features..  then you can add the special blocks to httaccess or traffic filter

If your interested it's called Beethink IP Blocker....  http://www.beethink.com/  $29 after trial...    IT also LOGs the IP's
you can download complete country ip lists and use them.

[Peter]

...

should that read ...version 3.5.0+?

I've looked and the most recent version of Jamit is 3.5.3 
....

Yes, the most recent version is 3.5.3. Yes, the plugin requires JB version 3.6.0+, however, a simple modification outlined in the instructions will allow you to use it with earlier versions.

[Peter]

I like the traffic filter concept, i will probably install it as well. But - I have been in need for a good IP filter for some time without using / overloading the web server.
I was playing around with PeerGuardian and a couple other tools.. but they seemed a little buggy and the new version locks up once i add all the countries..

Anyway.. I'm int he USA and I don't want / need anyone on my web server or my e-mail server..  so  I found a new tool.. after the trial i bought it.. it's under $50.. and works great (much better then peer guardian ever did)

Take a look.. runs as it's own apllication - has a lot of cool features..  then you can add the special blocks to httaccess or traffic filter

If your interested it's called Beethink IP Blocker....  http://www.beethink.com/  $29 after trial...    IT also LOGs the IP's
you can download complete country ip lists and use them.

You are referring to a system which runs on client side and has nothing to do with the server (which holds your Job Board). The systems you mention (incl. Peer Guardian) will not protect anything on the server, but only on your home PC.

[Peter]

Also, as a last resort or first attempt you can use your host provider to block ip ranges. For instance I use a hosting service that uses cpanels for easier setup and such. I noticed one of the options is security and within there is a link for "IP Deny Manager .......

-wclang

Yes, thank you for mentioning this. Your hosting account uses cPanel, where you can access the IP Deny Manager. Effectively, the IP Deny manager writes (or edits) the .htaccess file, which tells the Apache server how to process HTTP requests.

I believe that I have also mentioned in my earlier post that you can create/edit your own .htaccess file. (You don't have to use the cPanel's IP Deny Manager.)

Here is an example of code inside the .htaccess file:

Code:

<Limit GET HEAD POST>
order allow,deny
# Country: COTE D'IVOIRE
# ISO Code: CI
# Total Networks: 16
# Total Subnets:  112,896
deny from 41.189.32.0/19
deny from 41.189.96.0/19
deny from 41.191.68.0/22
deny from 41.202.64.0/19
deny from 41.202.96.0/19
deny from 41.202.128.0/19
deny from 41.206.64.0/19
deny from 41.207.0.0/19
deny from 41.207.192.0/19
deny from 41.216.240.0/20
deny from 41.223.208.0/22
deny from 196.47.128.0/18
deny from 196.201.64.0/19
deny from 196.223.4.0/24
deny from 213.136.96.0/19
deny from 213.150.192.0/19
#
allow from all
</Limit>

ADVANTAGE OF USING .htaccess
Since .htaccess controls Apache's behavior, you have effectively moved your traffic filtering down one level, and the server will not have to work as hard.

DISADVANTAGE OF USING .htaccess AND WHY THE TRAFFIC FILTER IS USEFUL
The Traffic Filter plugin takes advantage of the power of the PHP code, which is more powerful than the notation in .htaccess. The Traffic Filter can do more. The Traffic Filter will log all events in the database and you know whom you are blocking, when and how many. You will also know if you are blocking legitimate traffic. If you rely only .htaccess, you have no idea what is going on.

As I have suggested earlier, you could verify your redirect (blocking) rules inside the Traffic Filter first. After that, you move those rules into the .htaccess file.

There are many resources on the web about the .htaccess file. One of them is http://www.askapache.com/htaccess/htaccess.html .

You should exercise extreme CAUTION when messing with the .htaccess file.

[Stranger]

My email server (hmailserver) is currently under attack from China.

I host my board on my server here behind me.

I'm trying to configure the Traffic Filter and I get the following error when I click the link to install the Database tables;

"Fatal error: Maximum execution time of 30 seconds exceeded in E:\webservices\htdocs\XXXXXXXXXjobs\db.php on line 40"

How do I get around this?

I have included the code for the 2 lines in "include/functions.php.

I will be getting a new router that will allow me to block certain IP addresses in the very near future, also.

Any help is appreciated.

Thanks.

[Peter]

......
I'm trying to configure the Traffic Filter and I get the following error when I click the link to install the Database tables;

"Fatal error: Maximum execution time of 30 seconds exceeded in E:\webservices\htdocs\XXXXXXXXXjobs\db.php on line 40"

How do I get around this?........

Hi Stranger,

Sorry for late reply.

Here is the fix for your situation. (Not every user needs to do this. Apparently, your server is very, very slow.)

In the folder /TrafficFilter/ is a file called install_tables.php . Open it and in the top part of the file you will find:

Code:

$queryLimit  = 5000; // how many queries to be executed in each step; suggested 5000
$pageRefresh = 5;   // seconds for each installation step; suggested 5 sec

Change the above number to:

Code:

$queryLimit  = 1000;
$pageRefresh = 20;

This will make the database installation very slow, which is OK, since it is done only when installing the Traffic Filter (database).

If your server is very slow, you may need to make the number even smaller.

Peter

[Peter]

Version 2.2 of the Traffic Filter has been released and is available at http://market.jamit.com/.

The new version includes bug fixes, functionality enhancements (redirect/blocking rules by host name, referrer), performance, user interface (GUI).

Included is also an EXAMPLE CONFIGURATION file, which contains millions of malicious IP addresses and settings to prevent many other threats, spamming and data mining robots. I have been compiling this file since August 2008, so the value of this file is tremendous!

My plan is to publish a new version with updated example configuration file about every 6 months unless I have some other major improvements that warrant earlier release.

Existing users are advised to upgrade.

Peter