The first thing to know is that there are 4 different ways to do this on the SRX.
1) Surfcontrol - This is know as Integrated Web Filtering and uses a cloud based Surfcontrol server to categorise URLs. This option requires a license and of course you can test with the 30 day trial license. The cloud based Surfcontrol server doesn't permit or deny anything. When queried by the SRX it provides a category that is permitted or denied by the policy you create on the SRX. The URL-to-category site results from the Surfcontrol server are cached locally which means subsequent requests for the same site don't require a lookup on the Surfcontrol server. This is a sensible approach as it means you dont have to download the whole database from Surfcontrol every time you start the SRX - it just requests the info from Surfcontrol on as as needed realtime basis.
Note: Surfcontrol is now owned by Websense though Juniper still name this option as Surfcontrol.
2) Enhanced Web Filtering - This is another type of Integrated solution which uses the Websense ThreatSeeker Cloud (TSC). Similar to Surfcontrol option above it provides a cattergory for the site to be checked but in addition also provides a reputation for the site.
A license is required for this option.
Juniper consider this option "next-generation URL filtering solution, building upon the existing SurfControl solution".
3) Websense - This is known Redirect Web Filtering and uses your local Websense server. This option does not require a licence. The main difference here to the Surfcontrol option is the the Websense server itself, through its own polices is doing the denying or permitting.
4) Local lists. This option uses local black and white lists to permit or deny access to URLs. No license required.
In this post I will configure and test the Local lists
CONFIGURATION
a) Configure UTM URL Pattern Custom Objects
Lets start by trying to block access to Cisco and permitting access to Juniper.
So we stick Juniper and Cisco URLs in good and bad lists respectively
blogger@LEFTY# set security utm custom-objects url-pattern good-list value http://www.juniper.net
blogger@LEFTY# set security utm custom-objects url-pattern bad-list value http://www.cisco.com
Note: If you know the IPs of the sites you can add them too. Also wildcards * and ? can be used as well with some limitations on use.
b) Put the Custom Objects in Custom URL Categories
blogger@LEFTY# set security utm custom-objects custom-url-category good-category value good-list
blogger@LEFTY# set security utm custom-objects custom-url-category bad-category value bad-list
Lets commit that and see what it looks like...
blogger@LEFTY# run show configuration security utm
custom-objects {
url-pattern {
good-list {
value http://www.juniper.net;
}
bad-list {
value http://www.cisco.com;
}
}
custom-url-category {
good-category {
value good-list;
}
bad-category {
value bad-list;
}
}
}
c) Configure the Web Filtering Feature Profile
blogger@LEFTY# set security utm feature-profile ?
Possible completions:
> anti-spam Configure anti-spam feature
> anti-virus Configure anti-virus feature
+ apply-groups Groups from which to inherit configuration data
+ apply-groups-except Don't inherit configuration data from these groups
> content-filtering Configure content filtering feature
> web-filtering Configure web-filtering feature
[edit]
blogger@LEFTY# set security utm feature-profile web-filtering ?
Possible completions:
+ apply-groups Groups from which to inherit configuration data
+ apply-groups-except Don't inherit configuration data from these groups
> juniper-enhanced Configure web-filtering juniper enhanced engine
> juniper-local Configure web-filtering juniper local engine
> surf-control-integrated Configure web-filtering surf-control integrated engine
> traceoptions Trace options for web-filtering feature
type Configure web-filtering engine type
url-blacklist Configure custom URL for blacklist category
url-whitelist Configure custom URL for whitelist category
> websense-redirect Configure web-filtering websense redirect engine
blogger@LEFTY# set security utm feature-profile web-filtering url-blacklist bad-category
blogger@LEFTY# set security utm feature-profile web-filtering url-whitelist good-category
blogger@LEFTY# set security utm feature-profile web-filtering type ?
Possible completions:
juniper-enhanced
juniper-local
surf-control-integrated
websense-redirect
blogger@LEFTY# set security utm feature-profile web-filtering type juniper-local
blogger@LEFTY# set security utm feature-profile web-filtering juniper-local ?
Possible completions:
+ apply-groups Groups from which to inherit configuration data
+ apply-groups-except Don't inherit configuration data from these groups
> profile Juniper local profile
blogger@LEFTY# set security utm feature-profile web-filtering juniper-local profile wf-profile ?
Possible completions:
+ apply-groups Groups from which to inherit configuration data
+ apply-groups-except Don't inherit configuration data from these groups
custom-block-message Juniper local custom block message
default Juniper local profile default
> fallback-settings Juniper local fallback settings
timeout Juniper local timeout (1..1800 seconds)
blogger@LEFTY# set security utm feature-profile web-filtering juniper-local profile wf-profile custom-block-message ">>NOT PERMITTED<<"
blogger@LEFTY# set security utm feature-profile web-filtering juniper-local profile wf-profile fallback-settings ? Possible completions:
<[Enter]> Execute this command
+ apply-groups Groups from which to inherit configuration data
+ apply-groups-except Don't inherit configuration data from these groups
default Fallback default settings
server-connectivity Fallback action when device cannot connect to server
timeout Fallback action when connection to server timeout
too-many-requests Fallback action when requests exceed the limit of engine
| Pipe through a command
blogger@LEFTY# set security utm feature-profile web-filtering juniper-local profile wf-profile fallback-settings too-many-requests ? Possible completions:
block
log-and-permit
blogger@LEFTY# set security utm feature-profile web-filtering juniper-local profile wf-profile fallback-settings too-many-requests block
blogger@LEFTY# set security utm feature-profile web-filtering juniper-local profile wf-profile fallback-settings default ?
Possible completions:
block
log-and-permit
blogger@LEFTY# set security utm feature-profile web-filtering juniper-local profile wf-profile fallback-settings default block
blogger@LEFTY# run show configuration security utm feature-profile web-filtering
url-whitelist good-category;
url-blacklist bad-category;
type juniper-local;
juniper-local {
profile wf-profile {
default log-and-permit;
custom-block-message ">>NOT PERMITTED<<";
fallback-settings {
default block;
too-many-requests block;
}
}
}
d) Apply the Web Filtering profile to the UTM Policy
blogger@LEFTY# set security utm utm-policy ?
Possible completions:
<name> UTM policy name
[edit]
blogger@LEFTY# set security utm utm-policy utm-protect ?
Possible completions:
> anti-spam Anti-spam profile
> anti-virus UTM policy anti-virus profile
+ apply-groups Groups from which to inherit configuration data
+ apply-groups-except Don't inherit configuration data from these groups
> content-filtering Content-filtering profile
> traffic-options Traffic options
> web-filtering Web-filtering profile
blogger@LEFTY# set security utm utm-policy utm-protect web-filtering http-profile wf-profile
e) Add the UTM Policy to the Security Policy
blogger@LEFTY> show configuration security policies from-zone trust to-zone untrust
policy trust-to-untrust {
match {
source-address any;
destination-address any;
application any;
}
then {
permit {
application-services {
idp;
utm-policy utm-protect;
}
}
}
}
Note: we still have IDP running on the policy so these features ca co-exist.
Quite a few steps there to get this all going In summary:
* Create the custom objects - the URLs you want to permit or deny
* Apply the custom objects to custom URL categories
* Configure the web filtering feature profile which uses the custom URL categories
* Apply the web filtering feature profile to the UTM policy
* Apply the UTM policy to the security policy
TESTING
First lets consider the order checking of the lists involved
1) Check black list - drop if a match is found
2) Check white list - permit if a match is found
3) Check the default fallback action and apply as appropriate - If there is no fallback action then permit.
Test setup is as follows:
Trust side - an XP PC host
Untrust side - the Internet.
First check web filtering status
blogger@LEFTY> show security utm web-filtering status
UTM web-filtering status:
Server status: Juniper local URL filtering
Now check stats before doing anything.
blogger@LEFTY> show security utm web-filtering statistics
UTM web-filtering statistics:
Total requests: 0
white list hit: 0
Black list hit: 0
Web-filtering sessions in total: 4000
Web-filtering sessions in use: 0
Fallback: log-and-permit block
Default 0 0
Timeout 0 0
Connectivity 0 0
Too-many-requests 0 0
Ok, now we will access both Cisco and Juniper sites and see what happens.
Juniper worked and we got this response in our browser when trying to access the Cisco site...
Lets check the stats...
blogger@LEFTY> show security utm web-filtering statistics
UTM web-filtering statistics:
Total requests: 31
white list hit: 7
Black list hit: 2
Web-filtering sessions in total: 4000
Web-filtering sessions in use: 0
Fallback: log-and-permit block
Default 0 0
Timeout 0 0
Connectivity 0 0
Too-many-requests 0 0
Hits on the black and white lists as expected.
Now lets try the fallback option by accessing a site not in either list - www.checkpoint.com
It worked as the default option is log-and-permit.
Lets find that log then..
First I setup a log file with below config and then try again,
blogger@LEFTY> show configuration system syslog file WEB
any any;
match WEBFILTER;
archive size 500k files 3 world-readable;
Well that just generated 19 lines of logging. Looks like its logging every single element as its downloaded from the page. Here is a few lines form the top...
blogger@LEFTY> show log WEB
Aug 26 15:36:34 LEFTY clear-log[5796]: logfile clearedblogger@LEFTY> show log WEB
Aug 26 15:36:54 LEFTY RT_UTM: WEBFILTER_URL_PERMITTED: WebFilter: ACTION="URL Permitted" 10.10.10.50(1398)->216.200.241.66(80) CATEGORY="N/A" REASON="BY_OTHER" PROFILE="wf-profile" URL=www.checkpoint.com OBJ=/ USERNAME=N/A ROLES=N/A
Aug 26 15:36:55 LEFTY RT_UTM: WEBFILTER_URL_PERMITTED: WebFilter: ACTION="URL Permitted" 10.10.10.50(1403)->184.87.21.68(443) CATEGORY="N/A" REASON="BY_OTHER" PROFILE="wf-profile" URL=184.87.21.68 OBJ=/ USERNAME=N/A ROLES=N/A
Aug 26 15:36:55 LEFTY RT_UTM: WEBFILTER_URL_PERMITTED: WebFilter: ACTION="URL Permitted" 10.10.10.50(1402)->216.200.241.66(80) CATEGORY="N/A" REASON="BY_OTHER" PROFILE="wf-profile" URL=www.checkpoint.com OBJ=/css/homepage2012.css USERNAME=N/A ROLES=N/A
Aug 26 15:36:57 LEFTY RT_UTM: WEBFILTER_URL_PERMITTED: WebFilter: ACTION="URL Permitted" 10.10.10.50(1404)->203.206.129.66(80) CATEGORY="N/A" REASON="BY_OTHER" PROFILE="wf-profile" URL=download.zonealarm.com OBJ=/bin/js/lib-plugins/jquery.cycle.all.2.72.js USERNAME=N/A ROLES=N/A
Ok,so know that logging permits is going to generate a hell of a lot of logs. So I am changing the default to just permit, clear the log and try Checkpoint again...
blogger@LEFTY# set security utm feature-profile web-filtering juniper-local profile wf-profile default permit
And we check the log and the log is empty.
Lets try now changing the default to block and try Checkpoint again.
Here is what we see...
And here is the log...
blogger@LEFTY> show log WEB
Aug 26 15:55:43 LEFTY clear-log[5872]: logfile cleared
Aug 26 15:55:50 LEFTY RT_UTM: WEBFILTER_URL_BLOCKED: WebFilter: ACTION="URL Blocked" 10.10.10.50(1473)->216.200.241.66(80) CATEGORY="N/A" REASON="BY_OTHER" PROFILE="wf-profile" URL=www.checkpoint.com OBJ=/ USERNAME=N/A ROLES=N/A
For comparison lets access Cisco (Still blocked by the Black list) again and see what the log shows...
blogger@LEFTY> show log WEB
Aug 26 16:51:01 LEFTY clear-log[5931]: logfile cleared
Aug 26 16:51:47 LEFTY RT_UTM: WEBFILTER_URL_BLOCKED: WebFilter: ACTION="URL Blocked" 10.10.10.50(1479)->184.87.16.170(80) CATEGORY="bad-category" REASON="BY_BLACK_LIST" PROFILE="wf-profile" URL=www.cisco.com OBJ=/ USERNAME=N/A ROLES=N/A
So a couple of points here..
* Sites that get blocked seem to log even though there is no action specified in the config to log.
* Reason "BY_OTHER" refers to the default action taking place. Whereas if a site is blocked by the actual back list then the reason in the log is "BY_BLACK_LIST"
Lets now remove the white list from the feature profile and reset the default action to permit.
blogger@LEFTY# delete security utm feature-profile web-filtering url-whitelist
blogger@LEFTY# set security utm feature-profile web-filtering juniper-local profile wf-profile default permit
So the web filtering feature profile looks like this
blogger@LEFTY# run show configuration security utm feature-profile web-filtering
url-blacklist bad-category;
type juniper-local;
juniper-local {
profile wf-profile {
default permit;
custom-block-message ">>NOT PERMITTED<<";
fallback-settings {
default block;
too-many-requests block;
}
}
}
Result - Checkpoint and Juniper (Which are now not in any specific list) work and Cisco fails.
At the end of the day what combination of lists you choose to use depends on whether you are trying to explicitly permit or block.
The fallback settings come into play when the SRX can not process any more web filtering traffic.
Here, to finish up this post is the complete UTM Web Filtering Local config with colour highlights showing how all the different elements "glue" together.
blogger@LEFTY> show configuration security utm
custom-objects {
url-pattern {
good-list {
value http://www.juniper.net;
}
bad-list {
value http://www.cisco.com;
}
}
custom-url-category {
good-category {
value good-list;
}
bad-category {
value bad-list;
}
}
}
feature-profile {
web-filtering {
url-blacklist bad-category;
type juniper-local;
juniper-local {
profile wf-profile {
default permit;
custom-block-message ">>NOT PERMITTED<<";
fallback-settings {
default block;
too-many-requests block;
}
}
}
}
}
utm-policy utm-protect {
web-filtering {
http-profile wf-profile;
}
}
blogger@LEFTY> show configuration security policies
from-zone trust to-zone untrust {
policy trust-to-untrust {
match {
source-address any;
destination-address any;
application any;
}
then {
permit {
application-services {
idp;
utm-policy utm-protect;
}
}
}
}
}
Model: srx100h
JUNOS Software Release [12.1R3.5]
I have tired following Juniper's configuration guides but they make it confusing. Your instructions were clear and it worked! thanks!
ReplyDelete