[prev in list] [next in list] [prev in thread] [next in thread] 

List:       squid-users
Subject:    [squid-users] squid and AD configuration guidelines
From:       "Paul Mattingly" <Paul.Mattingly () concordelogistics ! com>
Date:       2006-02-27 12:03:45
Message-ID: 78B07F5DC9E7524E92B98C936EF801C401466FD4 () southallex01 ! ad ! concordelogistics ! com
[Download RAW message or body]

Hi

I have spent the last few months getting Squid to work seamlessly in a Windows 2003 \
AD environment. Being an MCSE I had very little *NIX knowledge but I had to try Squid \
out as ISA was not an option.

I would like to share my configuration with others so hopefully I can provide the \
same help I received. I make no guarantees, this is not a complete how-to, it's just \
what I did to get things running in my particular environment with the software \
versions specified. There is much improvement to be made and a great deal for me to \
learn, but this is working just fine at the moment.

Please please try this in a test environment first. I was dumb enough not to do so \
and ended up killing a production DC when trying to join the squid machine to the \
domain. An error in smb.conf over-wrote the DC's computer account in AD! Oops. I just \
treated the situation as if the DC had an unrecoverable hardware failure. Following \
an MS article, I removed the DC from AD by hand and rebuilt it under a new name. I \
felt this was the only way to be sure, and everything is back to normal now! Won't be \
forgetting that in a hurry; what doesn't kill you (or the network) can only make you \
stronger! :-P

So here we go  : - )


Hardware
ŻŻŻŻŻŻŻŻ
HP Netserver LC 2000 U3
Pentium III/1000Mhz
512MB RAM
1x18GB SCSI drive
2x36GB SCSI drive

I created two RADI0 volumes, one with one disk and one with two disks. This favours \
performance over fault-tolerance.


Software
ŻŻŻŻŻŻŻŻ
FreeBSD 6.0-RELEASE		http://www.freebsd.org/
Squid 2.5 STABLE12		http://www.squid-cache.org/
Samba 3.0.21a			http://www.samba.org/
Windows 2003 SP1 Active Directory environment


Operating System setup
ŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻ
FreeBSD was loaded with standard partitions (/ /var /usr and swaps) on the first \
disk. I created one large partition mounted /disk1 for the cache on the second disk. \
The OpenLDAP libraries from the ports collection are required to communicate with AD. \
You can use sysinstall during installation or later to install this. Configure, \
Packages, Select Media, Net, openldap-client-2.2.27. Next came the user and group \
accounts to run squid under. These were called proc_squid and grp_squid and created \
in the normal way as per the handbook. To allow use of the cache manager, Apache 1.3 \
was installed from /usr/ports/www/apache13/


Samba
ŻŻŻŻŻ
Samba is required to facilitate transparent NTLM authentication. Only winbind ends up \
running so it seems overkill to install the whole package. Follow the installation \
instructions and make sure to add

--with-winbind --with-ads

when you run the configure script. If you get errors that relate to LDAP not being \
installed you can specify where the libs are like this. I imagine this will vary \
between OSs, this is what FreeBSD required.

--libdir=/usr/local/lib/
--includedir=/usr/local/include/

You can use the smb.conf at the bottom of this page as a guideline for your own to \
get Samba running. An excellent FAQ is located at \
http://www.squid-cache.org/Doc/FAQ/FAQ-23.html which describes testing procedures. \
The only program I used from Samba was ntlm-auth which in turn relies on winbindd to \
function. This will authenticate the user transparently and pass the details of the \
account to Squid via the external helpers setup. Rather more info than you need (!) \
can be found here http://us1.samba.org/samba/docs/man/Samba-HOWTO-Collection/winbind.html


There is also an excellent guide regarding Samba and squid here.
http://pserver.samba.org/samba/docs/man/Samba-Guide/DomApps.html

The squid machine has to be joined to the AD domain, and you can do this with the \
following command

/usr/local/samba/bin/net ads join -U administrator%password

While you are in AD U+C checking the account is OK, you might as well create the \
account which the LDAP program will use to authenticate. Just a regular user account \
with no access will do just fine. Use the credentials when constructing the \
squid_ldap_group command line as detailed below.

There is a section in one of the FAQs about using a cron job to cycle the computer \
account password every so often. It's not obvious whether this is required or not, I \
certainly haven't had to do it yet. However, if the authentication should break down \
unexpectedly, it's one of the first things I will look at!

I encountered various different errors here and a summary follows

	BH NT_STATUS_ACCESS_DENIED

	[2005/12/14 14:12:09, 0] utils/ntlm_auth.c:winbind_pw_check(439)
	Login for user [DOMAIN]\[USER]@[SQUIDTEST] failed due to [winbind
	client not authorized to use winbindd_pam_auth_crap. Ensure permissions on
	/var/db/samba/winbindd_privileged are set correctly.]
	The permissions on /var/db/samba/winbindd_privileged is not set correctly

The directory I had to check was /usr/local/samba/var/locks/winbindd_privileged/

	User: root or cache_effective_user
	Group: cache_effective_group
	Permissions: o=rwx, g=rx, o=

Then everything was OK.

	squidhp# ./ntlm_auth --helper-protocol=squid-2.5-ntlmssp
	squid\administrator password
	[2006/02/01 10:23:18, 1] utils/ntlm_auth.c:manage_squid_ntlmssp_request(578)
	BH

Above is an example of testing the ntlm_auth program. I never got this to work \
properly by hand, but squid seems happy with it! It's an error that doesn't need \
fixing.

You will be ready to proceed if you are at the following position

winbindd running (use winbinnd -D to invoke)
wbinfo -t returns 'secret is good' or 'checking the trust secret via RPC calls \
succeeded' wbinfo -g return a list of your groups something like

DOMAIN\domain guests
DOMAIN\domain users
DOMAIN\group policy creator owners
etc....

wbinfo -u does the same as above for users


Squid
ŻŻŻŻŻ
Squid is now ready to be loaded. You must use

--enable-auth="basic,ntlm"
--with-external-acl-helpers="ldap_group"

There was a major problem with getting the ldap group program to compile properly. It \
couldn't find the ldap libraries even though I tried to specify them in the Makefile \
file. I ended up copying all the files related to ldap so there was a copy in both \
/usr/include/ and /usr/local/include. This was pretty messy but I did not have \
another option at the time. The error message

cannot find -lldap

also came up a few times. This was sorted by editing this file

../squid-2.5.STABLE12/helpers/external_acl/ldap_group/Makefile

The variable LDFLAGS must read

LDFLAGS = -g -L/usr/local/lib

Squid should compile with no errors and a squid_ldap_group executable should be \
created in the external helpers ldap_group directory

It's a good idea to test squid_ldap_group by hand at this point. The manual pages and \
help switch are useful. Here is the command line extracted from squid.conf

/squid-2.5.STABLE12/helpers/external_acl/ldap_group/squid_ldap_group -b \
"ou=example_OU,dc=example,dc=domain,dc=com" -f \
"(&(cn=%a)(member=%v)(objectClass=group))" -F "(|(samAccountName=%s)(cn=%s))" -h \
DC_hostname.example.domain.com -D username -w password -v3 -S

Entering a username and then a group separated by a space will return either OK or \
ERR depending on their membership. It appears that the program is more than just a \
membership lookup routine. Through testing, I discovered that each filter must \
evaluate to true for OK to be returned. So you can customize them to whatever \
criteria you like. The example above checks for a group with the user present in it \
and the fact that the user exists. It also checks the base OU specified and the whole \
tree beneath it.

All that was left was to take ownership of the appropriate directories, create the \
cache folders (I created /disk1/squid/var/cache/ ) and start winbindd and squid.

I used chmod and chown with -R to recursively set ownership and permissions for the \
cache directories and the other two squid folders. This may be overkill.

/usr/local/squid/sbin/squid -z  will initialise the cache folders

/usr/local/squid/sbin/squid -NCd1  is good for the first time you start as it will \
send debugging messages straight to the console. Just run ../squid on it's own when \
you are happy for squid to run in the background.

Samba documentation says you need smbd and nmbd but I found that it worked without \
either of them. I read a few documents that mentioned NSSWITCH and KRB5 configuration \
files but I never created or modified either of these.

If you see multiple ntlm_auth and squid_ldap_group processes this is normal. 5 \
processes are spawned by default to ensure all requests are handled efficiently. My \
server is very very quiet at the moment (0.8% CPU usage on average, 23 users) so I \
have reduced this to 3 processes for the moment. This is specified in squid.conf \
under auth_param ntlm children n. I feel that squid performance is crucial and hope \
to investigate this area further.


Squid ACLS
ŻŻŻŻŻŻŻŻŻŻ
My setup includes three groups of users. Those with no restriction whatsoever, those \
whose must pass a blacklist and those who must pass a whitelist. You can see how this \
was implemented from the squid.conf below. The cunning thing about this syntax is \
that if a user is accidentally joined to more than one of the internet groups in AD, \
the most restrictive group will apply. There is also system wide blocking for ads and \
unapproved subnets. Note the line 'acl auth_users proxy_auth REQUIRED' which ensures \
that any user connecting must undergo authentication. I have left out Basic as the \
only clients that will be connecting are IE and Firefox. Firefox 1.5 appears to \
support NTLM now, which is contrary to some articles I have read. There were no \
pop-ups and it worked transparently just as IE does.


Custom Error Messages
ŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻ
I have also created some custom error message which makes troubleshooting a lot \
easier. Different pages will come up for different errors so the user can immediately \
relay the problem they are having. This FAQ will help.

http://www.squid-cache.org/Doc/FAQ/FAQ-10.html#ss10.24

See below for my ad blocking message. I was trying to replace ads with the minimum of \
information. Squid will add a footer at the bottom of the page (see the FAQ) but the \
%s displays just the squid version which reduces the info a fair bit.


Cache Manager
ŻŻŻŻŻŻŻŻŻŻŻŻŻ
See below for the additional lines in httpd.conf which hosts cachemgr.cgi. This was a \
very quick install but I managed to limit the number of httpd servers and add a \
password. squid.conf holds the password under 'cachemgr_passwd password all' and you \
can edit MinSpareServers and StartServers within httpd.conf. I have these both set at \
1 because I can't foresee a tremendous amount of traffic heading that way.


smb.conf
ŻŻŻŻŻŻŻŻ
[global]
security = ads
password server = DC_hostname.example.domain.com
realm = EXAMPLE.DOMAIN.COM   #must be in CAPS
workgroup = DOMAIN_NETBIOS_NAME
encrypt passwords = yes
idmap uid = 10000 - 20000
idmap gid = 10000 - 20000
winbind enum users = yes
winbind enum groups = yes
log file = /var/log/log.%m
winbind separator = \\


squid.conf
ŻŻŻŻŻŻŻŻŻŻ
http_port 3128
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
cache_dir ufs /disk1/squid/var/cache 20000 16 256
debug_options ALL,1 33,2

auth_param ntlm program /usr/local/samba/bin/ntlm_auth \
--helper-protocol=squid-2.5-ntlmssp auth_param ntlm children 3
auth_param ntlm max_challenge_reuses 0
auth_param ntlm max_challenge_lifetime 2 minutes
auth_param ntlm use_ntlm_negotiate on

refresh_pattern ^ftp:		1440	20%	10080
refresh_pattern ^gopher:	1440	0%	1440
refresh_pattern .		0	20%	4320

acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443 563
acl Safe_ports port 80		# http
acl Safe_ports port 21		# ftp
acl Safe_ports port 443 563	# https, snews
acl Safe_ports port 70		# gopher
acl Safe_ports port 210		# wais
acl Safe_ports port 1025-65535	# unregistered ports
acl Safe_ports port 280		# http-mgmt
acl Safe_ports port 488		# gss-http
acl Safe_ports port 591		# filemaker
acl Safe_ports port 777		# multiling http
acl CONNECT method CONNECT

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

acl subnet src "/usr/local/squid/etc/subnet.txt"
deny_info ERR_SUBNET subnet

acl ads url_regex "/usr/local/squid/etc/adurls.txt"
deny_info ERR_ADBLOCK ads

acl ads2 url_regex "/usr/local/squid/etc/adurls2.txt"
deny_info ERR_ADBLOCK ads2

acl badwords url_regex "/usr/local/squid/etc/badwords.txt"
acl company_site_dom dstdomain "/usr/local/squid/etc/companydomains.txt"
acl company_site_url url_regex "/usr/local/squid/etc/companyurls.txt"

external_acl_type ldap_group ttl=0 children=3 %LOGIN \
../squid-2.5.STABLE12/helpers/external_acl/ldap_group/squid_ldap_group -b \
"ou=example_OU,dc=example,dc=domain,dc=com" -f \
"(&(cn=%a)(member=%v)(objectClass=group))" -F "(|(samAccountName=%s)(cn=%s))" -h \
DC_hostname.example.domain.com -D username -w password -v3 -S

acl full 		external ldap_group full_internet_access
acl restricted 	external ldap_group restricted_internet_access
acl company 	external ldap_group company_approved_internet_access

acl auth_users proxy_auth REQUIRED

http_access deny ads
http_access deny ads2
http_access deny !subnet

http_access allow company company_site_url
http_access allow company company_site_dom
http_access deny  company !company_site_url
http_access deny  company !company_site_dom

http_access allow restricted !badwords
http_access deny restricted badwords

http_access allow full

http_access deny !auth_users
http_access deny all

http_reply_access allow all
icp_access allow all

cache_mgr helpdesk@company.com

cache_effective_user proc_squid
cache_effective_group grp_squid

visible_hostname Squid

cachemgr_passwd password all

coredump_dir /disk1/squid/var/cache


httpd.conf
ŻŻŻŻŻŻŻŻŻŻ
ScriptAlias /squid/cgi-bin/ /usr/local/squid/libexec/
<Location /squid/cgi-bin/cachemgr.cgi>
order allow,deny
allow from workstation squid_IP
</Location>


Custom error message
ŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻ
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" \
"http://www.w3.org/TR/html4/loose.dtd"> <HTML><HEAD><META HTTP-EQUIV="Content-Type" \
CONTENT="text/html; charset=iso-8859-1"> <TITLE>ERROR: The requested URL could not be \
retrieved</TITLE> <STYLE \
type="text/css"><!--BODY{background-color:#ffffff;font-family:verdana,sans-serif}PRE{font-family:sans-serif}--></STYLE>
 </HEAD><BODY>
Ad blocked by %s


I am very impressed with Squid, it's a worthy rival to it's competitors. Hopefully \
this guide is of some help to you and I welcome any comments and suggestions. As I \
said before, this is no guaranteed guide, it's just what worked in my environment.


Paul


[prev in list] [next in list] [prev in thread] [next in thread]