Sunday, April 26, 2015

clearos 6.4 + Squid 3.5.2 Gateway Mode https caching

clearos 6.4 + Squid 3.5.2 Gateway Mode https caching

clearos 6.4 + squid 3.5.2 (Gateway Mode [https cache :P)
http://forum.clearos.or.id/index.php?topic=2458.0

# yum update 

# yum-config-manager --enable clearos-core

# yum --enablerepo=clearos-core,clearos-developer,clearos-epel install clearos-devel app-devel

# yum install openssl openssl-devel fakeroot ccze ebtables perl-File-ReadBackwards libcap libcap-devel libtool-ltdl-devel automake gcc glibc-devel e2fsprogs-devel sharutils

# yum --enablerepo=* install libtool libtool-ltdl libtool-ltdl-devel

# reboot

# service squid stop

# service remove squid 

# mkdir /squid3

# mkdir /etc/apache2

# cd squid3

# wget ... bla bla .. squid.org 

# tar -zxvf squid 3.5.2

# cd squid 3.5.2

# ./bootstrap.sh 

# ./configure options:  '--prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--libexecdir=/usr/lib/squid' '--sysconfdir=/etc/squid' '--localstatedir=/cache' '--libdir=/usr/lib' '--includedir=/usr/include' '--datadir=/usr/share/squid' '--infodir=/usr/share/info' '--mandir=/usr/share/man' '--srcdir=.' '--disable-dependency-tracking' '--disable-strict-error-checking' '--enable-storeio=ufs,aufs,diskd' '--enable-removal-policies=lru,heap' '--disable-ipv6' '--disable-wccp' '--disable-wccpv2' '--enable-kill-parent-hack' '--disable-snmp' '--enable-cachemgr-hostname=squid' '--enable-ssl' '--enable-cache-digests' '--disable-select' '--enable-http-violations' '--enable-linux-netfilter' '--enable-follow-x-forwarded-for' '--disable-ident-lookups' '--enable-ssl-crtd' '--disable-auth-basic' '--enable-x-accelerator-vary' '--enable-zph-qos' '--with-default-user=squid' '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid' '--with-swapdir=/var/spool/squid' '--with-aufs-threads=32' '--with-dl' '--with-large-fi  les' '--with-openssl' '--enable-ltdl-convenience' '--enable-delay-pools' '--with-filedescriptors=65536

# make 

# make install

# squid -v

# mkdir /cache

# chown squid:squid /cache && chmod 755 /cache

# chown squid:squid /var/log/squid

# cd

# mkdir /etc/squid/cert

# cd /etc/squid/cert 

# openssl genrsa 4096 > /etc/squid/cert/private.pem

# openssl req -new -x509 -days 3650 -key /etc/squid/cert/private.pem -out /etc/squid/cert/public.pem

# openssl x509 -in /etc/squid/cert/public.pem -outform DER -out /etc/squid/cert/gobrother.der

# /usr/lib/squid/ssl_crtd -c -s /etc/squid/ssl_db

# chown -R squid:squid /etc/squid/ssl_db

# chown -R nobody /etc/squid/ssl_db/

# touch /var/log/squid/cache.log

# touch /var/log/squid/access.log

# chown -R squid:squid /etc/squid/cert

# chown -R squid:squid /var/log/squid/

# chown -R squid:squid /var/log/squid/cache.log

# chown -R squid:squid /var/log/squid/access.log

# chmod 755 /etc/squid/store-id.pl squid.conf 

# cd /etc/apache2

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

# If you just change the port or add more ports here, you will likely also
# have to change the VirtualHost statement in
# /etc/apache2/sites-enabled/000-default.conf

Listen 3129
Listen 3128

<IfModule ssl_module>
 Listen 443
</IfModule>

<IfModule mod_gnutls.c>
 Listen 443
</IfModule>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

buat file + save dengan nama (apaajaboleh.conf)
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


# chmod 755 /etc/apache2/apaajaboleh.conf

# chown -R squid:squid /etc/apache2/apaajaboleh.conf

# cd /etc/rc.d

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

edit rc.local + save

iptables -t mangle -F
iptables -t mangle -X
iptables -t mangle -N DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT
iptables -t mangle -A INPUT -j ACCEPT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT

iptables -t mangle -A PREROUTING ! -d xx.xx.xx.xx/24 -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3128
iptables -t mangle -A PREROUTING ! -d xx.xx.xx.xx/24 -p tcp --dport 443 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129


/sbin/ip rule add fwmark 1 lookup 100
/sbin/ip route add local 0.0.0.0/0 dev lo table 100

echo 0 > /proc/sys/net/ipv4/conf/lo/rp_filter
echo 1 > /proc/sys/net/ipv4/ip_forward

exit 0

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

# squid -z 

# squid -Nd1 &

# service firewall restart

+++++++ catatan pribadi ++++++++++
tidak untuk diperjual belikan 
resiko ditanggung penumpang 
+++++++++++++++++++++++++++++++++
sumber: https://mikrotiksquid.wordpress.com/2015/04/01/clearos-6-4-squid-3-5-2-gateway-mode-https-caching/
  

Friday, January 16, 2015

CLEAROS 6.5 STANDALONE MODE + SQUID 3.4.2 + TPROXY + HTTPS SSL BUMP

sumber : http://forum.clearos.or.id/index.php?topic=2479.0

Bismillahirrahmanirrahim

EVEN THE BEST CAN BE IMPROVED

1. Hal-hal yang berbau teknis mohon di selesaikan di thread ini, sehingga diketahui oleh semua pembaca setiap kesalahan dan kekurangan dari thread saya
2. Untuk itu demi penyempurnaan thread ini di mohon teman-teman forum COSI yang lain untuk membantu
3. Buat Mengingatkan Di Baca Dulu!
WARNING: HTTPS was designed to give users an expectation of privacy and security. Decrypting HTTPS tunnels without user consent or knowledge may violate ethical norms and may be illegal in your jurisdiction. Squid decryption features described here and elsewhere are designed for deployment with user consent or, at the very least, in environments where decryption without consent is legal. These features also illustrate why users should be careful with trusting HTTPS connections and why the weakest link in the chain of HTTPS protections is rather fragile. Decrypting HTTPS tunnels constitutes a man-in-the-middle attack from the overall network security point of view. Attack tools are an equivalent of an atomic bomb in real world: Make sure you understand what you are doing and that your decision makers have enough information to make wise choices.

GOAL:
1. Menerapkan tproxy pada Server ClearOS Mode Standalone
2. mencache youtube
3. Cache HTTPS
sekali lagi https di katakan tercache jika ada tulisan di access.log
GET https://....
 
TOPOLOGI
                                port ethernet no.3 ------- SQUID
Klien ---- Switch ---- port ethernet no.2
                                port ethernet no.1 ------- Modem


    
Oke langsung saja, saya asumsikan COS sudah berjalan

1. Update CLearOS anda
Kode: [Pilih]
# yum update
# shutdown -r now

2. Install Development Tools
Kode: [Pilih]
# yum-config-manager --enable clearos-core
# yum --enablerepo=clearos-core,clearos-developer,clearos-epel install clearos-devel app-devel

3. Instal Paket-paket yang di butuhkan
Kode: [Pilih]
# yum install openssl openssl-devel fakeroot ebtables libcap libcap-devel automake gcc glibc-devel e2fsprogs-devel sharutils
# yum --enablerepo=* install libtool libtool-ltdl libtool-ltdl-devel perl-File-ReadBackwards ccze

4. Download dan Install Squid-3.4.2
Kode: [Pilih]
# yum install squid
# yum remove squid
# wget https://www.dropbox.com/s/vkfhwk28hy9tul8/squid-3.4.2.tar.gz
# tar -zxvf squid-3.4.2.tar.gz
# cd squid-3.4.2
# ./bootstrap.sh 
# ./configure --prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libexecdir=/usr/lib/squid --sysconfdir=/etc/squid --localstatedir=/cache --libdir=/usr/lib --includedir=/usr/include --datadir=/usr/share/squid --infodir=/usr/share/info --mandir=/usr/share/man --disable-dependency-tracking --enable-storeio=ufs,aufs,diskd --enable-removal-policies=lru,heap --enable-icmp --enable-esi --enable-icap-client --disable-wccp --disable-wccpv2 --enable-kill-parent-hack --enable-cachemgr-hostname=localhost--enable-ssl --enable-cache-digests --enable-linux-netfilter --enable-follow-x-forwarded-for --enable-x-accelerator-vary --enable-zph-qos --with-default-user=squid --with-logdir=/var/log/squid --with-pidfile=/var/run/squid.pid --with-large-files --enable-ltdl-convenience --with-filedescriptors=65536 --enable-ssl --enable-ssl-crtd --enable-ssl-crtd --with-openssl=/usr/lib/openssl  --disable-auth --build=i686-linux-gnu build_alias=i686-linux-gnu
# make && make install
# squid -v
# chown squid:squid /cache && chmod 777 /cache
# chown squid:squid /var/log/squid

5. Hapus squid.conf lama dan download squid.conf dan storeid.pl
Kode: [Pilih]
# rm /etc/squid/squid.conf
# cd /etc/squid
# wget https://www.dropbox.com/s/zy8y5ygnwy3txay/squid.conf
# wget https://www.dropbox.com/s/6x7d86iyogb8qgi/store-id.pl?m=
# chmod 777 /etc/squid/store-id.pl

6. Download file squid untuk /etc/init.d
download filenya di https://www.dropbox.com/s/pqxzuiofh1yqia7/squid.init.d.txt
rubah file name menjadi "squid" tanpa tanda petik kemudian masukkan ke folder /etc/init.d
Beri izin execute squid
Kode: [Pilih]
# chmod +x /etc/init.d/squid

7. Setup SSL Bump Squid 3.4.2
Kode: [Pilih]
# cd /etc/squid
# mkdir ssl_cert
# cd ssl_cert
# openssl req -new -newkey rsa:1024 -days 365 -nodes -x509 -keyout myCA.pem  -out myCA.pem
# openssl x509 -in myCA.pem -outform DER -out myCA.der
# cd
# mkdir /var/squid
# cd /var/squid
# mkdir ssl_db
# cd
# chown -R nobody /var/squid/ssl_db/
# /usr/lib/squid/ssl_crtd -c -s /var/squid/ssl_db/certs
# chown -R squid:squid /var/squid/ssl_db/
# squid -k parse
# squid -z

8. Load modul-modul tproxy dan tambahkan juga di rc.local
Kode: [Pilih]
# modprobe xt_TPROXY
# modprobe xt_socket
# modprobe nf_tproxy_core
# modprobe xt_mark
# modprobe nf_nat
# modprobe nf_conntrack_ipv4
# modprobe nf_conntrack
# modprobe nf_defrag_ipv4
# modprobe ipt_REDIRECT
# modprobe iptable_nat

Cek Hasilnya
Kode: [Pilih]
# dmesg | grep PROXY
NF_TPROXY: Transparent proxy support initialized, version 4.1.0
NF_TPROXY: Copyright (c) 2006-2007 BalaBit IT Ltd.

9. Tambahkan rule TProxy ke /etc/rc.d/rc.local
Kode: [Pilih]
modprobe xt_TPROXY
modprobe xt_socket
modprobe nf_tproxy_core
modprobe xt_mark
modprobe nf_nat
modprobe nf_conntrack_ipv4
modprobe nf_conntrack
modprobe nf_defrag_ipv4
modprobe ipt_REDIRECT
modprobe iptable_nat

iptables -t mangle -F
iptables -t mangle -X

iptables -t mangle -N DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT
iptables -t mangle -A INPUT -j ACCEPT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A PREROUTING ! -d XX.XX.XX.XX/XX -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129
iptables -t mangle -A PREROUTING ! -d XX.XX.XX.XX/XX -p tcp --dport 443 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3127

/sbin/ip rule add fwmark 1 lookup 100
/sbin/ip route add local 0.0.0.0/0 dev lo table 100

echo 0 > /proc/sys/net/ipv4/conf/lo/rp_filter
echo 1 > /proc/sys/net/ipv4/ip_forward
exit 0

10. Disable selinux
Kode: [Pilih]
# vi /etc/selinux/config
Ubahmenjadi SELINUX=disabled

11. Tambah dan edit settingan sysctl.conf
Kode: [Pilih]
# vi /etc/sysctl.conf
net.ipv4.ip_nonlocal_bind=1
net.ipv4.conf.default.forwarding=1
net.ipv4.conf.all.forwarding=1

net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 0

12. Tambahkan baris berikut di rc.local
Kode: [Pilih]
echo 1024 65535 > /proc/sys/net/ipv4/ip_local_port_range
ulimit -HSn 65535
/usr/sbin/squid -Nd1 &

13. Restart firewall dan jalankan squidnya
Kode: [Pilih]
# service firewall restart
# /usr/sbin/squid -Nd1 &

14. Tanamkan myCA.der di Browser

15. Konfigurasi di MikroTik & Mangle dan Routing TPROXY (sesuaikan dengan IP Proxy Anda)
#copy paste command dibawah ini menggunakan menu New Terminal di Winbox#
kasi nama interface dan IP address

Kode: [Pilih]
/interface ethernet
set 2 name=ether3-proxy

/ip address 
add address=XX.XX.XX.XX/XX interface=ether3-proxy

/ip firewall mangle
add action=mark-routing chain=prerouting comment="TPROXY ROUTING" disabled=no dst-port=80,443 in-interface=ether2-local new-routing-mark=tproxy_rm passthrough=no \
    protocol=tcp
add action=mark-connection chain=prerouting disabled=no dst-port=80,443 in-interface=ether3-proxy new-connection-mark=tproxy_cm passthrough=yes protocol=tcp \
    src-address=!XX.XX.XX.XX
add action=mark-routing chain=prerouting connection-mark=tproxy_cm disabled=yes in-interface=!ether3-proxy new-routing-mark=tproxy_rm passthrough=no

/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=XX.XX.XX.XX routing-mark=tproxy_rm scope=30 target-scope=10


Coba browsing jika ada
TCP_MISS/504 5135 GET https://.... berarti anda sudah sukses mencache content HTTPS

NOTE:
- Tutorial ini hasil copas saya dari berbagai tempat di dasarkan dari Forum Clear Indonesia, Forum Ubuntu Indonesia, Grup Squid Mikrotik Indonesia, dan tempat lain yang tidak bisa saya sebutkan 1 per 1
- Terima Kasih yang sebesar-besarnya atas clue dan petunjuknya kepada Om MikroTiker N SquidLover, giens26, Syaifuddin JW, Andi Micro, dan semua pihak yang telah membantu dan tidak dapat saya sebutkan. Semoga Amal Baik Mereka di catat sebagai amal baik di sisi-Nya