pre-pend log timestamp

Adds a timestamp before alog from a script

awk -v sec=$(date -u +%s) '{print strftime("%Y-%m-%d %H:%M:%S",sec), $0; fflush();}'

in cron :

awk -v sec=$(date -u +\%s) '{print strftime("\%Y-\%m-\%d \%H:\%M:\%S",sec), $0; fflush();}'

So :

# echo hello world | awk -v sec=$(date -u +%s) '{print strftime("%Y-%m-%d %H:%M:%S",sec), $0; fflush();}'

 

2014-10-30 08:40:26 test

nJoy 🙂

Invalidate Varnish cache

1)   netstat -plnt

2)   varnishadm -T 127.0.0.1:6082 ban.url /wp-content/themes/catch-box -S /etc/varnish/secret

 

Will invalidate all content under  /wp-content/themes/catch-box.

 

nJoy  😛

 

Fix Raid on LSI controllers when a disk is shown as ubad

This can be done with both megacli and storcli.

Since the way forward is storcli this how to will be based on that.

 

storcli /c0 show all

 

Drive Information :
=================
 
-------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model                  Sp 
-------------------------------------------------------------------------------
8:1       7 Onln   0  465.25 GB SATA HDD N   N  512B WDC WD5003ABYX-01WERA1 U  
8:2       6 Onln   0  465.25 GB SATA HDD N   N  512B WDC WD5003ABYX-01WERA1 U  
8:3       5 Onln   0  465.25 GB SATA HDD N   N  512B WDC WD5003ABYX-01WERA1 U
.
.
8:14      4 UBad   - 223.062 GB SATA SSD N   N  512B INTEL SSDSC2CW240A3    U  
-------------------------------------------------------------------------------

 

Say 8:14 is marked a Ubad. (Unconfigured bad)

storcli /c0 /e8 /s14 set good

storcli /c0 show all

It should now have the disk marked as Ugood ( Unconfigured Good)

-------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model                  Sp 
-------------------------------------------------------------------------------
8:1       7 Onln   0  465.25 GB SATA HDD N   N  512B WDC WD5003ABYX-01WERA1 U  
8:2       6 Onln   0  465.25 GB SATA HDD N   N  512B WDC WD5003ABYX-01WERA1 U  
8:3       5 Onln   0  465.25 GB SATA HDD N   N  512B WDC WD5003ABYX-01WERA1 U
.
.
8:14      4 uGood  F 223.062 GB SATA SSD N   N  512B INTEL SSDSC2CW240A3    U  
-------------------------------------------------------------------------------

storcli /c0 -/fall show

 

Controller = 0
Status = Success
Description = Operation on foreign configuration Succeeded
 
 
FOREIGN CONFIGURATION :
=====================
 
----------------------------------------
DG EID:Slot Type State       Size NoVDs 
----------------------------------------
 F -        Cac0 Frgn  223.062 GB     1 
----------------------------------------
 
NoVDs - Number of VDs in disk group|DG - Diskgroup
Total foreign drive groups = 1

 

This would show an external configuration which needs clearing.

Now we need to clear the config data from the previous RAID conf cached on the disk.

 

storcli /cx /fall import

 

This will clear the foreign config and start the rebuild.

 

To follow the rebuild :

 

storcli /c0 /e8 /sall show rebuild 
Controller = 0
Status = Success
Description = Show Drive Rebuild Status Succeeded.
———————————————————
Drive-ID Progress% Status Estimated Time Left
———————————————————
/c0/e8/s0 – Not in progress –
/c0/e8/s1 – Not in progress –
/c0/e8/s2 – Not in progress –
/c0/e8/s3 – Not in progress –
/c0/e8/s4 – Not in progress –
/c0/e8/s5 – Not in progress –
/c0/e8/s6 – Not in progress –
/c0/e8/s7 – Not in progress –
/c0/e8/s8 – Not in progress –
/c0/e8/s9 – Not in progress –
/c0/e8/s10 – Not in progress –
/c0/e8/s11 – Not in progress –
/c0/e8/s12 – Not in progress –
/c0/e8/s13 – Not in progress –
/c0/e8/s14 93 In progress –
/c0/e8/s15 – Not in progress –
———————————————————

 

Or for one drive in particular:

 

storcli /c0 /e8 /s14 show rebuild

Controller = 0
Status = Success
Description = Show Drive Rebuild Status Succeeded.
———————————————————————————————————-
Drive-ID          Progress% Status            Estimated Time Left
———————————————————————————————————-
/c0/e8/s14       93               In progress –
———————————————————————————————————-

 

That’s it.

 

[ NOTE : ]   Rebuild might take more than 1 day in default mode…. Just saying be patient.

 

nJoy 😉

fstab entry for sshfs

Sample entry

root@192.168.168.100:/mnt/streamstorage53/stage.k/ura/web /opt/kaltura/web fuse.sshfs _netdev,auto 0 0

 

nJoy 😉

 

Decoding parameters in Bash script (1 space delimited)

This is a small script that takes a bash command and decodes the parameters in non sequential order except definers.

#!/bin/bash

if [ "$#" -ne 1 ] || ! [ -d "$1" ]; then
  echo "Usage: $0 " >&2
  exit 1
fi




while [[ $# > 1 ]]
do
  key=”$1″
  echo $key
  shift
  case $key in
   -c|–color|color)
     COLOR=$1
     shift
     ;;
   -s|–section|section)
     SECTION=”yes”
     ;;
    *)

;;
esac
done

nJoy 😉

Enable X11 Forwarding on Centos/Redhat

1) Install the following:
xorg-x11-xauth
xorg-x11-fonts-*
xorg-x11-utils
2) Enable the following in the sshd_config file
X11Forwarding yes
3) Use an appropriate X-Server on your desktop

nJoy 😉

Rate Limiting in iptables

iptables -A INPUT -p tcp –syn –dport 80 -m connlimit –connlimit-above 15 –connlimit-mask 32 -j REJECT –reject-with tcp-reset

Allowing Apache to see client ip from Behind a trusted proxy.

With high speed caching based on nginx, varnish or CDNs in general the client ip gets lost. All
IPs get reported as 127.0.0.1 since the proxy is making the socket request. Solution is two phased:

1) enable reporting the client IP to the X-Forwarded-For header at the proxy or CDN.

This depends on the proxy will be covered in separate posts.

2) installing and configuring the mod_remoteip in Apache 2.

 

Project link L https://github.com/ttkzw/mod_remoteip-httpd22
mkdir /usr/local/src/mod_remoteip
cd /usr/local/src/mod_remoteip
wget https://raw.githubusercontent.com/ttkzw/mod_remoteip-httpd22/master/mod_remoteip.c
wget https://raw.githubusercontent.com/ttkzw/mod_remoteip-httpd22/master/mod_remoteip.conf
wget https://raw.githubusercontent.com/ttkzw/mod_remoteip-httpd22/master/Makefile

yum install httpd-devel
make
make install

Configuration in the Apache config file:

in /etc/httpd/conf/httpd.conf

# Load and configure mod_remoteip for Google PageSpeed Service
LoadModule remoteip_module /usr/lib64/httpd/modules/mod_remoteip.so
RemoteIPHeader X-Forwarded-For

service apache reload

voila 🙂

How to test varnish vcl file

Unlike most other configuration systems varnish went with a compiled configuration so if there is a mistake all you get is :

 

Starting varnish HTTP accelerator: [FAILED]

To check what the problem is use the following :

varnishd -C -f default.vcl

varnishd -C -f default.vcl
Message from VCC-compiler:
Expected return action name.
(input Line 37 Pos 13)
return (hit_for_pass);
————############–
Running VCC-compiler failed, exit 1[root@MyHost1 varnish]# vim default.vcl
[root@MyHost1 varnish]# varnishd -C -f default.vcl
Message from VCC-compiler:
Invalid condition ‘&’ on numeric variable
only ‘==’, ‘!=’, ‘<‘, ‘>’, ‘<=’ and ‘>=’ are legal
(input Line 51 Pos 17)
if (obj.hits &gt; 0) {
—————-#——–

Rightly so the error shouts back at you with a # under it.

 

Very clear and to the point, the vcl I had was HTML’ed and replaced > with &gt;

 

That’s it..

nJoy 😉