Category: Howtos

Script to verify that a path is synchronized across multiple machines via ssh

Posted on by David Saliba

This is a script I wrote for work to look through a number of remote servers via ssh (shared keys or include a .pem (id_rsa) file to compare a paths and all it’s subfolders.

The result is a report of which files are out of synch and if duplicates found they are listed separately.

#!/bin/bash
## Parameters  sync_check [-i id.pem] [-u <user for ssh>] -p <path_to_diff>  <target-system-IP1> <target-system-IP2> ...

# David Saliba [dx@sudoall.com]

RED='\033[0;31m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color


PEMFILE=""
SSH_USER="root"

function usage {
	echo 
	echo -e "\t$0 usage as follows :"
	echo -e "\t$0 [-i <pem_file>] [-u <ssh_user_name>] -p <path_to_diff> <ref-system-IPADDR1> <target-system-IP1> <target-system-IP2>..."
	echo 
}


[ $# -le 2 ] && echo && echo -e "${RED}Incorrect number of parameters ${NC}" && usage && exit 128


while getopts "i:u:p:" opt; do
echo $opt $OPTARG
  case $opt in
  	i ) PEMFILE=$OPTARG; shift $((OPTIND-1)); OPTIND=1 ; 
          if [[ -f "$PEMFILE" ]]; then 
              PEMFILE="-i ""$PEMFILE" 
          else
          	  echo "Pemfile : $PEMFILE not found. Quitting .. " 
          	  exit 1  
          fi
          ;;
	u ) SSH_USER=$OPTARG ; shift $((OPTIND-1)); OPTIND=1 ;;
	p ) PATH_TO_CHECK=$OPTARG ; shift $((OPTIND-1)); OPTIND=1 ;;
    * ) usage
       exit 128
  esac
done



extfile="_target.md5.sync_check"

		echo; echo

rm -f /tmp/*.sync_check
for i in $* ; do
	ipcalc -cs $i
	if [ $? == 0 ] ; then
		echo -n "Retrieving Signature List from ..  :[$i] : " 
		ssh $PEMFILE $SSH_USER@$i "hostname"

		COMMAND="ssh $PEMFILE $SSH_USER@$i \"find  $PATH_TO_CHECK -type f -exec md5sum {} \\; | sed -e 's/^/$i\t/'| sort -k 2 \""
		#echo "Running : $COMMAND  /tmp/$i""_target.md5.sync_check"
		bash -c "$COMMAND" > /tmp/$i$extfile
		
	else
		echo "Parameter $i not a valid IP skipping .."
	fi
done

echo ;echo; echo "Comparing data ... "; echo

 OLDIFS=$IFS; 
 IFS=$'\n'; 

 for i in `cat /tmp/*.sync_check  | sort -k3 | uniq -f 1 -c | sort | egrep -v "^[\ \t]*$#" | sort -k4 ` ;do 
 	
 	NUMBER_OF_HITS=`echo "$i" | awk '{print ($1)}'`

 	echo $i

 	if [ $NUMBER_OF_HITS -gt 1 ] ; then
 		
 		MD5Onward=`echo "$i" | awk '{print substr($0, index($0,$3))}'`
		echo -e -n "${YELLOW}"	
			grep -h "$MD5Onward" /tmp/*.sync_check  | awk '{print "\t\t",NR, $0}'
		echo -e -n "${NC}"	


 	fi


 done ;

 IFS=$OLDIFS


echo
echo

link to file here: http://jist.sudoall.com/sync_check/sync_check.sh

nJoy 😉

Reviving dead Centos 4 repos

Posted on by David Saliba

Edit
/etc/yum.repos.d/CentOS-Base.repo

Comment out the “mirrorlist” entries.
Uncomment the “baseurl” entries and change them to

http://vault.centos.org/4.9/os/$basearch

nJoy 😉

start docker with customer system name and hostname

Posted on by David Saliba

docker run --name david -h david-01 -it david/c6nodepm2yo /bin/bash

nJoy 😉

shows :

docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f4da454e156e david/c6nodepm2yo "/bin/bash" 12 seconds ago Exited (0) 3 seconds ago david
d5ec5101ba52 david/centos6:node "/bin/bash" 27 hours ago Up 26 hours 0.0.0.0:8000->80/tcp jovial_borg

and when connected :

[root@david-01 /]#

Centos 7 Firewall open a port

Posted on by David Saliba

Use this command to find your active zone(s):

firewall-cmd --get-active-zones

It will say either public, dmz, or something else. You should only apply to the zones required.

In the case of dmz try:

firewall-cmd --zone=dmz --add-port=2888/tcp --permanent

Otherwise, substitute dmz for your zone, for example, if your zone is public:

firewall-cmd --zone=public --add-port=2888/tcp --permanent

Then remember to reload the firewall for changes to take effect.

firewall-cmd --reload

Docker log files

Posted on by David Saliba

Ubuntu – /var/log/upstart/docker.log
Boot2Docker – /var/log/docker.log
Debian, GNU/Linux – /var/log/daemon.log
CentOS6 – /var/log/daemon.log | grep docker
Fedora – journalctl -u docker.service
OpenSuSE – journalctl -u docker.service

on Centos 7
journalctl -u docker.service

nJoy 😉

Docker Remote API on CentOS

Posted on by David Saliba

After installing Docker on CentOS we need to Docker remote API port on CentOS.

$ cat /etc/sysconfig/docker
other_args=""

Edit the file /etc/sysconfig/docker as below.

other_args="-H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock"

After that, restart docker and try to access the host from another host.

$ sudo /etc/init.d/docker restart
...
$ curl $hostname:4243/images/json
...

For a systemd based Distro like Centos7
Linux with systemd (Ubuntu 15.04, Debian 8,…)

Using systemd, we’ll need to enable a systemd socket to access the Docker remote API:

Create a new systemd config file called /etc/systemd/system/docker-tcp.socket to make docker available on a TCP socket on port 2375.

[Unit]
Description=Docker HTTP Socket for the API

[Socket]
ListenStream=2375
BindIPv6Only=both
Service=docker.service

[Install]
WantedBy=sockets.target
Register the new systemd http socket and restart docker
systemctl enable docker-tcp.socket
systemctl stop docker
systemctl start docker-tcp.socket

Open your browser and verify you can connect to http://localhost:2375/_ping

(more…)

Save SSL keys for AWS Cloudfront

Posted on by David Saliba

Assuming you have AWS cli installed :

aws iam upload-server-certificate --server-certificate-name www.site.tld-cf --certificate-body file://public.key --private-key file://private.key --certificate-chain file://bundle.pem --path /cloudfront/

nJoy 😉

 

hex2bin in node.js

Posted on by David Saliba

Basically it’s all over-engineered and does not work well.

responses are out of alignment and though text-wise they are the same bit wise everything is all over the place :

curl http://phpimpl.domain.com/testhex.php | xxd

00000000: de56 a735 4739 c01d f2dc e14b ba30 8af0 .Q.%G9.....;.0..

curl http://nodejs.domain.com/ | xxd

00000000: c39e 56c2 a725 4739 c380 c3ad c3b1 c39c ..Q..%G9........
 00000010: c3a1 37c2 6b30 c28f c3b0 ..;..0....

The proper way to implement this in node is :

function hex2bin(hex){
return new Buffer(hex,”hex”);
}

curl http://nodejs.domain.com/ | xxd

00000000: de56 a735 4739 c01d f2dc e14b ba30 8af0 .Q.%G9…..;.0..

nJoy 😉