Category: Uncategorized

Installing sample data in elastic search

Posted on by admin

After installing elastic search it is useful for testing and training to load some sample data.

1) create mapping :

curl -XPUT http://localhost:9200/shakespeare -d '
{
 "mappings" : {
  "_default_" : {
   "properties" : {
    "speaker" : {"type": "string", "index" : "not_analyzed" },
    "play_name" : {"type": "string", "index" : "not_analyzed" },
    "line_id" : { "type" : "integer" },
    "speech_number" : { "type" : "integer" }
   }
  }
 }
}
';

2) Load the data using the bulk api:

wget "https://github.com/ropensci/elastic_data/blob/master/data/shakespeare_data.json?raw=true" -O  shakespeare.json

curl -XPUT localhost:9200/_bulk --data-binary @shakespeare.json

 

nJoy 😉

 

 

 

Preferred Tools on Linux

Posted on by admin

Might need Epel or other repo

For CentOS 6 :

yum install epel-release -y
yum install net-tools man screen htop vim-enhanced wget iotop iftop sysstat usbutils nfs-utils bind-utils ntp tcpdump telnet tree mlocate zip unzip pciutils git -y

For Ubuntu

 apt-get install -y glances language-pack-en net-tools man screen htop vim wget iotop iftop sysstat usbutils  ntp tcpdump telnet tree mlocate zip unzip pciutils git npm sysv-rc-conf

nJoy 😉

To cleanup a brick previously used under glusterfs

Posted on by admin

This must be done after you REALLY know this brick is going to be re-used elsewhere not in the same volume it was used before.
NOTE: Potential data loss 

setfattr -x trusted.glusterfs.volume-id $brick_path
setfattr -x trusted.gfid $brick_path
rm -rf $brick_path/.glusterfs

Playing with Glusterfs

Posted on by admin

More details in a later post but I finally realized the order of things ? Thought I’d share.

 

Reference sites:
http://www.gluster.org/community/documentation/index.php/Getting_started_configure
http://www.redhat.com/magazine/009jul05/features/gfs_practices/

mkdir ~/gluster
cd ~/gluster

wget -l 1 -nd -nc -r -A.rpm http://download.gluster.org/pub/gluster/glusterfs/LATEST/RHEL/epel-6/x86_64/

 

yum install glusterfs-libs-*.el6.x86_64.rpm -y
yum install glusterfs-*.el6.x86_64.rpm -y
yum install glusterfs-fuse-*.el6.x86_64.rpm -y
yum install glusterfs-cli-*.el6.x86_64.rpm -y
yum install glusterfs-server-*.el6.x86_64.rpm -y
yum install glusterfs-geo-replication-*.el6.x86_64.rpm -y

service glusterd start
chkconfig glusterd on

service glusterfsd start
chkconfig glusterfsd on

gluster peer probe <hostname of the other server in the cluster, or IP address if you don’t have DNS or /etc/hosts entries>


dd if=/dev/zero of=~/test.bin count=10000k
losetup /dev/loop0 test.bin

fdisk /dev/loop0 ---> new partition all blocks fdisk --> n -> p -> 1 -> from 1 to 637 in this case

yum install xfsprogs xfsdump -y

mkfs.xfs -i size=512 /dev/loop0 -f

--- Testing fs --------
mkdir /mnt/test
mount /dev/loop0 /mnt/test



--------fstab -----

/root/test.bin /mnt/test xfs loop 0 0

>>>>> test rebooting


mkdir -p /mnt/test/brick

gluster volume create gv0 replica 2 192.168.1.81:/mnt/test/brick 192.168.1.79:/mnt/test/brick

gluster volume info

Volume Name: gv0
Type: Replicate
Volume ID: cb3110c8-82b0-45f5-9e38-98652a95b54b
Status: Created
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: 192.168.1.81:/mnt/test/brick
Brick2: 192.168.1.79:/mnt/test/brick 




gluster volume start gv0

Just the client

  yum install glusterfs-libs-3.5.0-2.el6.x86_64.rpm -y
  yum install glusterfs-3.5.0-2.el6.x86_64.rpm -y
  yum install glusterfs-fuse-3.5.0-2.el6.x86_64.rpm -y

nJoy 😉

Replacing a piece of XML with awk

Posted on by admin

This script searches for an initial tag and and closing one and replaces the content.

# !/bin/bash
awk ‘
BEGIN {pr = 1;one = 0}
/<Name>OPENING<\/Name>/ {pr = 0;}
{ if (pr) print }
{ if (!pr && !one) {print (“\t\t <Name>OPENING</Name> \n \t\t\t <Value>false</Value> \n \t\t<Type>STOPHERE</Type> \n ” ); one =1 ;}}
/<Type>STOPHERE<\/Type>/ {pr = 1;}
‘< $1

Not the most elegant solution but lends itself to multiple replacements.

 

find . | grep file.xml | xargs -I {J} -n 1 bash -c ‘ ./aw.sh {J} > {J}.new ; mv -f {J}.new {J}  ‘

nJoy 😉

 

 

 

Track ftp password from tcpdump Linux

Posted on by admin

Very nice and simple:

tcpdump port ftp -l -A | egrep -i ‘pass=|pwd=|log=|login=|user=|username=|pw=|passw=|passwd=|password=|pass:|user:|username:|password:|login:|pass |user ‘ –color=auto –line-buffered -B20

nJoy  😉