Author: David Saliba

Bash Datestamps

Posted on by David Saliba 
        Format/result         |       Command              |          Output
------------------------------+----------------------------+------------------------------
YY-MM-DD_hh:mm:ss             | date +%F_%T                | $(date +%F_%T)
YYMMDD_hhmmss                 | date +%Y%m%d_%H%M%S        | $(date +%Y%m%d_%H%M%S)
YYMMDD_hhmmss (UTC version)   | date --utc +%Y%m%d_%H%M%SZ | $(date --utc +%Y%m%d_%H%M%SZ)
YYMMDD_hhmmss (with local TZ) | date +%Y%m%d_%H%M%S%Z      | $(date +%Y%m%d_%H%M%S%Z)
YYMMDDhhmmss                  | date +%Y%m%d%H%M%S         | $(date +%Y%m%d%H%M%S)
YYMMDDhhmmssnnnnnnnnn         | date +%Y%m%d%H%M%S%N       | $(date +%Y%m%d%H%M%S%N)
Seconds since UNIX epoch:     | date +%s                   | $(date +%s)
Nanoseconds only:             | date +%N                   | $(date +%N)
Nanoseconds since UNIX epoch: | date +%s%N                 | $(date +%s%N)
ISO8601 UTC timestamp         | date --utc +%FT%TZ         | $(date --utc +%FT%TZ)
ISO8601 Local TZ timestamp    | date +%FT%T%Z              | $(date +%FT%T%Z)


njoy ;-)

Mounting an LVM file system

Posted on by David Saliba

So after recovering a box which had a controller failure we needed to mount the old fs to recover some stuff.

when trying to mount /dev/sdc2 we got:

mount: unknown filesystem type ‘LVM2_member’

fix is easy :

modprobe dm-mod

vgchange -ay

lvscan now yields:

ACTIVE '/dev/VolGroup/lv_root' [50.00 GiB] inherit
ACTIVE '/dev/VolGroup/lv_home' [178.46 GiB] inherit
ACTIVE '/dev/VolGroup/lv_swap' [3.94 GiB] inherit

mount /dev/VolGroup/lv_root /mnt/tempdisk/

et viola` !!

nJoy 😉

 

 

 

Backup and Restore Elastic search

Posted on by David Saliba

While elastic search is usually run as a cluster, for the sake of this tutorial I am showing the _snapshot and _restore tools.

 

mkdir  /mnt/backups/my_backup
chmod 777 -R /mnt/backups/

Must available on all nodes.


 curl -XPUT 'http://localhost:9200/_snapshot/my_backup' -d '{
 "type": "fs",
 "settings": {
        "location": "/mnt/backups/my_backup",
   "compress": true
    }
}'






[root@centos-base mnt]# curl -XGET 'http://localhost:9200/_snapshot/my_backup?pretty'

{
  "my_backup" : {
    "type" : "fs",
    "settings" : {
      "compress" : "true",
      "location" : "/mnt/backups/my_backup"
    }
  }
}



 curl -XGET 'http://localhost:9200/_snapshot?pretty'                                       {
  "my_backup" : {
    "type" : "fs",
    "settings" : {
      "compress" : "true",
      "location" : "/mnt/backups/my_backup"
    }
  }
}

_____________________________________________________________________________________________

changing


 curl -XPUT 'http://localhost:9200/_snapshot/my_backup' -d '{
 "type": "fs",
 "settings": {
  "location": "/mnt/backups/my_backup",
 "compress": true,
 "verify":true
	}	
	}'


 curl -XPUT "localhost:9200/_snapshot/my_backup/snapshot_`date | tr -d " " | tr -d ":" | tr '[:upper:]' '[:lower:]' `?wait_for_completion=true&pretty"
 
 

 
_____________________________________________________________________________________________

restoring

 mkdir -p /mnt/backups/my_backup
 chmod -R 777 /mnt/backups/

 
 Create repository
 -----------------------
 
 curl -XPUT 'http://localhost:9200/_snapshot/my_backup' -d '{
 "type": "fs",
 "settings": {
  "location": "/mnt/backups/my_backup",
 "compress": true,
 "verify":true
	}	 
	}'

	
 restore from file system
 --------------------------------
 
  curl -XPOST "localhost:9200/_snapshot/my_backup/snapshot_satapr25223454cest2015/_restore"

nJoy 😉

Sending Elasticsearch to a syslog server

Posted on by David Saliba

yum install rsyslog -y

Add the following to rsyslog.conf on the client system

############

$ModLoad imfile
$InputFileName /var/log/elasticsearch/elasticsearch.log
$InputFileTag elasticsearch
$InputFileStateFile stat-elasticsearch
$InputFileSeverity Info
$InputFileFacility daemon
$InputRunFileMonitor
#local3.* hostname:<portnumber>

daemon.* @192.168.1.66:514


############

 

Also if you want all logs to go through to syslog server:

 

*.* @192.168.1.66

 

at the end of the file.

Issue a :

service rsyslog restart

and watch the logs flow in.

 

nJoy 😉

 

 

 

 

Installing sample data in elastic search

Posted on by David Saliba

After installing elastic search it is useful for testing and training to load some sample data.

1) create mapping :

curl -XPUT http://localhost:9200/shakespeare -d '
{
 "mappings" : {
  "_default_" : {
   "properties" : {
    "speaker" : {"type": "string", "index" : "not_analyzed" },
    "play_name" : {"type": "string", "index" : "not_analyzed" },
    "line_id" : { "type" : "integer" },
    "speech_number" : { "type" : "integer" }
   }
  }
 }
}
';

2) Load the data using the bulk api:

wget "https://github.com/ropensci/elastic_data/blob/master/data/shakespeare_data.json?raw=true" -O  shakespeare.json

curl -XPUT localhost:9200/_bulk --data-binary @shakespeare.json

 

nJoy 😉

 

 

 

Preferred Tools on Linux

Posted on by David Saliba

Might need Epel or other repo

For CentOS 6 :

yum install epel-release -y
yum install net-tools man screen htop vim-enhanced wget iotop iftop sysstat usbutils nfs-utils bind-utils ntp tcpdump telnet tree mlocate zip unzip pciutils git -y

For Ubuntu

 apt-get install -y glances language-pack-en net-tools man screen htop vim wget iotop iftop sysstat usbutils  ntp tcpdump telnet tree mlocate zip unzip pciutils git npm sysv-rc-conf

nJoy 😉