Sending Elasticsearch to a syslog server

yum install rsyslog -y

Add the following to rsyslog.conf on the client system

############

$ModLoad imfile
$InputFileName /var/log/elasticsearch/elasticsearch.log
$InputFileTag elasticsearch
$InputFileStateFile stat-elasticsearch
$InputFileSeverity Info
$InputFileFacility daemon
$InputRunFileMonitor
#local3.* hostname:<portnumber>

daemon.* @192.168.1.66:514


############

 

Also if you want all logs to go through to syslog server:

 

*.* @192.168.1.66

 

at the end of the file.

Issue a :

service rsyslog restart

and watch the logs flow in.

 

nJoy ๐Ÿ˜‰

 

 

 

 

fixing permission denied issue with udp 512 port graylog

use iptables to pre-route NAT the udp port :

 

iptables -A PREROUTING -t nat -i eth0 -p udp –dport 514 -j REDIRECT –to-port 10515

 

This will bypass the limit in the OS to ports < 1024 to non=root users .

nJoy ๐Ÿ˜‰

 

Installing sample data in elastic search

After installing elastic search it is useful for testing and training to load some sample data.

1) create mapping :

curl -XPUT http://localhost:9200/shakespeare -d '
{
 "mappings" : {
  "_default_" : {
   "properties" : {
    "speaker" : {"type": "string", "index" : "not_analyzed" },
    "play_name" : {"type": "string", "index" : "not_analyzed" },
    "line_id" : { "type" : "integer" },
    "speech_number" : { "type" : "integer" }
   }
  }
 }
}
';

2) Load the data using the bulk api:

wget "https://github.com/ropensci/elastic_data/blob/master/data/shakespeare_data.json?raw=true" -O  shakespeare.json

curl -XPUT localhost:9200/_bulk --data-binary @shakespeare.json

 

nJoy ๐Ÿ˜‰

 

 

 

Preferred Tools on Linux

Might need Epel or other repo

For CentOS 6 :

yum install epel-release -y
yum install net-tools man screen htop vim-enhanced wget iotop iftop sysstat usbutils nfs-utils bind-utils ntp tcpdump telnet tree mlocate zip unzip pciutils git -y

For Ubuntu

 apt-get install -y glances language-pack-en net-tools man screen htop vim wget iotop iftop sysstat usbutils  ntp tcpdump telnet tree mlocate zip unzip pciutils git npm sysv-rc-conf

nJoy ๐Ÿ˜‰

Pretty Json in Bash

Simply pipe through

python -m json.tool

e.g.

 

curl -u Administrator:kpasswordย http://172.16.3.136:8091//pools/default | python -m json.tool

 

Restoring a bucket in Couchbase

[root@mofo bin]# ./cbrestore /tmp/cbbackup/2015-01-19T220530Z/2015-01-19T220530Z-full/ -u Administrator -ppassword http://mofo.lan:8091 –bucket-source=beer-sample –bucket-destination=DAVID
[####################] 100.0% (7303/estimated 7303 msgs)
bucket: beer-sample, msgs transferred…
: total | last | per sec
byte : 2541549 | 2541549 | 3415000.2
done
[root@mofo bin]#