AWS permissions on buckets for users and roles

Posted on by admin

Full access for specific IAM user/role

Type: bucket

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Action":[
            "s3:*"
         ],
         "Effect":"Allow",
         "Principal":{
            "AWS":[
               "arn:aws:iam::ACCOUNT_ID:user/USERNAME_A",
               "arn:aws:iam::ACCOUNT_ID:user/USERNAME_B",
               "arn:aws:iam::ACCOUNT_ID:user/USERNAME_C",
               "arn:aws:iam::ACCOUNT_ID:role/ROLE_A",
               "arn:aws:iam::ACCOUNT_ID:role/ROLE_B",
               "arn:aws:iam::ACCOUNT_ID:role/ROLE_C"
            ]
         },
         "Resource":[
            "arn:aws:s3:::BUCKET_NAME",
            "arn:aws:s3:::BUCKET_NAME/*"
         ]
      }
   ]
}

Install node.js on Centos 6.8

Posted on by admin

Epel’s yum install does not cut it anymore at time of writing installing node.js causes a mass of 404s and file not found.

 

current best approach :

Enterprise Linux and Fedora

Including Red Hat® Enterprise Linux® / RHEL, CentOS and Fedora.

Node.js is available from the NodeSource Enterprise Linux and Fedora binary distributions repository. Support for this repository, along with its scripts, can be found on GitHub at nodesource/distributions.

Note that the Node.js packages for EL 5 (RHEL5 and CentOS 5) depend on the EPEL repository being available. The setup script will check and provide instructions if it is not installed.

Run as root on RHEL, CentOS or Fedora, for Node.js v4 LTS Argon:

curl --silent --location https://rpm.nodesource.com/setup_4.x | bash -

Alternatively for Node.js v5:

curl --silent --location https://rpm.nodesource.com/setup_5.x | bash -

Alternatively for Node.js 0.10:

curl --silent --location https://rpm.nodesource.com/setup | bash -

Then install, as root:

yum -y install nodejs

Optional: install build tools

To compile and install native addons from npm you may also need to install build tools:

yum install gcc-c++ make
# or: yum groupinstall 'Development Tools'

Available architectures:

  • i386 (32-bit, not available for EL7)
  • x86_64 (64-bit)

Supported Red Hat® Enterprise Linux® versions:

  • RHEL 5 (32-bit and 64-bit)
  • RHEL 6 (32-bit and 64-bit)
  • RHEL 7 (64-bit)

Supported CentOS versions:

  • CentOS 5 (32-bit and 64-bit)
  • CentOS 6 (32-bit and 64-bit)
  • CentOS 7 (64-bit)

Supported CloudLinux versions:

  • CloudLinux 6 (32-bit and 64-bit)

Supported Fedora versions:

  • Fedora 21 (Twenty One) (32-bit and 64-bit)
  • Fedora 20 (Heisenbug) (32-bit and 64-bit)
  • Fedora 19 (Schrödinger’s Cat) (32-bit and 64-bit)

Other distributions known to be supported:

  • Oracle Linux (mirrors RHEL very closely)
  • Amazon Linux (tested on 2014.03)

nJoy 😉

 

Enable Gzip Compression in Apache

Posted on by admin

This is a simple sample configuration for the impatient.

Compress only a few types

AddOutputFilterByType DEFLATE text/html text/plain text/xml

The following configuration, while resulting in more compressed content, is also much more complicated. Do not use this unless you fully understand all the configuration details.

Compress everything except images

<Location />
# Insert filter
SetOutputFilter DEFLATE

# Netscape 4.x has some problems...
BrowserMatch ^Mozilla/4 gzip-only-text/html

# Netscape 4.06-4.08 have some more problems
BrowserMatch ^Mozilla/4\.0[678] no-gzip

# MSIE masquerades as Netscape, but it is fine
# BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

# NOTE: Due to a bug in mod_setenvif up to Apache 2.0.48
# the above regex won't work. You can use the following
# workaround to get the desired effect:
BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html

# Don't compress images
SetEnvIfNoCase Request_URI \
\.(?:gif|jpe?g|png)$ no-gzip dont-vary

# Make sure proxies don't deliver the wrong content
Header append Vary User-Agent env=!dont-vary
</Location>

 

 

 

ref: Apache Doc

nJoy 😉

 

Use ssh keys to encrypt and decrypt messages

Posted on by admin

Encryption with RSA Key Pairs

During the Thanksgiving holiday I wondered, “how hard would it be to encrypt and decrypt files with my SSH key?” Encryption is the purpose of public/private RSA key pairs, after all.

With `openssl`, it’s not too hard.

(Note: If you’re on OSX, you should install the latest versions of OpenSSL and OpenSSH with Homebrew.)

First, let’s start with our plaintext file:

echo "Yo test!!" > clear.txt

Before we can encrypt the text file with our public key, we must export our public key into a PEM format suitable for OpenSSL.

openssl rsa -in ~/.ssh/id_rsa -pubout > ~/.ssh/id_rsa.pub.pem
 
cat ~/.ssh/id_rsa.pub.pem

It should look something like this:

-----BEGIN PUBLIC KEY-----
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-----END PUBLIC KEY-----

Encrypt

cat plain.txt | openssl rsautl -encrypt -pubin -inkey ~/.ssh/id_rsa.pub.pem > encrypted.txt

The important command in the pipeline is `openssl`. The first argument passed to `openssl` is the OpenSSL command you are running. It has a wide variety of commands covering a wide range of cryptographic functionality. For our purposes, we’re doing public/private RSA encryption, so we’re using the RSA Utility, or `rsautl`, command. Next, the `-encrypt` key indicates we are encrypting from plaintext to cipher text, and finally the `-pubin` flag indicates we are loading a public key from `-inkey [public key file]`.

Print the contents of the encrypted with `cat encrypted.txt`.

You should see non readable stuff.

Decrypt

cat encrypted.txt | openssl rsautl -decrypt -inkey ~/.ssh/id_rsa

“Yo test!!”

Voila`! We’re back to clear text.

 

Gather metatdata on AWS from within EC2 instance.

Posted on by admin

So I need to get information about a particular set of machines in the cloud (AWS).

EC2_INSTANCE_ID="`wget -q -O - http://instance-data/latest/meta-data/instance-id || die \"wget instance-id has failed: $?\"`"
test -n "$EC2_INSTANCE_ID" || die 'cannot obtain instance-id'
EC2_AVAIL_ZONE="`wget -q -O - http://instance-data/latest/meta-data/placement/availability-zone || die \"wget availability-zone has failed: $?\"`"
test -n "$EC2_AVAIL_ZONE" || die 'cannot obtain availability-zone'
EC2_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'`"

 

nJoy 😉

 

 

Unexpected Fail over warning on Couchbase servers

Posted on by admin

This is the “expected” behavior. Let me explain it, with a cluster of 3 nodes and 1 replica.

So you have started with 1 node, so in this case you have only “active documents” (no replica)

Then you add another node, and do a rebalance. Once it is done you have 50% of the active data on each node, and 50% of the replica on each node.

Let’s add a new node again, just to have a more “realistic” cluster of 3 nodes. So the node is added and cluster is rebalanced. This means now you have, as you can guess 33.33% on each node (Active and Replica)

So what you have notice is that the Rebalance is an expensive operation, since the cluster has to move data between all the nodes. (moving active and replicas).

You have a now a well balanced 3 nodes cluster.

Now you stop one node, or one node crashes… this means that some of the data are not accessible (they are still here not available, you do not lose anything).

Here you have 2 options:
– if you restart the server, nothing to do the cluster is back online entirely. (3 nodes cluster well balances)

you do a failover on the node that is off. Let’s explain this in detail.
Failover:
So what is happening here: Couchbase will do that as fast as possible to be sure all the data are available(read and write). So the only thing that is happening here is: promote the replicas to active (for the keys that were active on the node that is off now)

So what is the status now?
– all the data are accessible in read/write for the application on 2 nodes, so you have 50% of the active data on each node.
– BUT you do not have all the replicas since:
– the replicas that are on the node that is off are “not present”
– the replicas that have been promoted are not present anymore

This is why you see the message “Fail Over Warning: Rebalance required, some data is not currently replicated!” in your console.

Does it make sense?

So to be able to get back in a status that is “balanced” you need to do a rebalance.

Note: when you failover of node, this node is removed from the cluster, and to add it back you need to add it, and rebalanced. (the data that are on this server are just “ignored”)

Hope this clarify the message.

Some pointers about this:
– http://docs.couchbase.com/couchbase-manual-2.2/#couchbase-admin-tasks-failover5
– http://docs.couchbase.com/couchbase-manual-2.2/#couchbase-admin-tasks-failover-addback4