Run Elasticsearch Cluster on Spot Instances


Running Elasticsearch cluster is not cheap, as it generally requires a couple of high spec servers to provides both performances and reliabilities. Below is a typical 3 nodes Elasticsearch cluster that is running on AWS: 3 nodes, one in each availability zone. each node is both master and data node, and it is formed by ec2 … Continue reading Run Elasticsearch Cluster on Spot Instances

Hourly Intermittent Elastic Beanstalk Service Interruption


We noticed that one of our Java applications that is deployed on Elastic Beanstalk has hourly service interruption. Each outage only lasts for about 40 - 60 seconds. By looking at the CloudWatch, the ELB has intermittent 5xx errors every one hour, like 09:01, 10:01, 11:01 ... After some investigations, I found it out that it … Continue reading Hourly Intermittent Elastic Beanstalk Service Interruption

Double SSH Hops example


Client -ssh only--> Jumpbox00 -ssh only--> Jumpbox01 --http only-> Internal network Here is how to ssh to jumpbox01, and visit websites in internal networks from Client. 1) Ensure you have a private key that is trusted by both jumpbox00 and jumpbox01. For example, jb.pem under ~/.ssh/, then run the following command: ssh-add ~/.ssh/jb.pem  2) Add the following … Continue reading Double SSH Hops example

Troubleshoot high CPU usage java process


This is a real troubleshooting example that I just did yesterday for a high CPU usage java application. The application uses tomcat and runs on AWS EC2. Login into the box, and change to root user so you can see all users' process. sudo su - Install htop if you have not installed it before, the run it. … Continue reading Troubleshoot high CPU usage java process

Bamboo plan for AWS build part 4


Swap URL: This is for blue/green deployment. The idea is to update the floating DNS to point to different stack's ELB CNAME. It allows us to easily divert the traffics to different backend stack, but keep it transparent to users. Cloudformation snippet "Resources": { "Route53DNSRecord": { "Properties": { "Comment": "Floating DNS Record", "HostedZoneName": { "Fn::Join": … Continue reading Bamboo plan for AWS build part 4