Controlling Multiple Linux Boxes at Once

For running a few commands to multiple boxes

https://github.com/denis-sokolov/multi-ssh

http://sourceforge.net/projects/clusterssh/

https://github.com/duncs/clusterssh

https://code.google.com/p/parallel-ssh/

http://rundeck.org/

http://web.taranis.org/shmux/

find . -iname “*myfile*”

grep -iR “mytext” *

Of course, Puppet or Chef is a better option.    Or just use AWS ElasticBeanStalk or CloudFoundry.

Microservices, Devops, Scala, Akka, Spring Boot and more…

Spring Boot Microservices

RXJava/Reactive

 Microservices

New Open Source From IBM

NodeJS Metrics

Agentless System Crawler

DevOps Report

Scala / AKKA

Everything about Tor

Containers

Java 8

Redis Speed Burn

Redis commands per second

Improve performance with Twemproxy or dynomite

https://github.com/twitter/twemproxy

https://github.com/Netflix/dynomite

https://blog.recurly.com/2014/05/clustering-redis-maximize-uptime-scale

https://github.com/areina/smitty

https://github.com/wandoulabs/codis

http://redis.io/topics/clients

http://redis.io/commands/MONITOR

http://redis.io/topics/benchmarks

http://skipperkongen.dk/2013/08/27/how-many-requests-per-second-can-i-get-out-of-redis/

http://redis.io/topics/partitioning

https://www.datadoghq.com/wp-content/uploads/2013/09/Understanding-the-Top-5-Redis-Performance-Metrics.pdf

http://objectrocket.com/blog/how-to/10-quick-tips-about-redis

http://redis.io/topics/cluster-tutorial

Mac Ulimit

http://unix.stackexchange.com/questions/108174/how-to-persist-ulimit-settings-in-osx-mavericks

 

Test Data and Generated Data

http://www.fakenamegenerator.com/advanced.php?t=country&n%5B%5D=us&c%5B%5D=us&gen=50&age-min=19&age-max=85

http://www.wordgenerator.net/fake-word-generator.php

http://www.json-generator.com/    <–  nice one

http://www.generatedata.com/

http://www.agilemobiledeveloper.com/2014/01/06/data-generator-and-data-migration/

https://www.mockaroo.com/

http://www.fakemailgenerator.com/   <- really but they receive email

http://www.databasetestdata.com/generated-data

http://www.convertcsv.com/generate-test-data.htm

http://c2.com/cgi/wiki?TestDataGenerator

https://github.com/mvisonneau/docker-generatedata

http://finraos.github.io/DataGenerator/

https://randomapi.com/

https://randomuser.me/

http://www.freedomaingenerator.com/

QuickHit: Check Mail Checking from AWS EC2 with JavaMail

To avoid having to install maven or gradle or specialized tools, we just put in a simple class and shell script.    First download the mail jars you need from here.   A simple wget will work on the links.  Keep them in the same dir as the .java and you can use the shell script below.   Real quick and dirty, but works for testing if your local EC2 instance can send email and there’s no firewall or other issues.

import javax.mail.Address;
import javax.mail.Message;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
import java.util.Properties;

try { Properties props = System.getProperties();
                props.put("mail.smtp.host", "localhost");
                props.put("mail.debug", "true");
                Session session = Session.getDefaultInstance(props, null);
                session.setDebug(true);
                Transport transport = session.getTransport("smtp");
                MimeMessage message = new MimeMessage(session);
                Address fromAddress = new InternetAddress("root@localhost.localdomain");
                message.setFrom(fromAddress);
                InternetAddress to = new InternetAddress("tspann@COOLDOMAIN.com");
                message.addRecipient(Message.RecipientType.TO, to);
                message.setSubject("Test Mail");
                message.setText("Test Body");
                transport.connect(host, from);
                message.saveChanges();
                Transport.send(message);
                transport.close();
            } catch (Exception x) {
                x.printStackTrace();
            }
    }
}

mailcheck.sh

javac -cp imap-1.5.4.jar:javax.mail.jar:smtp-1.5.4.jar MailCheck.java
java -cp imap-1.5.4.jar:javax.mail.jar:smtp-1.5.4.jar:. MailCheck

Setup Jenkins and DevTools on Ubuntu on AWS EC2

 

Install Git, Java7, Java 8 and Maven

sudo apt-get install git

sudo apt-add-repository ppa:webupd8team/java

sudo apt-get update

sudo apt-get install oracle-java8-installer

sudo apt-get install oracle-java7-installer

sudo apt-get install oracle-java8-set-default

sudo apt-get install maven

 

Install Jenkins

sudo wget -q -O – https://jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add –
sudo sh -c ‘echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list’
sudo apt-get update
sudo apt-get install jenkins

 

Install MongoDB

 

http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/

sudo apt-key adv –keyserver hkp://keyserver.ubuntu.com:80 –recv 7F0CEB10

echo “deb http://repo.mongodb.org/apt/ubuntu “$(lsb_release -sc)”/mongodb-org/3.0 multiverse” | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list

sudo apt-get update

sudo apt-get install -y mongodb-org

sudo apt-get install -y mongodb-org=3.0.4 mongodb-org-server=3.0.4 mongodb-org-shell=3.0.4 mongodb-org-mongos=3.0.4 mongodb-org-tools=3.0.4

sudo service mongod start

sudo tail -5000 /var/log/mongodb/mongod.log

 

Install Postgresql

https://www.digitalocean.com/community/tutorials/how-to-install-and-use-postgresql-on-ubuntu-14-04

sudo apt-get update
sudo apt-get install postgresql postgresql-contrib

 

Install Redis

https://www.digitalocean.com/community/tutorials/how-to-install-and-use-redis

sudo apt-get install build-essential

sudo apt-get install tcl8.5

sudo wget http://download.redis.io/releases/redis-stable.tar.gz

sudo tar xzf redis-stable.tar.gz

cd redis-stable

make

make test

sudo make install

cd utils

sudo ./install_server.sh

sudo service redis_6379 start

redis-cli

Install Scala

http://blog.prabeeshk.com/blog/2014/10/31/install-apache-spark-on-ubuntu-14-dot-04/

http://www.scala-lang.org/download/2.11.7.html

sudo wget http://downloads.typesafe.com/scala/2.11.7/scala-2.11.7.deb

sudo dpkg -i scala-2.11.7.deb

 

Install SBT and Gradle

http://www.scala-sbt.org/0.13/tutorial/Installing-sbt-on-Linux.html

echo “deb http://dl.bintray.com/sbt/debian /” | sudo tee -a /etc/apt/sources.list.d/sbt.list
sudo apt-get update
sudo apt-get install sbt

install typesafe activator

sudo wget http://downloads.typesafe.com/typesafe-activator/1.3.5/typesafe-activator-1.3.5.zip

 

sudo apt-get install unzip

curl -s get.gvmtool.net | bash

source “/root/.gvm/bin/gvm-init.sh”

gvm install gradle

 

Notes

https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+on+Ubuntu

https://wiki.jenkins-ci.org/display/JENKINS/Step+by+step+guide+to+set+up+master+and+slave+machines

https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds

Setup back for jenkins

https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+Best+Practices

 

/var/log/jenkins/jenkins.log

 

Add 8080 to all same ports as 80

http://MYIP:8080/configure

 

 

AWS Summit 2015 Slides of Interest

AWS Security

http://www.slideshare.net/AmazonWebServices/nyc-room2-architectingforgreatersecurityinawsbillshinn

http://www.slideshare.net/AmazonWebServices/intro-to-aws-security-50364629

Cost Optimization

http://www.slideshare.net/AmazonWebServices/nyc-room1-costoptimizationfinal20150708

http://www.slideshare.net/AmazonWebServices/creating-a-culture-of-cost-management-in-your-organization-50364904

DevOps

http://www.slideshare.net/AmazonWebServices/nyc-room1-accelerating-devops-pipelines-with-awskoswilms

http://www.slideshare.net/AmazonWebServices/devops-at-amazon-a-look-at-our-tools-and-processes-50364582

AWS EC2 Contianers (DOCKER)

http://www.slideshare.net/AmazonWebServices/amazon-ec2-container-service-manage-dockerenabled-apps-in-ec2-50366981

Introduction to EC2

http://www.slideshare.net/AmazonWebServices/intro-to-aws-amazon-ec2-and-compute-services-50366686

Scaling

http://www.slideshare.net/AmazonWebServices/deep-dive-scaling-up-to-your-first-10-million-users-50366610

VPC

http://www.slideshare.net/AmazonWebServices/deep-dive-amazon-virtual-private-cloud-50365029

http://www.slideshare.net/AmazonWebServices/deep-dive-amazon-virtual-private-cloud-50365085

AWS Elasticache

http://www.slideshare.net/AmazonWebServices/intro-to-aws-database-services-50364961

AWS Lambda

http://www.slideshare.net/AmazonWebServices/aws-lambda-eventdriven-code-in-the-cloud-50364719

AWS EBS

http://www.slideshare.net/AmazonWebServices/deep-dive-maximizing-amazon-ec2-and-amazon-elastic-block-storeperformance