Microservices update

https://blog.yourkarma.com/building-microservices-at-karma

http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/tutorial-spot-instances-java.html

http://www.javaworld.com/article/2940528/developer-tools-ide/eclipses-latest-release-train-carries-docker-java-technologies.html?phint=newt%3Dinfoworld_java&phint=idg_eid%3Dd207dd9989acf6b7ba24c1afc4a835a2#tk.IFWNLE_ifw_java_2015-06-26

http://nginx.com/blog/building-microservices-using-an-api-gateway/

https://medium.com/aws-activate-startup-blog/upgrades-without-tears-part-1-introduction-to-blue-green-deployment-on-aws-e5bcf90eb60b

https://medium.com/aws-activate-startup-blog/upgrades-without-tears-part-2-blue-green-deployment-step-by-step-on-aws-29c0ffe99c60

http://martinfowler.com/articles/refactoring-pipelines.html

http://www.oracle.com/technetwork/articles/java/horstmann-1958988.html

http://blog.codacy.com/2015/06/26/why-we-chose-scala/

http://www.infoq.com/presentations/How-We-Mostly-Moved-from-Java-to-Scala

http://radar.oreilly.com/2015/03/3-simple-reasons-why-you-need-to-learn-scala.html

NOSQL (MongoDB) Injection

https://www.owasp.org/index.php/Testing_for_NoSQL_injection

http://blog.websecurify.com/2014/08/hacking-nodejs-and-mongodb.html

http://www.nosqlmap.net/

https://media.blackhat.com/bh-us-11/Sullivan/BH_US_11_Sullivan_Server_Side_WP.pdf

http://2012.zeronights.org/includes/docs/Firstov%20-%20Attacking%20MongoDB.pdf

http://ieee-security.org/TC/SPW2015/W2SP/papers/W2SP_2015_submission_11.pdf

http://tinple.io/tech/mongo-security-part-two

http://blog.websecurify.com/2014/08/attacks-nodejs-and-mongodb-part-to.html

Using https://en.wikipedia.org/wiki/Rainbow_table

http://www.reddit.com/r/netsec/

https://thecodebarbarian.wordpress.com/2014/09/04/defending-against-query-selector-injection-attacks/

http://docs.mongodb.org/manual/administration/security-checklist/

http://sakurity.com/blog/2015/06/04/mongo_ruby_regexp.html

http://software-talk.org/blog/2015/02/mongodb-nosql-injection-security/

https://github.com/shirishp/NoSQLInjectionDemo

https://blogs.adobe.com/security/files/2011/04/NoSQL-But-Even-Less-Security.pdf

http://www.slideshare.net/44Con/pentesting-nosql-dbs-with-nosql-exploitation-framework

Streaming Large JSON Files with Java 8

As expected, the main options are the two big guys in JSON parsing,  GSON from Google and FasterXML Jackson 2.   Both are very solid and fast, but there’s also a couple of other libraries worth trying.

If you use Jackson, use the latest, FasterXML Jackson 2.  Not Codehaus.

Spring Cloud Configuration

Monday Cool Links

Monitoring and Metrics

https://github.com/nerdalert/cloud-bandwidth

https://www.pagerduty.com/blog/best-metrics-devops-culture/

http://www.slideshare.net/melaniemj/monitoring-is-never-done

https://speakerdeck.com/chrissiebrodigan/measuring-hard-to-measure-things

Culture

http://www.thoughtworks.com/insights/blog/5-tips-being-effective-tech-lead

Tools

https://github.com/EverythingMe/redash

http://githubengineering.com/brubeck/

Servers and Services

https://github.com/CiscoCloud/mesos-consul

DevOps Techniques, Tools and Processes

http://www.daniellesucher.com/presentations/gamedays/#/

http://www.slideshare.net/brendangregg/monitorama-2015-netflix-instance-analysis

Microservices

http://www.slideshare.net/dbryant_uk/devoxxuk-2015-the-seven-deadly-sins-of-microservices-full-version

Reactive

http://saulmm.github.io/when-Iron-Man-becomes-Reactive-Avengers2/

http://www.philosophicalhacker.com/2015/06/19/introduction-to-rxjava-for-android-pt-2/

Big Data

http://blog.acolyer.org/2015/06/18/spinning-fast-iterative-dataflows/

http://vision.cloudera.com/apache-kafka-a-platform-for-real-time-data-streams-part-2/

http://www.slideshare.net/julienledem/how-to-use-parquet-hadoopsummitsanjose2015

Microservices Top 20 Links – 2015

 

Why 12 Factor Application Patterns, Microservices and CloudFoundry Matter? Part 2 – What to change and how.

I first described a difficult scenario that I faced a few years ago.   This article is my recommendations to the current custodians of that project on what to change and how.

With the help of open source frameworks like Spring Cloud and open source platforms like Cloudfoundry, you aren’t forced into having a fragile, manual SDLC.   The first step is to embrace change and join a vibrant community that is succeeding in software development.

 

Overview

  • Increase agility through process, tools and culture
  • Examine the code and depedencies, determine modules based on business domain.
  • Refactor the monolithic EAR into microservices living in JARs guided by the 12 Factor Application patterns and microservices recommendations.
  • 12FA – III – Store configuration in the environment (or even better use a configuration server)
  • Change the Development Process, Frameworks, Libraries and Tools.
  • Change the Build Process and Tools.  Embrace Gradle, Maven, JDK 8.
  • Change the Delivery Process.   Enact Continuous delivery.
  • Change the Database modifications and migration Process and Tools.
  • Change the run-time platform from legacy application server to a PaaS.
  • Centralize the logging.    Using PCF Loggregator and tools like ELK and PaperTrail.
  • Embrace industry standard open source tools, libraries and frameworks
  • Good code fails when you don’t have good process and a platform to help you.
  • Good teams fail when you don’t have a good culture that embraces DevOps, microservices and not giant monoliths.

 

So how could we have improved our process?   The first step is to admin you have a problem and seek help.   By looking to the community, open source and successful startups that have faced similar struggles with monolithic J2EE code bases (Ebay, Netflix), we can determine some functional patterns.

 

Patterns and Principles for Change

  • Use Jenkins to deploy your app with no downtime to CloudFoundry.
  • Use CloudFoundry deployments to do A/B testing with no downtime or difficulty.
  • Bind to database, using the same process in development, test, Q/A and production.   No manual configuration or configuration changes.   Operators make available the correct database settings through the marketplace.  Same manifest and bindings will point to different databases in different spaces.
  • To migrate database changes use flywaydb or liquibase.   These are very well integrated into Spring Boot.
  • Look at NoSQL for part of your database workload, polyglot persistence makes sense in many cases.   For MongoDB development, you handle change management with Mongeez, which is automatically done with JHipster.
  • Smoke tests and automated tests tied to Jenkins.
  • Test against CloudFoundry development for local development, fast deployments, run unit and integration tests locally.
  • Use Maven or Gradle for builds, not ANT.
  • Use Circuit breakers in your web calls to avoid breaking systems and fail safely.
  • Use APM and JMX for deep checks using Pivotal Ops Metrics for CloudFoundry.
  • Have automated scripts check one log stream.  Use tools like ELK (ElasticSearch+Logstash+Kibana) to examine logs.
  • Use CloudFoundry containers that don’t make you SSH to many machines, removing a big manual step that could be a security issue.
  • Move to using smaller, lighterweight Spring Boot Jars and Wars; not giant EAR files.
  • Use lightweight web containers like Jetty, Tomcat, tcServer, or Undertow.
  • Use lightweight frameworks, remember YAGNI and KISS.
  • Read and embrace:  Release It!
  • Break up the apps into smaller, logical components that model real business functionality.
  • Refactor existing code guided by tests, enhancing and adding more tests as the code evolves.  Remembering to use code coverage tools like Jacoco.
  • Design clean RESTful APIs guided by SRP (Single Responsibility Principle).
  • Use smart, modern opinionated frameworks for development like Spring Boot, DropWizard and RatPack.
  • Install GVM and use it to install SpringBoot, Gradle, Groovy, LazyBones and other great tools.  example:   gvm install springboot
  • Use Spring Cloud Connectors, Spring Cloud Netflix projects for Service Discovery and other useful services to enable microservices orchestration and development.
  • Use a very helpful application generator like JHipster.
  • Use Domain Driven Design.
  • Keep to the basics of OOD for existing code and Functional Reactive Programming for new code.
  • Document your API with Spring REST Docs or Swagger.  There is a great presentation on that here.
  • Start your green field projects with one button click or use the Spring CLI (spring init -d=web,data-jpa,actuator,data-rest –build=maven newappdir/).

 

References

Service Discovery

Scala and more… Links

http://techblog.realestate.com.au/java-to-scala-cheatsheet/

http://danielwestheide.com/scala/neophytes.html

http://stackoverflow.com/tags/scala/info

http://www.codecommit.com/blog/scala/roundup-scala-for-java-refugees

http://www.scala-lang.org/docu/files/ScalaByExample.pdf

https://speakerdeck.com/agemooij/between-zero-and-hero-scala-tips-and-tricks-for-the-intermediate-scala-developer

http://twitter.github.io/effectivescala/

http://twitter.github.io/scala_school/basics.html

http://www.artima.com/scalazine

Scala + Mongo

http://www.slideshare.net/search/slideshow?searchfrom=header&q=scala+mongo

Scala + redis

http://www.slideshare.net/luksow/microservices-playworkshopiteratorswarsjawa2014?qid=aa0fd58f-c9c5-4f57-925c-dcbe9c6c35b1&v=default&b=&from_search=2

Scala + Postgresql

Akka

http://www.slideshare.net/ktoso/akka-persistence-event-sourcing-in-30-minutes?qid=d2a6270f-ffad-425f-a15f-e1e3123e275d&v=default&b=&from_search=2

http://www.slideshare.net/bantonsson/akka-persistencescaladays2014

http://www.slideshare.net/patriknw/akka-persistence-webinar?qid=d2a6270f-ffad-425f-a15f-e1e3123e275d&v=default&b=&from_search=5

http://www.slideshare.net/jboner/akka-simpler-scalability-faulttolerance-concurrency-remoting-through-actors?qid=d2a6270f-ffad-425f-a15f-e1e3123e275d&v=default&b=&from_search=6

http://www.slideshare.net/NLJUG/eventsourced-architectures-with-akka-sander-mak

http://www.slideshare.net/romantimushev/akka-patterns

http://www.slideshare.net/RoyRusso1/introduction-to-akka-atlanta-java-users-group?qid=d2a6270f-ffad-425f-a15f-e1e3123e275d&v=default&b=&from_search=9

http://www.slideshare.net/matyjas/scaling-web-apps-with-akka?qid=d2a6270f-ffad-425f-a15f-e1e3123e275d&v=default&b=&from_search=10

http://www.slideshare.net/shinolajla/real-world-akka-recepies-v3

http://www.slideshare.net/NLJUG/akka-in-practice-age-mooij-and-raymond-roestenburg

Spark

Rx-Netty, Netflix Karyon 2

http://www.javacodegeeks.com/2015/06/rx-netty-and-karyon2-based-cloud-ready-microservice.html

https://github.com/Netflix/karyon

https://github.com/Netflix/karyon/tree/master/karyon2-examples

https://github.com/ReactiveX/RxNetty

https://github.com/ReactiveX/RxNetty/blob/0.x/rxnetty-examples/src/main/java/io/reactivex/netty/examples/http/helloworld/HelloWorldServer.java

https://github.com/ReactiveX/RxNetty/blob/0.x/rxnetty-examples/src/test/java/io/reactivex/netty/examples/http/helloworld/HelloWorldTest.java

https://github.com/Netflix/karyon/wiki

https://github.com/bijukunjummen/sample-ping-pong-netflixoss/tree/master/sample-pong

Slides from 2015 NE Scala

http://www.nescala.org/

http://spray.io/nescala2015/#/

http://www.slideshare.net/tobym/a-sane-approach-to-microservices

https://www.dropbox.com/s/9pjla37wg3imnvg/Interpreter.pdf

http://plastic-idolatry.com/erik/nescala2015.pdf

http://rapture.io/talks/inference/boston.html

https://speakerdeck.com/tixxit/speed-correctness-or-simplicity-choose-3

https://docs.google.com/presentation/d/1hBGqRHfOrBMnVxuCcb20oaFv_wkjdPBGush9FpFbzgs/pub?slide=id.i0

https://www.dropbox.com/sh/vt9u9s4p7mprst6/AAC9Xc-Xzn5UXOHmXhWuuzzla?dl=0

http://www.meetup.com/nescala/

https://speakerdeck.com/tixxit/speed-correctness-or-simplicity-choose-3

https://twitter.com/nescalas

http://www.meetup.com/ny-scala/

https://www.parleys.com/channel/scala-days-san-francisco-2015

http://www.meetup.com/scala-phase

http://phillyemergingtech.com/