Top 25 Reactive Big Data (JVM) Resources and Articles

Ingestion system in Scala/Akka/Spray/… seems like a pumped up super Flume
Smack Stack (Mesos + )
Avro Resources
Kite works with avro http://kitesdk.org/docs/1.0.0/
Key Features:
  • Row based with schema
  • schema in the file
  • schema is json
  • block compression, splittable files
  • schema evolution
Spark : @databricksMesos : @mesosphere Akka : @typesafe Cassandra : @DataStax Kafka : @ConfluentInc
Akka/Avro Persistence?  https://github.com/wandoulabs/chana
Kafka-Pgsql
Kafka-Avro-REST
Akka / Event Sourcing
Apache Gora
Java 8 Streams
Do we store our Avro in HBase?
Apache Flume for Avro Consumption
Do we store and process Avro files on HDFS?
AKKA
hacked
Replace Chron and Quartz
RDS Postgres or EC2 Postgres
jHiccup (JVM check)

Reactive Programming, Reactive Streams, Lean, Unit Tests, DDD, Microservices, 12 Factor Apps

http://domainlanguage.com/ddd/
http://bravenewgeek.com/stream-processing-and-probabilistic-methods/
http://flexibleboss.com/2015/04/20/lean-and-flexible/
http://theory.stanford.edu/~aiken/publications/papers/asplos06.pdf
http://www.odata.org/

http://www.cnbc.com/id/102620215
http://softwaregarden.io/common-mistakes-junior-developers-writing-unit-tests/
https://github.com/light/JOps/blob/master/README.md
https://twitter.com/hashtag/craftconf%20slides?s=03

https://github.com/agilemobiledev/email-service-akka/blob/master/src/main/java/com/email/EmailService.java

Rebuilding your infrastructure

http://engineeringblog.yelp.com/2015/03/using-services-to-break-down-monoliths.html

Marathon and Mesos
HAProxy, SmartStack, Zookeeper
Alerting with ElasticSearch (https://elastalert.readthedocs.org/en/latest/)
Postgresql
Redis Driver – Lettuce – has some nice features
Java Redis Orm
Advanced Java Data Structures
Redis Meetup – NY 04-August-2015
SSL Tunnel with Redis
Java with Redis
Redis Proxy
Testing Points
Linux Performance Tools
Docker/Microservices issues
Consul and Puppet for Service Discovery & Mgmt
Drill for Mongo query
API
Moving to Scala

Microservices, Devops, Scala, Akka, Spring Boot and more…

Spring Boot Microservices

RXJava/Reactive

 Microservices

New Open Source From IBM

NodeJS Metrics

Agentless System Crawler

DevOps Report

Scala / AKKA

Everything about Tor

Containers

Java 8

Scala and more… Links

http://techblog.realestate.com.au/java-to-scala-cheatsheet/

http://danielwestheide.com/scala/neophytes.html

http://stackoverflow.com/tags/scala/info

http://www.codecommit.com/blog/scala/roundup-scala-for-java-refugees

http://www.scala-lang.org/docu/files/ScalaByExample.pdf

https://speakerdeck.com/agemooij/between-zero-and-hero-scala-tips-and-tricks-for-the-intermediate-scala-developer

http://twitter.github.io/effectivescala/

http://twitter.github.io/scala_school/basics.html

http://www.artima.com/scalazine

Scala + Mongo

http://www.slideshare.net/search/slideshow?searchfrom=header&q=scala+mongo

Scala + redis

http://www.slideshare.net/luksow/microservices-playworkshopiteratorswarsjawa2014?qid=aa0fd58f-c9c5-4f57-925c-dcbe9c6c35b1&v=default&b=&from_search=2

Scala + Postgresql

Akka

http://www.slideshare.net/ktoso/akka-persistence-event-sourcing-in-30-minutes?qid=d2a6270f-ffad-425f-a15f-e1e3123e275d&v=default&b=&from_search=2

http://www.slideshare.net/bantonsson/akka-persistencescaladays2014

http://www.slideshare.net/patriknw/akka-persistence-webinar?qid=d2a6270f-ffad-425f-a15f-e1e3123e275d&v=default&b=&from_search=5

http://www.slideshare.net/jboner/akka-simpler-scalability-faulttolerance-concurrency-remoting-through-actors?qid=d2a6270f-ffad-425f-a15f-e1e3123e275d&v=default&b=&from_search=6

http://www.slideshare.net/NLJUG/eventsourced-architectures-with-akka-sander-mak

http://www.slideshare.net/romantimushev/akka-patterns

http://www.slideshare.net/RoyRusso1/introduction-to-akka-atlanta-java-users-group?qid=d2a6270f-ffad-425f-a15f-e1e3123e275d&v=default&b=&from_search=9

http://www.slideshare.net/matyjas/scaling-web-apps-with-akka?qid=d2a6270f-ffad-425f-a15f-e1e3123e275d&v=default&b=&from_search=10

http://www.slideshare.net/shinolajla/real-world-akka-recepies-v3

http://www.slideshare.net/NLJUG/akka-in-practice-age-mooij-and-raymond-roestenburg

Spark

Rx-Netty, Netflix Karyon 2

http://www.javacodegeeks.com/2015/06/rx-netty-and-karyon2-based-cloud-ready-microservice.html

https://github.com/Netflix/karyon

https://github.com/Netflix/karyon/tree/master/karyon2-examples

https://github.com/ReactiveX/RxNetty

https://github.com/ReactiveX/RxNetty/blob/0.x/rxnetty-examples/src/main/java/io/reactivex/netty/examples/http/helloworld/HelloWorldServer.java

https://github.com/ReactiveX/RxNetty/blob/0.x/rxnetty-examples/src/test/java/io/reactivex/netty/examples/http/helloworld/HelloWorldTest.java

https://github.com/Netflix/karyon/wiki

https://github.com/bijukunjummen/sample-ping-pong-netflixoss/tree/master/sample-pong

Slides from 2015 NE Scala

http://www.nescala.org/

http://spray.io/nescala2015/#/

http://www.slideshare.net/tobym/a-sane-approach-to-microservices

https://www.dropbox.com/s/9pjla37wg3imnvg/Interpreter.pdf

http://plastic-idolatry.com/erik/nescala2015.pdf

http://rapture.io/talks/inference/boston.html

https://speakerdeck.com/tixxit/speed-correctness-or-simplicity-choose-3

https://docs.google.com/presentation/d/1hBGqRHfOrBMnVxuCcb20oaFv_wkjdPBGush9FpFbzgs/pub?slide=id.i0

https://www.dropbox.com/sh/vt9u9s4p7mprst6/AAC9Xc-Xzn5UXOHmXhWuuzzla?dl=0

http://www.meetup.com/nescala/

https://speakerdeck.com/tixxit/speed-correctness-or-simplicity-choose-3

https://twitter.com/nescalas

http://www.meetup.com/ny-scala/

https://www.parleys.com/channel/scala-days-san-francisco-2015

http://www.meetup.com/scala-phase

http://phillyemergingtech.com/

Git R Done – Scalathon 2012 – Notes

Scalathon 2012
Anatomy
giter8
   conscript application
   typesafe
template engine
github or any git repository
push to github to update template
no authentication
use github account
use branches,tags
use master/head for template
IRC Freenode
Scalathon | the international Scala hackathon | July 15-17 2011 | http://scalathon.org
#scalathon
Assumptions
works for github, but can work for any git repository
users or org/repo
Template
dispatch/dispatch.g8
g8 is assumed extension
90% convention

<n8han> https://github.com/n8han/giter8
<n8han> cs n8han/giter8
<n8han> g8 dispatch/dispatch

use in the bin directory only

will install it locally

generates sbt script / directory structure

publishing to sonatype is eventually mirrored to maven

resolver

sbt 0.12

http://www.scala-sbt.org/learn.html

https://github.com/n8han/conscript#readme

sbt run

ny scala meetup

meetup

need a suite of tests to ensure that nothing was broken when they added features

giter8 – depends on github api, now using git interface

will use jgit soon

>g8

giter8 0.4.5
Usage: g8 [TEMPLATE] [OPTION]…
Apply specified template.

OPTIONS
-a, –auth <login>:<password>
Authorizes oauth access to Github
-b, –branch
Resolves a template within a given branch
–paramname=paramvalue
Set given parameter value and bypass interaction.


Apply template and interactively fulfill parameters.
g8 n8han/giter8

Apply template from a remote branch
g8 n8han/giter8 -b some-branch

Apply given name parameter and use defaults for all others.
g8 n8han/giter8 –name=template-test

Acquire Github authorization
g8 -a login:password

 

can pass in named parameters

Build your own giter8 template

 

g8 n8han/giter8

 

your scala sbt is in sbt plugins are a sbt beneath

 

https://github.com/sbt/xsbt-gpg-plugin/

 

proguard is not working with latest typesafe stack

issue with android + scala

we reject that jars have to be skinny

latest sbt on windows is 0.11.3

 

tree on windows 7 console

 

C:Usersspanntbinnoisewebtemplate>tree
Folder PATH listing for volume PC COE
Volume serial number is F6F0-B6B2
C:.
├───project
│ ├───project
│ │ └───target
│ │ ├───scala-2.9.1
│ │ │ └───sbt-0.11.3
│ │ │ ├───cache
│ │ │ │ ├───compile
│ │ │ │ └───update
│ │ │ └───classes
│ │ └───streams
│ │ ├───$global
│ │ │ ├───compilers
│ │ │ │ └───$global
│ │ │ ├───ivy-configuration
│ │ │ │ └───$global
│ │ │ ├───ivy-sbt
│ │ │ │ └───$global
│ │ │ ├───project-descriptors
│ │ │ │ └───$global
│ │ │ └───update
│ │ │ └───$global
│ │ └───compile
│ │ ├───$global
│ │ │ └───$global
│ │ ├───compile
│ │ │ └───$global
│ │ ├───compile-inputs
│ │ │ └───$global
│ │ ├───copy-resources
│ │ │ └───$global
│ │ └───defined-sbt-plugins
│ │ └───$global
│ └───target
│ ├───config-classes
│ ├───scala-2.9.1
│ │ └───sbt-0.11.3
│ │ └───cache
│ │ ├───compile
│ │ └───update
│ └───streams
│ ├───$global
│ │ ├───compilers
│ │ │ └───$global
│ │ ├───ivy-configuration
│ │ │ └───$global
│ │ ├───ivy-sbt
│ │ │ └───$global
│ │ ├───project-descriptors
│ │ │ └───$global
│ │ └───update
│ │ └───$global
│ └───compile
│ ├───$global
│ │ └───$global
│ ├───compile
│ │ └───$global
│ ├───compile-inputs
│ │ └───$global
│ ├───copy-resources
│ │ └───$global
│ └───defined-sbt-plugins
│ └───$global
└───src
└───main
└───g8
└───src
└───main
└───scala

 
default.properties
srcmaing8
this is input for the template mechanism
 
default.properties used for the build, this doesn’t show up
 
creates a standard Scala SBT project
 
Stub.scala
 
 
sbt
 
“sbt console”
> g8-test
 
 
SBT
build.sbt
 
srcmainscala
 

Directory of C:UsersspanntDesktopScalamydirsrcmainscala

07/27/2012 01:50 PM <DIR> .
07/27/2012 01:50 PM <DIR> ..
07/27/2012 01:50 PM 589 Hello$.class
07/27/2012 01:50 PM 605 Hello.class
07/27/2012 01:50 PM 292 Hello.scala
3 File(s) 1,486 bytes
2 Dir(s) 115,960,180,736 bytes free

sbt

run

info, warning, error – logs

sbt
[info] Set current project to default-469610 (in build file:/C:/Users/spannt/Des
ktop/Scala/mydir/)
> run
[info] Updating {file:/C:/Users/spannt/Desktop/Scala/mydir/}default-469610…
[info] Resolving org.scala-lang#scala-library;2.9.1 …
[info] Done updating.
[info] Compiling 1 Scala source to C:UsersspanntDesktopScalamydirtargetsc
ala-2.9.1classes…
[info] Running Hello
Hello World
[success] Total time: 3 s, completed Jul 27, 2012 1:55:44 PM
>

leave sbt running

works like SPRING ROO

SBT grabs it’s own SCALA on the web

 

conscript example giter8 (g8) template – https://github.com/softprops/conscript-example.g8

conscript (cs) example – https://github.com/softprops/conscript-example

 

can combine java and scala

integrate scala calling java

continuous build on src changes

 

~run

~compile

~test

compile, compile test, run test

can use ~ before anything

 

change  sbt

up xmx

up stack

hotspot options for permgen

 

put into build.sbt

https://github.com/harrah/xsbt/wiki/Getting-Started-Basic-Def

cross scala version if your library supports many version

sbt commands

reload

+compile, +test, +run

compile for every version in cross scala versions

entry point if you have many, if you have only one entry point it will find it

libraryDependencies += "org.apache.derby" % "derby" % "10.4.1.3"
can use Java and scala libraries
%%
  append my scala version to the version number

https://github.com/ymasory/sbt-prototype
test writer
https://github.com/rickynils/scalacheck
Junit
ScalaUnit
ScalaCheck

more build.properties
sbt.version=0.11.3

SBT is continuous build . like Roo or maven but more

src/main/scala

build.sbt

see sbt-prototype

don’t look at googlecode
Publishing to SBT local repo (at least for my home directory) publishTo := Some(Resolver.file(“maven-local-repo”, file(“/home/jsuereth/.m2/repository”)))
#sbt
#scala
IRC freenode
Setting up an Akka “hello world” project using SBT (and also ‘sbt make-pom’ to let Netbeans open the project and be my .scala editor) https://github.com/tvaughan77/scalathon-akka
https://github.com/harrah/xsbt/wiki/Getting-Started-Basic-Def

https://github.com/ymasory/sbt-prototype

https://github.com/rickynils/scalacheck

one run for ~compile always on

one running profilers/test

https://github.com/softprops/conscript-example.g8

https://github.com/softprops/conscript-example

http://adopting-conscript.lessis.me/#0

https://github.com/tvaughan77/scalathon-akka

 

Scala Basics

doesnt care what the name or location of the file

can put dozens of objects into one file

we’re talking about http://pamflet.databinder.net/Pamflet.html in the front

programming in scala 2nd edition  : odersky

http://pragprog.com/magazines/2012-07/scala-for-the-intrigued

Free Node   IRC

irc.freedone.net

#scalathon

Setting up SBT

https://github.com/harrah/xsbt/wiki/Getting-Started-Setup
Scala on Heroku

http://blog.heroku.com/archives/2011/10/3/scala/

https://devcenter.heroku.com/articles/scaling-out-with-scala-and-akka

http://twitter.github.com/scala_school/

http://www.planetscala.com/

http://scala-ide.org/

https://devcenter.heroku.com/articles/scala

https://github.com/foursquare/rogue/

http://www.assembla.com/wiki/show/liftweb/Mongo_Record_Basics

http://www.mongodb.org/display/DOCS/Scala+Language+Center

http://api.mongodb.org/scala/casbah/current/tutorial.html

http://www.assembla.com/wiki/show/liftweb/MongoDB

http://liftweb.net/

https://github.com/jdegoes/blueeyes

https://github.com/Mironor/Play-2.0-Scala-MongoDb-Salat-exemple

https://github.com/vznet/play-mongo-jackson-mapper

http://fabiokung.com/2011/08/06/running-java-web-applications-on-heroku-cedar-stack/
Main

http://www.jamesward.com/2011/10/19/running-play-framework-scala-apps-on-heroku
http://www.jamesward.com/2012/02/21/play-framework-2-with-scala-anorm-json-coffeescript-jquery-heroku

http://whatiscomingtomyhead.wordpress.com/2012/01/28/getting-started-with-play-2-0-heroku-and-scala/

http://scala.herokuapp.com/

http://typesafe.com/stack/download

downloaded heroku toolbelt

downloaded sbt

http://www.scala-lang.org/downloads
http://scala-ide.org/ 2.0.1

http://scala-ide.org/docs/tutorials/index.html

http://ant.apache.org/ivy/download.cgi

http://thedailywtf.com/

http://www.codecademy.com/courses/basic-jquery?curriculum_id=4f4bdd96848740000300026a

http://yuvimasory.com/software

https://gist.github.com/3191654

https://gist.github.com/3190486

https://github.com/harrah/xsbt

https://github.com/ymasory/scalaz-talk-phase

 

 

 
 
 

 

 

 

 

 

 

ETE Presentations

Chariot Solutions Presentations / ETE Presentations
http://chariotsolutions.com/presentations

Chariot Solutions Emerging Technology Philly ETE 2012 Screencasts
http://emergingtech.chariotsolutions.com/category/screencasts/philly-ete-2012/

Chariot Solutions Videos and ETE Videos
http://www.youtube.com/user/ChariotSolutions/featured

CoffeeScript Edge
http://phillyemergingtech.com/2012/system/presentations/the_coffeescript_edge.pdf

Emerging Languages
http://phillyemergingtech.com/2012/system/presentations/Payne_Philly_ETE_2012_slides.pdf

StratisfiedJS (Structured JS)
http://onilabs.com/stratifiedjs

Large Scale Agile
http://phillyemergingtech.com/2012/system/presentations/Large-Scale_Agile_slides.pdf

Spring ROO with Addons
http://phillyemergingtech.com/2012/system/presentations/roo-addons.key.pdf

Backbone.js / Real-time Web Apps
http://phillyemergingtech.com/2012/system/presentations/realtime-web-ete-2012.pdf

Rich-Web Apps with Server Side Java
http://s3.amazonaws.com/chariot-website-production/presentation_documents/documents/000/000/632/vaadin-ria-in-server-side.pdf?AWSAccessKeyId=AKIAJTXMORYAM7NJWIJQ&Expires=1340418954&Signature=7%2BrH1fNE2Zn7vqlaw9ZDYQCQbvs%3D

Vaadin Java Framwork
https://vaadin.com/home

Interesting Stack (Backbone, NodeJS, Restify, MongoDB)
http://backbonetutorials.com/nodejs-restify-mongodb-mongoose/

Dependecy Injection
http://s3.amazonaws.com/chariot-website-production/presentation_documents/documents/000/000/607/di-without-the-gymnastics.pdf?AWSAccessKeyId=AKIAJTXMORYAM7NJWIJQ&Expires=1340422342&Signature=VVMi2uiG2c6m6oaP3%2Bn61pDmHbI%3D

HTML5 Apps with Java and Scala with Play
http://chariotsolutions.com/presentations/html5-apps-in-java-scala-with-the-play-framework

FindBugs
http://chariotsolutions.com/presentations/effective-use-of-findbugs-in-large-software-develo

Massive Scaling
http://chariotsolutions.com/presentations/massively-scaling-to-millions-of-players

Lean, Kanban and Large Scale Agile
http://chariotsolutions.com/presentations/lean-kanban-and-large-scale-agile

Let’s Play TDD
http://jamesshore.com/Blog/Lets-Play/Episode-199.html

Cross Platform Mobile Experience
http://phillyemergingtech.com/2012/system/presentations/Doug_Bellenger-PhillyETE_2012.pdf

Real-Time Web Apps with Backbone
http://chariotsolutions.com/presentations/building-real-time-web-applications

Better Agile Through Tribes
http://chariotsolutions.com/presentations/better%C2%A0agile-thought-throughtribes

Java EE in the Cloud(s)
http://chariotsolutions.com/presentations/java-ee-in-the-clouds

CSS#
http://chariotsolutions.com/presentations/evolution-of-css-layout-through-css3-and-beyond

EmberJS
http://chariotsolutions.com/presentations/emberjs-attacking-boilerplate-where-it-lives

PJAX
http://chariotsolutions.com/presentations/pjax-and-the-next-generation-of-server-side-web-fr

Grails 2.0
http://chariotsolutions.com/presentations/whats-new-in-grails-20

JavaScript Testing / BDD