Rebuilding your infrastructure

http://engineeringblog.yelp.com/2015/03/using-services-to-break-down-monoliths.html

Marathon and Mesos
HAProxy, SmartStack, Zookeeper
Alerting with ElasticSearch (https://elastalert.readthedocs.org/en/latest/)
Postgresql
Redis Driver – Lettuce – has some nice features
Java Redis Orm
Advanced Java Data Structures
Redis Meetup – NY 04-August-2015
SSL Tunnel with Redis
Java with Redis
Redis Proxy
Testing Points
Linux Performance Tools
Docker/Microservices issues
Consul and Puppet for Service Discovery & Mgmt
Drill for Mongo query
API
Moving to Scala

Apache Drill with AVRO and MongoDB

Drill with REST and JDBC -> AVRO, MONGO
Set up Zookeeper based cluster
!quit
firewall was blocking it < watch these ports
select * from dfs.`/Users/timothyspann/Downloads/apache-drill-1.1.0/sample-data/region.parquet`
SELECT * FROM cp.`employee.json` LIMIT 20
show databases;
use mongo.`security-mydatabase`;
show tables;
 
Mongo Issue:   I have similar observations. At the moment I can only query elements that are strings. Numbers and dates do not work.
This is Drill 1.1 with Mongo 3.0 issue
bin/sqlline –u jdbc:drill:schema=dfs;zk=local

10gen: M101J MongoDB for Java Developers – Week 2

M101J

This class has been going great.

The flow of the class is great and it’s worked well on Windows with Eclipse, Maven and JDK 1.6.

Some notes and links:

BSON
document oriented database
The Spark web framework for Java is nice, very small and simple.

 

MongoClient client = new MongoClient();
DB db = client.getDB(“school”);
DBCollection people = db.getCollection(“people”);

 

Hibernate OGM – MongoDB vs Kundera vs Jongo vs MongoDB API vs Morphia vs Spring Data Mongo – MongoDB Drivers for Java

I am working on a Spring MVC app that demonstrates all of the different MongoDB Java APIs.

Some Links
http://code.google.com/p/morphia/wiki/QuickStart

http://www.mongodb.org/display/DOCS/Java+Tutorial

http://jongo.org/#updating

https://github.com/hibernate/hibernate-ogm

https://openshift.redhat.com/community/blogs/configuring-hibernateogm-for-your-jboss-app-using-mongodb-on-openshift-paas

http://www.hibernate.org/subprojects/ogm.html

https://github.com/impetus-opensource/Kundera/wiki

https://github.com/impetus-opensource/Kundera/wiki/Getting-Started-in-5-minutes

http://blog.fisharefriends.us/morphia-vs-spring-data-mongodb/

http://www.ibm.com/developerworks/java/library/j-morphia/index.html

Maven POM Settings For Various Drivers

	
		morphia
		Morphia
		http://morphia.googlecode.com/svn/mavenrepo/
		default

		    sonatype-nexus
		    Kundera Public Repository
		    https://oss.sonatype.org/content/repositories/releases

			true

			false

		kundera-missing
		Kundera Public Missing Resources Repository
		http://kundera.googlecode.com/svn/maven2/maven-missing-resources

			true

			true

        com.google.code.morphia
        morphia
        0.99

        org.hibernate.ogm
        hibernate-ogm-core
        4.0.0-SNAPSHOT
        provided

	org.mongodb
	mongo-java-driver
	2.10.1

 org.springframework.data
 spring-data-mongodb
 1.0.4.RELEASE

Hibernate OGM for MongoDB
https://community.jboss.org/wiki/PortingSeamHotelBookingExampleToOGM
https://github.com/ajf8/seam-booking-ogm
https://openshift.redhat.com/community/blogs/configuring-hibernateogm-for-your-jboss-app-using-mongodb-on-openshift-paas
https://github.com/openshift/openshift-ogm-quickstart

Kundera (JPA for MongoDB)
https://github.com/impetus-opensource/Kundera-Examples/wiki/Using-Kundera-with-Spring
https://github.com/impetus-opensource/Kundera
https://github.com/impetus-opensource/kundera-mongo-performance
https://github.com/impetus-opensource/Kundera-Examples
https://github.com/impetus-opensource/Kundera/wiki/Sample-Codes-and-Examples
https://github.com/impetus-opensource/Kundera-Examples/wiki/Twitter
http://architects.dzone.com/articles/sqlifying-nosql-%E2%80%93-are-orm
https://github.com/xamry/twitample
https://github.com/impetus-opensource/Kundera-Examples/wiki/Cross-datastore-persistence-using-Kundera
http://prabhubuzz.wordpress.com/2012/05/25/mongodb-cassandra-jpa-service-using-kundera/
http://gora.apache.org/
http://xamry.wordpress.com/2011/05/02/working-with-mongodb-using-kundera/
https://github.com/impetus-opensource/Kundera/wiki/Getting-Started-in-5-minutes
https://github.com/impetus-opensource/Kundera/wiki/Concepts

Raspberry Pi and Development

Here are some links I have found useful in setting up my Raspberry Pi machine for development and more.
Raspberry Pi
Raspberry Pi on My TV

 

 


MongoDB
Development on Pi
Multimedia
C-64 Emulator (Games)
UPnP
Java on Raspberry Pi

Eclipse/Java

Some Useful Updates via Raspian/Debian
sudo apt-get update
sudo apt-get install eclipse

Java library for Raspberry Pi
Java Library for RPi (Pi4J)
JVM on Pi
Oracle JVM Downloads
JDK1.7 on Pi

General Raspberry Pi Usage

Node.JS on Raspberry Pi
Scala on Raspberry Pi

Installing the TypeSafe Stack

 

sj:mug Holiday Mongo Talks

MongoDB @ HP in Mt Laurel, NJ

 

 

My local demo is hitting a MongoDB instance at MongoLab, the provide free hosting for a small dataset.

 

I have a simple application based on Spring Data for Mongodb.

 

MongoOperations mongoOps = null;
 AppConfig config = new AppConfig();
 try {
 mongoOps = config.mongoTemplate();
 } catch (Exception e) {
 e.printStackTrace();
 logger.error("Mongo Connection Error", e);
 }

 Person person = mongoOps.findOne(new Query(where("name").is("Joe")), Person.class);
 logger.info(person.toString());

 model.addAttribute("MongoInfo", person.toString());

 

august 2012 – 2.2
aggregation framework
improved concurrency
mongo is great for email storage / documents
aggregation  – unix style piping
TTL (time to live)
garbage collect data
unwind an array

 

 

mongodb meeting with SJ

Introduction to MongoDB

 

 

A first time meetup in South Jersey at the Fire Hall in Pennsauken.   With some great support from 10gen with info sheets, stickers and pizza.

 

Notes from A. Riveria presentation:

  • Document Data Model
  • Database Technology anywhere…
  • Written in C++
  • Source Code on Github
  • Runs on most platforms
  • BSON (Binary JSON) – binary-encoded serialization
  • JSON collections
  • Embedding of objects/arrays
  • Table => Collection
  • Row => Document
  • Join => Embedded
  • Foreign Key => Reference / Link  (Object Id – autogenerated serial # for each document)
  • Partition => Shard (Shard keys across nodes)
  • Indexes on fields and collections of indexes
  • Great for lat/long
  • Geospatial queries
{} – select all where
db.users.find( {}, {username:1, firstname: 1, lastname:1, email:1}
1- true
Results (JSON):
{  username: “smith”,
    firstname: “angel”,
   lastname: “Rivera”,
   email: “a@a.com”
}

 

Insert

var insertString = {username:”smith”, value:1}

   can use quotes around the fieldnames.

 

db.users.insert(insertString);

 

(users is our collection)

 

Update

db.users.update({username: “jones”}, {$set: {email:”a@yahoo.com”}}

 

first section is where, set part does the update on the field(s)

 

MAKE SURE YOU USE $SET modifier in updates.

Only updates first record it encounters.

 

Save

Upserts.  Update or if not found then create/insert.

 

Remove/Delete

 

db.users.remove( {username:”aname”});

 

deletes a document

 

Mongo Query Operators

 

$ne (not equal)

$lt (less than)

$lte (less than equal to)

$gt

 

mongoimport

Also supports CSV, best to do JSON, the default.    Arrays in a field does not work in a CSV.

mongoimport –db sjmug -c users -v –file data.json –host localhost –port 27018 –jsonArray

 

 

 

 

Thanks to 10gen for sponsorship!

 

 

 

Node.js and Mongodb Links

No, Not Python, It’s the Holy Grails…

I’ve been evaluating a few different technologies for doing some rapid web development work and recently I have been looking at the bounty of available tools, libraries, frameworks and associates of Spring.  Spring ROO and Groovy/Grails are the big two RAD tools from SpringSource and they look pretty amazing.  I don’t think I’ve found the Holy Grail of Web App development, but these two tools are really helpful in rapidly developing apps.  I am hoping someone combines Roo or Grails with Vert.x to make a really awesome alternative to Node.JS.  I like Node.JS and have been doing JavaScript for a decade, but I prefer the languages of the JVM and Vert.x’s support for multiple languages is pretty awesome.  Hmmm, Vert.x in Grails.

With the excellent STS inside Eclipse, this gives Groovy and Java a great IDE for development, testing and debugging.  And that is something that is lacking for Node.JS.  Node.JS has a few tools out there, but nothing open source, free, full of awesome mature plugins (PMD, Checkstyle, GIT, Maven, JUnit, FindBugs, …) and just plain easy to use.  Eclipse also has plugins for working with CloudFoundry, Heroku, OpenShift and other Cloud environments.

Between Grails and Roo, I really like the fact that I can remove Roo at anytime and still have a fully functionally application setup that utilizes standard Spring tools.  It’s also easier to add other cool Spring projects like Spring Social, Spring Mobile, Spring Data and more.

Both Grails and Roo support MongoDB very well and that’s great too see.  I have a few Mongo DB instances on MongoLab, MongoHQ and OpenShift.  These services all have great free developer services that are great for learning, prototyping and for tutorials.  I am really loving Heroku and OpenShift.  It’s hard to pick one, so I didn’t.  I use them both.

Vert.x + Groovy is really cool, but again not much IDE support, tools or extra libraries.  It should grow start building support now, but competition with Node.JS is pretty fierce.

Interesting, Roo vs Grails Jobs.

 

 

But if you look at just Spring (which Roo is just a tool to help with a Spring project):

 

 

There’s also some great documentation, tutorials and books on Groovy/Grails since it’s been out for a while:  http://grails.org/Tutorials  http://www.infoq.com/minibooks/grails-getting-started On the other hand, Roo is newer, but since it generates standard Spring + Java, you are really just getting a good head start on your project.

Heroku has a nice article on using both with their awesome platform:   https://devcenter.heroku.com/articles/grails  and https://devcenter.heroku.com/articles/spring-mvc-hibernate.

I will post a Github repository of some samples soon.

 

Spring MVC + Spring Data for Mongodb + Mongolab + OpenShift

I created a small tutorial application following the Spring Data for Mongodb guide with a little bacon for flavor. I wanted to use real hosting for MongoDB, so I created a database at MongoLab.   It was quick and easy and the hosting is fast and works great even remotely for local development.

The tutorial application in process is one to register your Bacon Creation for this year’s BaconFest.  The winner gets a mini BaconExplosion.

This is ONLY a test, you will not receive BACON.
image

ETE Presentations

Chariot Solutions Presentations / ETE Presentations
http://chariotsolutions.com/presentations

Chariot Solutions Emerging Technology Philly ETE 2012 Screencasts
http://emergingtech.chariotsolutions.com/category/screencasts/philly-ete-2012/

Chariot Solutions Videos and ETE Videos
http://www.youtube.com/user/ChariotSolutions/featured

CoffeeScript Edge
http://phillyemergingtech.com/2012/system/presentations/the_coffeescript_edge.pdf

Emerging Languages
http://phillyemergingtech.com/2012/system/presentations/Payne_Philly_ETE_2012_slides.pdf

StratisfiedJS (Structured JS)
http://onilabs.com/stratifiedjs

Large Scale Agile
http://phillyemergingtech.com/2012/system/presentations/Large-Scale_Agile_slides.pdf

Spring ROO with Addons
http://phillyemergingtech.com/2012/system/presentations/roo-addons.key.pdf

Backbone.js / Real-time Web Apps
http://phillyemergingtech.com/2012/system/presentations/realtime-web-ete-2012.pdf

Rich-Web Apps with Server Side Java
http://s3.amazonaws.com/chariot-website-production/presentation_documents/documents/000/000/632/vaadin-ria-in-server-side.pdf?AWSAccessKeyId=AKIAJTXMORYAM7NJWIJQ&Expires=1340418954&Signature=7%2BrH1fNE2Zn7vqlaw9ZDYQCQbvs%3D

Vaadin Java Framwork
https://vaadin.com/home

Interesting Stack (Backbone, NodeJS, Restify, MongoDB)
http://backbonetutorials.com/nodejs-restify-mongodb-mongoose/

Dependecy Injection
http://s3.amazonaws.com/chariot-website-production/presentation_documents/documents/000/000/607/di-without-the-gymnastics.pdf?AWSAccessKeyId=AKIAJTXMORYAM7NJWIJQ&Expires=1340422342&Signature=VVMi2uiG2c6m6oaP3%2Bn61pDmHbI%3D

HTML5 Apps with Java and Scala with Play
http://chariotsolutions.com/presentations/html5-apps-in-java-scala-with-the-play-framework

FindBugs
http://chariotsolutions.com/presentations/effective-use-of-findbugs-in-large-software-develo

Massive Scaling
http://chariotsolutions.com/presentations/massively-scaling-to-millions-of-players

Lean, Kanban and Large Scale Agile
http://chariotsolutions.com/presentations/lean-kanban-and-large-scale-agile

Let’s Play TDD
http://jamesshore.com/Blog/Lets-Play/Episode-199.html

Cross Platform Mobile Experience
http://phillyemergingtech.com/2012/system/presentations/Doug_Bellenger-PhillyETE_2012.pdf

Real-Time Web Apps with Backbone
http://chariotsolutions.com/presentations/building-real-time-web-applications

Better Agile Through Tribes
http://chariotsolutions.com/presentations/better%C2%A0agile-thought-throughtribes

Java EE in the Cloud(s)
http://chariotsolutions.com/presentations/java-ee-in-the-clouds

CSS#
http://chariotsolutions.com/presentations/evolution-of-css-layout-through-css3-and-beyond

EmberJS
http://chariotsolutions.com/presentations/emberjs-attacking-boilerplate-where-it-lives

PJAX
http://chariotsolutions.com/presentations/pjax-and-the-next-generation-of-server-side-web-fr

Grails 2.0
http://chariotsolutions.com/presentations/whats-new-in-grails-20

JavaScript Testing / BDD

Converting to MongoDB

http://blog.programmableweb.com/2011/08/15/how-i-scrapped-mysql-for-mongodb-in-three-hours/

Video
http://www.10gen.com/presentations/mongosf-2011/how-we-switched-from-hibernate-to-mongodb-in-a-week-with-morphia

 

EclipseLink JPA to MongoDB
http://java-persistence-performance.blogspot.com/2012/04/eclipselink-jpa-supports-mongodb.html

http://www.eclipse.org/eclipselink/

 

http://www.playframework.org/modules/morphia

Cloudify Groovy and MongoDB
http://www.cloudifysource.org/guide/qsg/quick_start_guide
http://www.cloudifysource.org/guide/contribute/how_to
http://www.cloudifysource.org/guide/qsg/quick_start_guide#
http://www.cloudifysource.org/guide/qsg/quick_start_guide_ec2
http://www.cloudifysource.org/guide/setup/cloudify_prerequisites 

MongoDB Windows Services
http://www.mongodb.org/display/DOCS/Windows+Service

 Morphia QuickStart
http://code.google.com/p/morphia/wiki/QuickStart

http://blog.mongodb.org/post/5217011262/improving-scalable-java-application-development-with-mon

http://www.springsource.org/spring-data/mongodb

http://www.mkyong.com/mongodb/spring-data-mongodb-hello-world-example/

https://github.com/mongodb/mongo-java-driver

 

Simple MongoDB Example Servlet
http://www.mastertheboss.com/jboss-application-server/318-mondodb-tutorial-for-java-ee-developers.html

http://www.jboss.org/resteasy

http://www.slideshare.net/kbanker/mongodb-schema-design

 

We will also need some JSON.

http://www.json.org/

 

http://cookbook.mongodb.org/patterns/count_tags/

http://stackoverflow.com/questions/1476295/when-to-use-mongodb-or-other-document-oriented-database-systems

http://blog.iprofs.nl/2011/11/25/is-mongodb-a-good-alternative-to-rdbms-databases-like-oracle-and-mysql/

http://mostlymagic.wordpress.com/

http://www.mongodb.org/display/DOCS/Trees+in+MongoDB

http://cookbook.mongodb.org/index.html

http://abdennour-insat.blogspot.com/2012/04/mongodb-helper-hibernate-for-mongodb_09.html

http://skillsmatter.com/podcast/nosql/mongodb-grails

 

http://www.youtube.com/watch?v=qjlFVe_lzZA

http://blog.mongodb.org/post/18510469058/grails-in-the-land-of-mongodb

http://www.slideshare.net/ecspike/java-development-with-mongodb

http://www.redantelope.com/2010/04/using-mongodb-and-groovy/

http://www.gridshore.nl/2010/09/20/learning-mongodb/

http://www.gridshore.nl/2011/06/15/creating-presentations-with-node-js/

http://nosql.mypopescu.com/post/816470307/tutorial-mongodb-in-java

http://blog.mongolab.com/2011/11/ample-mongodb-examples/

https://github.com/mongolab/mongodb-driver-examples

http://support.mongolab.com/entries/20432588-is-there-a-mongolab-tutorial-for-mongodb

http://howtonode.org/express-mongodb

http://www.gridshore.nl/2010/09/20/learning-mongodb/

http://www.gridshore.nl/2010/09/27/still-learning-mongodb/#more-1088

http://www.gridshore.nl/2012/02/28/a-mongodb-based-axon-framework-event-store/#more-1225

http://www.gridshore.nl/2011/07/28/combining-java-and-node-js-through-redis-pubsub-and-a-json-remote-interface/#more-1163

http://try.mongodb.org/

http://backbonetutorials.com/nodejs-restify-mongodb-mongoose/

Android + MongoDB + Node.js (in progress)

Mobilize Your MongoDB
http://www.10gen.com/presentations/mongodb-berlin/2012/mobilize-your-mongodb-developing-iphone-and-android-apps-in-the-cloud
http://androidclip.com/mongodb-mobile-app-backend-301/

MongoDB with Morphia
http://www.javacodegeeks.com/2011/11/using-mongodb-with-morphia.html

http://blog.jeffdouglas.com/2012/03/25/build-an-api-with-node-js-express-mongodb-and-cloud-foundry/
http://blog.jeffdouglas.com/2012/02/02/using-node-js-to-host-development-web-services/ 

LinkedIn’s Platform
http://www.slideshare.net/phegaro/linkedin-mobile-how-do-we-do-it

Node.js at Heroku
http://www.jamesward.com/2011/06/21/getting-started-with-node-js-on-the-cloud/

http://www.mihswat.com/2011/05/04/getting-started-with-cloud-foundry-using-a-node-js-and-mongodb-application/

VMWare Player for Cloud Foundry
http://www.vmware.com/products/player/

Cloud Foundry
http://docs.cloudfoundry.com/infrastructure/micro/installing-mcf.html

 

 

 

mongoDB

Main Documents
http://www.mongodb.org/display/DOCS/Home

MongoDB Manual
http://docs.mongodb.org/manual/

I have MongoDB and RockMongo installed on my OpenShift PaaS site with JBoss.

JBOSS + MongoDB Quick Start
https://github.com/openshift/jbossas-mongoDB-quickstart

MongoDB + Java Tutorial on OpenShift
http://www.mongodb.org/display/DOCS/Java+Tutorial

MongoDB Driver for Java (JavaDocs)
http://api.mongodb.org/java/current/

MongoDB + Android Part 1 + PaaS
https://openshift.redhat.com/community/blogs/mobilize-your-mongodb-developing-iphone-and-android-apps-in-the-cloud-part-1
https://openshift.redhat.com/community/blogs/mobilize-your-mongodb-developing-iphone-and-android-apps-in-the-cloud-part-2

Spatial MongoDB in OpenShift
https://openshift.redhat.com/community/blogs/spatial-mongodb-in-openshift-be-the-next-foursquare-part-1

SoLoMo (social, local, and mobile)

Geospatial Indexing in Mongo
http://www.mongodb.org/display/DOCS/Geospatial+Indexing#GeospatialIndexing-TheEarthisRoundbutMapsareFlat

MongoDB Shell on OpenShift
https://openshift.redhat.com/community/blogs/getting-started-with-mongodb-shell-on-openshift

MongoMonitoringService (MMS) on OpenShift PaaS
https://openshift.redhat.com/community/blogs/getting-started-with-mongo-monitoring-service-mms-on-openshift

MongoDB Native Driver for Node.js
https://github.com/mongodb/node-mongodb-native

MMS Manual
https://mms.10gen.com/help/

MongoDB Free on OpenShift
https://openshift.redhat.com/community/blogs/its-big-its-free-its-easy-mongodb-on-openshift-keeps-getting-better

MongoDB Tutorial
http://www.mongodb.org/display/DOCS/Tutorial

BinaryJSON (Primary Data Representation for MongoDB)
http://bsonspec.org/

SQL to MongoDB Mapping Chart
http://www.mongodb.org/display/DOCS/SQL+to+Mongo+Mapping+Chart

MongoDB Quickstart
http://www.mongodb.org/display/DOCS/Quickstart+Windows

Performance Tuning
http://www.10gen.com/presentations/webinar/diagnostics-and-performance-tuning

Java Driver
http://www.10gen.com/presentations/webinar/whats-new-in-java-driver-version-2.5

Webinar:  Zero to MongoDB Intro for MongoDB with Java
http://www.10gen.com/presentations/webinars/zero-to-mongodb-introduction-for-mongodb-with-java 

Mongo Tutorial
http://www.mongodb.org/display/DOCS/Tutorial

Slides and Videos
http://www.mongodb.org/display/DOCS/Slides+and+Video

Articles
http://www.mongodb.org/display/DOCS/Articles

MongoDB + JavaScript
http://www.mongodb.org/display/DOCS/Javascript+Language+Center

MongoDB + Java
http://www.mongodb.org/display/DOCS/Java+Language+Center

MongoDB + Hadoop
http://www.mongodb.org/display/DOCS/Hadoop

ORM Node.JS for MongoDB
https://github.com/learnboost/mongoose

New MongoDB Docs
https://github.com/mongodb/docs

RHC / OpenShift Starting
https://openshift.redhat.com/app/getting_started