FLaNK AI - 15 April 2024

15-April-2024

Remembering my local history

https://en.wikipedia.org/wiki/Hightstown_rail_accident

FLaNK / KNIFe AI Weekly

http://knifeai.org/

Tim Spann @PaaSDev

https://pebble.is/PaaSDev

https://vimeo.com/flankstack

https://www.youtube.com/@FLaNK-Stack

https://www.threads.net/@tspannhw

https://medium.com/@tspann/subscribe

https://www.cloudera.com/campaign/apache-nifi-for-dummies.html

https://ossinsight.io/analyze/tspannhw

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

**This is Issue #133 **

https://github.com/tspannhw/FLiPStackWeekly

https://www.cloudera.com/solutions/dim-developer.html

New Releases

Articles

Videos

Building Real-Time Pipelines XTremeJ https://www.youtube.com/watch?v=SszeF57IdW4

Adding Generative AI to Real-Time Streaming Pipelines | Tim Spann | Conf42 LLMs 2024 https://www.youtube.com/watch?v=Yeua8NlzQ3Y

Meetup Talk NYC https://youtu.be/u8XNNEPEnKQ?si=VWe6n8OKOF7qk6Fl

https://www.conf42.com/Large_Language_Models_LLMs_2024_Tim_Spann_generative_ai_streaming

https://www.youtube.com/watch?v=fciswxMLfIM&ab_channel=DATAMINER

https://www.youtube.com/watch?v=y6kbRZae4TI&ab_channel=DATAMINER

Slides

Events

April 24/25, 2024: Cloudera. Virtual. EMEA | APAC: April 24, 2024 9:30 AM CEST | 1:00 PM IST AMER EVENT: Apr 25, 2024 9:00 AM PDT | 12:00 PM EDT Register Now: http://spr.ly/6047Z3AjN

May 1, 2024: Gen AI in the Enterprise Cloud. Virtual. https://www.linkedin.com/events/7180985346103410688/comments/

May 8-9, 2024: Data Summit 2024. Boston, MA. https://www.dbta.com/DataSummit/2024/default.aspx https://www.dbta.com/DataSummit/2024/Timothy-Spann.aspx

https://twitter.com/DBTADataSummit/status/1778393005646397636

May 21, 2024: Gen AI and Beyond with NiFi 2.0. Virtual.

June 12, 2024: Budapest Data + ML Forum. Virtual.  https://budapestdata.hu/2024/en/

Cloudera Events https://www.cloudera.com/about/events.html

https://www.cloudera.com/events/cloudera-now-cdp.html?internal_keyplay=ALL&internal_campaign=FY25-Q1-AMER-WS-Cloudera-Now-Events-Page-P06&cid=701Hr000000tW6qIAE&internal_link=p06

More Events: https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe

Code

• https://github.com/tspannhw/FLaNK-python-processors

Models

Tools

• https://github.com/owainlewis/awesome-artificial-intelligence
• https://github.com/cleanlab/cleanlab
• https://github.com/dair-ai/Prompt-Engineering-Guide
• https://www.youtube.com/watch?v=hC_qASRcBhU
• https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1
• https://datavolo.io/2024/02/collecting-logs-with-apache-nifi-and-opentelemetry/
• https://www.jaided.ai/easyocr/tutorial/
• https://ohmygit.org/
• https://www.beagleboard.org/boards/beagley-ai
• https://github.com/AIRI-Institute/OmniFusion

New

Retro Tips

Discount

Discount access to DataSummit 2024 https://secure.infotoday.com/RegForms/DataSummit/?Priority=24SPKR

© 2020-2024 Tim Spann

Data Warehouse Ingestion Patterns with Apache NiFi

Adding Generative AI to Real-Time Streaming Pipelines | Tim Spann | Conf...

Building Real-time Pipelines with FLaNK: A Case Study with Transit Data ...

FLaNK Ai - 08 April 2024

 

08-April-2024

Sorry for the delay I was travelling to Seattle for NLIT and also eclipse.

FLaNK / KNIFe AI Weekly

http://knifeai.org/

Tim Spann @PaaSDev

https://pebble.is/PaaSDev

https://vimeo.com/flankstack

https://www.youtube.com/@FLaNK-Stack

https://www.threads.net/@tspannhw

https://medium.com/@tspann/subscribe

https://www.cloudera.com/campaign/apache-nifi-for-dummies.html

https://ossinsight.io/analyze/tspannhw

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

**This is Issue #132 **

https://github.com/tspannhw/FLiPStackWeekly

https://www.cloudera.com/solutions/dim-developer.html

New Releases

Articles

Meetup Report https://medium.com/@tspann/march-2024-meetup-report-61e82b00cf57

Real-Time Irish Transit Analytics https://medium.com/@tspann/real-time-irish-transit-analytics-ea76164c9595

Adding Generative AI Results to SQL Streams https://medium.com/@tspann/adding-generative-ai-results-to-sql-streams-513e1fd2a6af

Getting Started with Autogen https://newsletter.victordibia.com/p/getting-started-with-autogen-a-framework

WatsonX with Milvus https://ruslanmv.com/blog/WatsonX-Assistant-with-Milvus-as-Vector-Database

Visualize Your Rag https://towardsdatascience.com/visualize-your-rag-data-evaluate-your-retrieval-augmented-generation-system-with-ragas-fc2486308557

AI + Energy https://www.linkedin.com/pulse/revolutionizing-energy-utilities-industry-unleashing-power-dilshan-62ogc/?trackingId=1ZW41ARoQ9I66w%2BrMs643w%3D%3D

Deep Dive into Vector Databases https://towardsdatascience.com/deep-dive-into-vector-databases-by-hand-e9ab71f54f80

Speak Diarization https://haystack.deepset.ai/blog/level-up-rag-with-speaker-diarization?

Customer Segmentation https://towardsdatascience.com/mastering-customer-segmentation-with-llm-3d9008235f41#3a33

More Billions into AI https://www.datanami.com/2024/03/28/amazon-invests-another-2-75-billion-into-anthropic/

Debezium and Kafka https://medium.com/appcent/debezium-and-kafka-connector-cdc-e0c61b1e1027

PDF https://simonwillison.net/2024/Mar/30/ocr-pdfs-images/

LLAMA on CPU https://justine.lol/matmul/

JAMBA https://www.ai21.com/blog/announcing-jamba

LLM with Long Context https://huggingface.co/papers/2404.02060

Cybersercurity Miss https://www.reuters.com/technology/cybersecurity/why-near-miss-cyberattack-put-us-officials-tech-industry-edge-2024-04-05/

Text to SQL Pinterest https://medium.com/pinterest-engineering/how-we-built-text-to-sql-at-pinterest-30bad30dabff

Videos

Meetup Talk NYC https://youtu.be/u8XNNEPEnKQ?si=VWe6n8OKOF7qk6Fl

Irish Rail Preview https://youtu.be/EIpH7RPO2Yo

TCF Pro 2024 https://www.youtube.com/watch?v=tLbdrOxg5Rs

Slides

https://www.slideshare.net/slideshow/2024-xtremej-building-realtime-pipelines-with-flank-a-case-study-with-transit-data/267065123

Events

April 8-11, 2024: NLIT Summit. Seattle. https://www.fbcinc.com/e/nlit/default.aspx 

April 11, 2024: Conf42 LLM. Virtual. https://www.conf42.com/llms2024

April 12, 2024: AI Max Conference. 23 Orchard Princeton https://www.startupgrind.com/events/details/startup-grind-princeton-presents-startup-grind-hosts-ai-max-summit/

April 2024: AI Meetup NJ https://www.meetup.com/nj-gai/

April 24/25, 2024: Cloudera. Virtual. EMEA | APAC: April 24, 2024 9:30 AM CEST | 1:00 PM IST AMER EVENT: Apr 25, 2024 9:00 AM PDT | 12:00 PM EDT Register Now: http://spr.ly/6047Z3AjN

May 1, 2024: Gen AI in the Enterprise Cloud. Virtual. https://www.linkedin.com/events/7180985346103410688/comments/

May 8-9, 2024: Data Summit 2024. Boston, MA. https://www.dbta.com/DataSummit/2024/default.aspx https://www.dbta.com/DataSummit/2024/Timothy-Spann.aspx

May 21, 2024: Gen AI and Beyond with NiFi 2.0. Virtual.

June 12, 2024: Budapest Data + ML Forum. Virtual.  https://budapestdata.hu/2024/en/

Cloudera Events https://www.cloudera.com/about/events.html

https://www.cloudera.com/events/cloudera-now-cdp.html?internal_keyplay=ALL&internal_campaign=FY25-Q1-AMER-WS-Cloudera-Now-Events-Page-P06&cid=701Hr000000tW6qIAE&internal_link=p06

More Events: https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe

Code

• https://github.com/tspannhw/FLaNK-python-processors

Models

• https://github.com/janhq/jan

Milvus Event

https://colab.research.google.com/drive/1U2dGpcn56a9AmAl3Ci4ZOTPGKqxICjGM?usp=sharing https://github.com/mlfoundations/open_clip https://huggingface.co/models?library=open_clip https://huggingface.co/adept/fuyu-8b https://zilliz.com/blog/exploring-multimodal-embeddings-with-fiftyone-and-milvus https://medium.com/voxel51/a-google-search-experience-for-computer-vision-data-voxel51-a9ee41390986#:~:text=Find%20video%20frames%20with%20cars%20in%20an%20intersection

Tools

• https://erichartford.com/uncensored-models
• https://github.com/jasonppy/VoiceCraft
• https://github.com/infiniflow/ragflow
• https://www.upscayl.org/
• https://www.helixnj.com/
• https://www.gbstudio.dev/
• https://github.com/indi4u/LLM/blob/main/using-Milvus-vector-db.ipynb
• https://www.airegex.pro/
• https://github.com/zilliztech/spark-milvus
• https://github.com/chiasmod0n/chiasmodon
• https://github.com/tamilselvanarjun/pydatascraper
• https://github.com/princeton-nlp/SWE-agent
• https://github.com/dvlab-research/MiniGemini
• https://www.thoughtworks.com/radar
• https://github.com/Renumics/renumics-rag/blob/main/notebooks/visualize_rag_tutorial_qs.ipynb
• https://www.assemblyai.com/docs
• https://editor.swagger.io/
• https://mattturck.com/mad2024/
• https://github.com/facebookresearch/nougat
• https://github.com/plandex-ai/plandex
• https://github.com/DAGWorks-Inc/burr
• https://github.com/YuelangX/Gaussian-Head-Avatar
• https://github.com/ftisiot/postgresql-ai-projects
• https://github.com/langchain-ai/rag-from-scratch
• https://github.com/OwlAIProject/Owl
• https://github.com/openscilab/nava
• https://github.com/heyform/heyform
• https://github.com/Stirling-Tools/Stirling-PDF
• https://github.com/charmbracelet/freeze
• https://github.com/katanaml/sparrow
• https://github.com/katanaml/sparrow/tree/main/sparrow-data/ocr
• https://github.com/KdaiP/StableTTS
• https://github.com/QwenLM/Qwen1.5
• https://github.com/pinterest/querybook
• https://github.com/drawdb-io/drawdb
• https://github.com/Libr-AI/OpenFactVerification
• https://tokyochallenge.odpt.org/en/index.html
• https://github.com/Azure/AI-in-a-Box
• https://hackernoon.com/how-colbert-helps-developers-overcome-the-limits-of-rag?utm_source=hootsuite

New

Advanced Python Library Installer (think Cargo) https://astral.sh/blog/uv

Retro Tips

Motion on RPI is useful, probably want to send a HTTP or Kafka message https://github.com/tspannhw/leprechaun-detector https://www.datainmotion.dev/2019/03/simple-leprechaun-detector-and-then-how.html

Discount

Discount access to DataSummit 2024 https://secure.infotoday.com/RegForms/DataSummit/?Priority=24SPKR

© 2020-2024 Tim Spann