[meetup] Data engineering and devops
Marcel-Jan transformed from Oracle guy to Hadoop specialist, Python fan and adopter of deep neural networks. During this meetup, he will share his experiences in becoming an Hadoop specialist. Thiago, squad lead at LINKIT, will share the pitfalls of building your data pipeline in a cloud world. A cool and informative meetup on various aspects of Data Engineering.
18:00 – Doors open. Food & drinks provided
19:00 – Marcel-Jan Krijgsman - Becoming a Hadoop Specialist
19:45 – Break
20:00 – Thiago de Faria - Native storage, ephemeral clusters and the pitfalls of building your data pipeline in a cloud world
20:45 – Pool, laughing, networking and drinks until someone kicks us out!
Title: Becoming a Hadoop Specialist
Description: How do you learn to become data engineering after 20 years of building up expertise in another field? And will anybody want to hire you? Marcel-Jan Krijgsman will tell you how he switched to data engineering by learning MOOCs (Massive Open Online Courses), visiting Meetups and conferences, listening podcasts, reading online documentation and books, watching videos and just trying stuff out on virtual machines. And not to forget: sharing your knowledge. As will happen in this Meetup.
Marcel-Jan will also share the products, concepts and languages he has learned about: Hadoop, MongoDB, Elasticsearch, Python, Deep Neural Networks, machine learning in general and many other things he learned along the way. What is cool about them and how can you learn quickly about them? It will be an action packed session.
About Marcel-Jan Krijgsman - Data Engineer/Data custodian at Port of Rotterdam:
For years Marcel-Jan Krijgsman the goto guy for Oracle performance issues and security knowledge. But in 2017, after amassing 20 years of Oracle experience, he decided it was time for a change. He wanted to go into Big Data and he started learning many MOOCs (Massive Open Online Courses). He became Hadoop specialist, Python fan and learned deep neural networks. When he started working as a real data engineer, he decided to make short Youtube videos of the things he learned about open source technology. They are meant to show why these products are cool for anyone who doesn't have the time to study them themselves. Recently he got his hands on a time crystal and started learning at a superhuman rate (since then he is disqualified from cycling races).
Title: Native storage, ephemeral clusters and the pitfalls of building your data pipeline in a cloud world
Description: You current company announced its cloud strategy, and it wants to go full-mode cloud! Tears of joy erupt around you... the "devops team" (whatever that means) feels accomplished and are not ashamed anymore to say to friends that they work on rack maintenance, and a huge Press Release is made. You hear the usual names and phrases: kubernetes, cloud-native, agile, agility, devops, culture change and all. The following week arrives, and you are the responsible for the migration of the data pipelines... what is there for you? What does it change? Does ETL still apply? Should I use native storage or build my own HDFS cluster? Pre-built tools or spawn instances and rebuild my infra? On this talk, we will discuss some of these potential problems and share failed decisions that I have taken, what I learned and share the one answer that annoys CTO's when they ask me what to do: "it depends." :-)
About Thiago de Faria - DataOps Lead at LINKIT:
Thiago started in Pure Mathematics, jumped to predicting things and realized it was utterly dependent on understanding the inner bits of software development and data pipelines. Thus, his primary concern changed: how to bridge the gap between ML and production? :-) *DataOps FTW!* He is an active part of the community (devopsdays Amsterdam, ITNEXT, amsterdam.ai & Codemotion), a knowledge-sharer, a proud father and a DataOps Lead. Thiago is pathologically curious and a continuous learner. He knows that high-performing data teams must decrease time-to-market and build production-ready applications, always!