Apache Kafka (Confluent) - hands-on workshop - Part 2

Schedule

Prerequisites:

• Intellij Idea community edition
• Gradle
• Java 11
• WSL (https://learn.microsoft.com/en-us/windows/wsl/install)
• Apache Kafka binaries
• Confluent cloud account (Free credits available)

Speakers:

• Lakshmi Narasimhan
  Principal Engineer at Platformatory who always loves data engineering at its core. Currently working on different projects around ETL, and event streaming mainly around Apache Kafka - https://www.linkedin.com/in/lakshminp/
• Ashwin Venkatesan
  Senior Consultant at Platformatory working mainly on Apache Kafka and Kubernetes. Ashwin has done multiple projects building scalable data pipelines incorporating Apache Kafka in production scenarios. He also dabbles in Data science and Machine learning in his free time. His hobbies include traveling, cooking, and following Manchester United : )

Agenda: Kafka workshop - 2

• Brief recap of what was covered in the previous workshop:
  • Basics of Apache Kafka: the difference between Kafka and database/message queue/ service bus etc.
  • Components of Kafka cluster
  • Topics, partitions
  • Producers, consumers
• Messages schema
  • Serialization, different serialization formats
  • Schema registry
  • Example producer and consumer with schema
• Kafka Connect
  • Overview of Kafka connect => source and sink connectors, architecture.
  • Examples of source and sink connectors, introduction to connector hub.
  • Examples of Microsoft-specific source and sink connectors.
• Kafka streams
  • Introduction to Kafka streams
  • Example of a Kafka streams application followed by demo
• KSQL
  • KSQL intro and architecture
  • Difference between Kafka streams and KSQL
  • Example application + demo
• Kafka security
  • Different authentication options
  • Authorization
  • Quotas
• Questions / Networking (Lunch will be provided)