kafka를 만진 메모

kafka란? 특징은?

kafka의 시스템 레벨 이벤트 구동 개발 이미지

시스템을 설정할 때의 유의점

docker로 Kafka server 만들기

docker-compose -f docker-compose-single-broker.yml up

rails + karafka로 메시지 교환

rails new karafka_example

gem 'karafka'

bundle exec karafka install

app/consumers/application_consumer.rb
app/responders/application_responder.rb
karafka.rb

ENV['RAILS_ENV'] ||= 'development'
ENV['KARAFKA_ENV'] = ENV['RAILS_ENV']
require ::File.expand_path('../config/environment', __FILE__)
Rails.application.eager_load!

class KarafkaApp < Karafka::App
  setup do |config|
    config.kafka.seed_brokers = %w[kafka://127.0.0.1:9092]
    config.client_id = 'example_app'
    config.backend = :inline
    config.batch_fetching = true
    config.logger = Rails.logger
  end

  Karafka.monitor.subscribe(Karafka::Instrumentation::Listener)

  consumer_groups.draw do
    consumer_group :bigger_group do
      batch_fetching false

      topic :users do
        consumer UsersConsumer
      end
    end
  end
end

KarafkaApp.boot!

class UsersConsumer < ApplicationConsumer
  def consume
    Karafka.logger.info "New [User] event: #{params}"
  end
end

class UsersResponder < ApplicationResponder
  topic :users

  def respond(event_payload)
    respond_to :users, event_payload
  end
end

bundle exec karafka server

UsersResponder.call({ event_name: "user_created", payload: { id: 1 } }

관련 자료