In consumer mode, Kafkacat reads messages from a topic and prints them to standard output (stdout). You can also redirect it to a file (i.e. JSON) This means that you can save all the data collected right before you start a simulated test from a Kafka broker. You can stop the consumption when you are done performing the simulated test. You can just grab the logs from this repo and re-play them as if they were being ingested in real-time.
Kafka Broker : A distributed publish-subscribe messaging system that is designed to be fast, scalable, fault-tolerant, and durable (Installed by HELK).
Install Kafkacat following the instructions from the official Kafkacat repo
If you are using a debian-based system, make sure you install the latest Kafkacat deb package.
You can also install it from source following the Quick Build instructions.
Export Security Events¶
Consume data being produced from a kafka broker with the following flags:
-b: Kafka Broker
-t: Topic in the Kafka Broker to consume the data from
-C: Consumer mode
-o: Offset to start consuming from (i.e. end)
$ kafkacat -b <Kafka-Broker-IP>:9092 -t winlogbeat -C -o end > empire_dcsync_$(date +%F%H%M%S).json
That’s it! You now can share that dataset with the community!