In consumer mode, Kafkacat reads messages from a topic and prints them to standard output (stdout). You can also redirect it to a file (i.e. JSON) This means that you can save all the data collected right before you start a simulated test from a Kafka broker. You can stop the consumption when you are done performing the simulated test. You can just grab the logs from this repo and re-play them as if they were being ingested in real-time.
Kafka Broker : A distributed publish-subscribe messaging system that is designed to be fast, scalable, fault-tolerant, and durable (Installed by HELK).
Install Kafkacat following the instructions from the official Kafkacat repo
If you are using a debian-based system, make sure you install the latest Kafkacat deb package.
You can also install it from source following the Quick Build instructions.
Export Security Events#
Consume data being produced from a kafka broker with the following flags:
-b: Kafka Broker
-t: Topic in the Kafka Broker to consume the data from
-C: Consumer mode
-o: Offset to start consuming from (i.e. end)
$ kafkacat -b <Kafka-Broker-IP>:9092 -t winlogbeat -C -o end > empire_dcsync_$(date +%F%H%M%S).json
That’s it! You now can share that dataset with the community!