Recently I ran into some issues in consuming Kafka AVRO messages. Everything went well up to the de-serialization of the record from the Kafka topic. It all just went crazy once it was being serialized into JSON. That’s pretty much where the proverbial buck stopped.
I was scratching my head on why this odd behavior was happening. Looking at the message published on the Kafka topic, it was following the defined AVRO schema as expected. There were no deviations.
Somehow when being converted to JSON it started throwing an exception. It was always at the same spot every time. It would not go any further.
Debugging the flow of the program, it showed me that what should have been this:
And that is why the program was throwing an error!
The solution I did was to remove null as a field type. Set the field to be strictly boolean. What I wanted was to initially have the state of this field as null. Then further down the line it can be set to either true or false.
After the AVRO schema was updated, the error went away.