Guide through Kafka Connector, on how to migrate your streaming to Snowflake
Companies need to work on a non-disruptive migration that allows to simplify and ease the process of real-time ingestions, snowflake has easy integration with Apache Kafka.
Customers using the publish/subscribe Kafka platform can now push data directly into the snowflake tables using Kafka Connector with a minimal effort.
A simple path would be one table for each topic, with messages containing one row each, the table will contain two semi-structured columns ( Called VARIANT in snowflake ) which are RECORD_METADATA and RECORD_CONTENT, the content could be either JSON or Avro.
Tables can be automatically created by the Kafka connector, or if the user has a predefined table, the other columns apart from those two mentioned must be nullable.
A small schema below to illustrate the flow post-migration