Data Infrastructure in Databricks

Why?
In order to productionize our data from our different systems, we need to have a common data infrastructure in place. The infrastructure needs to be capable of handling all types of data in a governable and consistent manner, so that we can better utlize our data.

How?
By using the Databricks architecture (lake, sql, layering and metadata) with Unity catalogue as a managing tool, and ingesting data from our different system into the data infrastructure.

What? - Definition of Done:
At the end of the work we will be able to handle and scale multi-tenancy of systems into our data lakehouse. We will be able to access it in a consistent and governed manner, with scalability and history that is not provided through the data warehouse in MSQL today.