Skip to main content

Schema Migration

As you change your data definition, Causal automatically keeps your data warehouse up to date. As described in Making a Change, you can add, remove, and change the data type of columns in the data warehouse.

When you update a warehouse view using the Causal compiler, it will automatically calculate the schema of your tables given the current data stored in the warehouse. If there is a partition that does not contain a column (perhaps in partitions before you introduced the data), the column will contain nulls. Old data that you are no longer collecting will not appear in the view, insuring that your data scientists will always be looking at data that is accurate and maintained.1

Causal's FDL compiler will signal an error if a developer tries to make a breaking change to the data warehouse. That way, you should be able to work with older data in your tables, even as the data that you collect changes over time. However, it is possible to force a breaking change into the system using the --no-compatibility-check command line argument. We strongly recommend that you only use this option in development environments. If a breaking change is introduced anyway, Causal will drop the incompatible partitions from the warehouse views.

  1. If you ever want to go back and see this data, you can use your revision control system to check out an old version of the FDL file and re-generate that view.↩