I’ve worked on a lot of warehouses using a number of different ETL tools. Most of these tools make my job as technical lead very difficult. They are opaque: they are difficult to test and hard to review. Ensuring consistency requires an enormous amount of effort.
I’m not the only one who has noticed these qualities, so all of the better ETL tools can be automated with scripts.
Read the rest on the Optimal BI blog.