A game changing announcement, not for just Snowflake users and considerers, but also the Data Engineering industry as whole, has arrived. Ascend’s unification of data ingestion, transformation, observability, delivery, and orchestration into a single platform along with Snowflake’s blistering speed means that data teams can deliver value to the business 10x faster with 95% less code and 90% less maintenance. We were lucky enough to participate in the private beta and here’s what we found by putting the integration through its paces...
Data Engineers are typically spending too much time supporting the infrastructure and not enough time delivering value. Without observability to trace root cause, they’re spending considerable time troubleshooting their pipelines, there is duplication in what’s being created, and they’re using separate tools for ingest and transforms, which creates context switching and slows them down. What they need are good tools that help them automate operations and reduce risk.
While Snowflake is easy to use and super fast, some customers build a lot of Tables, Views and Tasks within the warehouse, and before too long they lose track of the lineage of their pipelines, and support and enhancement become time consuming and risky. Implementing CI/CD pipelines in Snowflake is difficult to do well without a governing layer that manages which objects are automatically deployed to each environment.
Instead, Ascend's “single pane of glass” now allows our data teams to see all ingestions and transformations, their lineage and status all in one place. This means that instead of trying to remember which downstream transformations would be impacted by a change we can now see them on the screen. Our Data Engineers were amazed with the speed they could create new pipelines, and the majority of their time was spent crafting transformation queries in Ascend rather than performing manual infrastructure tasks. Other benefits include:
Transformations can be declarative. All inserts, updates and deletes are taken care of by the Ascend platform, freeing Data Engineers to solve more complex problems.
Data Engineers can choose to use the web interface, or declare transformations in code.
Ascend's Data Feeds allows Data Engineers to write a transformation once and re-use it in other downstream pipelines, with all downstream pipelines automatically receiving new data when the published Data Feed receives new data.
Want to know more about how this integration can add value to your investment in data? Click the "BOOK A CONSULTATION" link and we can take you through a demo.