"The value didn't justify the effort."
A Sr. Data Engineer at Swedish MediaTech after their Datafold PoC.

❌ Heavy setup → Noisy results → Alert fatigue → No way to start small
Comparison: https://datarecce.io/blog/recce-vs-datafold/

#DataEngineering #DataValidation #analyticsengineering

Don’t start with what changed. Start with what SHOULD change!

Because not every diff is a problem, and not every problem shows up as a diff.

👉 https://datarecce.io/blog/more-than-data-diff/

#dataengineering #datadiff #analyticsengineering #datavalidation

Want to better understand how your data models work, and what might break when they change?

Here are 5 types of column transformations in #dbt models:
1. Pass-through
2. Renamed
3. Derived
4. Source
5. Unknown

Each one helps you assess impact and trace data through your pipeline more clearly

We use these types in Recce to power column-level lineage and breaking change analysis

Read the deep dive
https://datarecce.io/blog/column-level-lineage-internals/

#SQL #Data #OpenSource #DataEngineering #AnalyticsEngineering

Stop duplicating dashboards to preview the impact of dbt model changes!

With Recce, you can:
✅ Diff models
✅ Record data impact
✅ Share results instantly — no dashboards, no SQL, no screenshots

One click. Instant clarity:

https://medium.com/inthepipeline/stop-duplicating-dashboards-just-to-preview-dbt-changes-21ff0ba8dd21

#dbt #DataOps #SQL #data #analyticsengineering #DataEngineering

GenAI just got real for analytics engineers. See how dbt Copilot is changing workflows in our latest blog👇

https://dataroots.ghost.io/genai-is-taking-over-the-modern-data-stack-and-dbt-copilot-is-leading-the-way-in-analytics-engineering-2/

#GenAI #dbt #AnalyticsEngineering #DataStack

Column-level lineage is now available in Recce 0.57

Add it to your dbt data validation workflow:

1. Lineage Diff - Focus on impacted models
2. Breaking Change Analysis - Eliminate irrelevant changes
3. Column-Level Lineage - Track column evolution

#dbt #DataEngineering #Data #SQL #Analytics #AnalyticsEngineering

❌ No more manually cross-referencing dbt docs from dev and prod

❌ No more manually checking schemas in your data warehouse

❌ No more manually comparing row-counts on models.

See a trend?

Read how an experienced data professional validates zero regression on a #dbt PR:

https://www.linkedin.com/posts/abdelm_recce-dbt-activity-7300436694808358914-aKiF?rcm=ACoAAAVLBgkB6r61wCsOcKgFDrtf_EEEe3UjdXs

#DataEngineering #dbt #Data #Analytics #AnalyticsEngineering #SQL #BigQuery #OpenSource

🚀 Started Module 4 of #DataEngineering Zoomcamp!
Just kicked off the Analytics Engineering module and I'm diving into transforming the Green Taxi, Yellow Taxi, and FHV NY Taxi datasets loaded in #BigQuery. Excited to see how dbt can help create analytical views for better decision-making! #dbt #DataTalksClub #GCP #AnalyticsEngineering #ETL

It's not enough to rely solely on running dbt tests in CI.

dbt tests can't cover every situation because, by nature, they can only be written for issues you can foresee.

The only way to check for data impact is to compare current data with historical data

Read Tim Hiebenthal's post "The current gaps in (your) dbt-tests":

https://handsondata.substack.com/p/the-current-gaps-in-your-dbt-tests

#dbt #DataEngineering #data #sql #analytics #AnalyticsEngineering

You don’t want to backtrack through time looking for the point that bad data was merged into your dbt project!

Use Recce for QA *before* merging to prod and get full insight into data impact.

Get open-source Recce now:
https://github.com/datarecce/recce

#data #analytics #dbt #DataEngineering #AnalyticsEngineering

Recce Sandbox is now live!

Think dbt Analyses but with query diff for iterating on a model without materializing or modifying your #dbt project

$ pip install -U recce
$ recce server

Start playing with data model code and previewing data impact in the Sandbox, risk-free

https://medium.com/inthepipeline/preview-dbt-data-model-changes-with-risk-free-data-diff-in-recce-sandbox-8bc030843c25

#dataengineering #dbt #data #analytics #analyticsengineering #impactassessment #recce

✳️ dbt Best Practices in Action ✳️

Optimizing your dbt workflows is essential for scalable, reliable data pipelines.

See how California Integrated Travel Project (Cal-ITP) used dbt best practices to enhance their data infrastructure:

https://medium.com/inthepipeline/dbt-best-practices-in-action-at-cal-itps-data-infra-project-0d11adf5513d

What’s the biggest challenge your team faces when implementing dbt?

#dbt #DataEngineering #OpenSource #analytics #AnalyticsEngineering

Live preview dbt data model changes without needing to rebuild the model!

Do it right now in Recce, here’s how

https://www.loom.com/share/3f59853e11b6401fa0c7e93714302f62

1. Edit model code
2. Diff with the original code
3. Instantly compare the data (no need to drop into dbt and rebuild)

#data #analytics #dbt #DataEngineering #AnalyticsEngineering

Data impact analysis on multiple dbt nodes at the same time

Row count, schema, and col match percentage

- Select nodes w/ #dbt selection syntax
- Manually select nodes
- Save and re-run checks
- Automate checks in CI

Get Recce now:
https://github.com/datarecce/recce

#OpenSource #DataEngineering #Analytics #Data #AnalyticsEngineering

Analytics engineers don't work alone - data validation involves stakeholders and PR reviewers

With Recce, Share your complete data validation environment with your team and review data impact together.

- Validate your work
- Create your checklist
- Share it with your team

https://medium.com/inthepipeline/enhanced-dbt-pr-review-with-reproducible-data-validation-environments-e6c37a15908f

#data #dbt #DataEngineering #AnalyticsEngineering

If you’re new to Data Recce here’s a tip:

Copy a screenshot of any diff to the clipboard with one click

Perfect for pasting into your GitHub PR comment!

Give it a try on your next PR when you want to:

- show the reviewer your work is correct
- share results for discussion
https://giphy.com/gifs/github-clipboard-recce-b4vDoxUZtWwtrzgaoC?tc=1

#dbt #Data #Analytics #DataEngineering #AnalyticsEngineering #tips #DataImpactAnalysis

Coalesce 2024 keynote announcements included among others:
* support for Apache Iceberg
* a DBT Copilot
* integrations with Power BI
* improved data quality and data freshness tools
* More visual editing support
* in general the vibe of making it easier to reference data across teams

#Coalesce #DBT #AnalyticsEngineering #Coalesce2024 #Data

https://coalesce.getdbt.com/

Just been doing a LinkedIn Learning course on dbt - a tool used in Analytics Engineering. Delighted that they were using DuckDB as the backend.
This kind of use is so huge for learners. Instead of faffing around to get credentials/access to a remote DB server just run locally for free and quickly get on with the learning of the app.

#DuckDB #DBT #AnalyticsEngineering #DB
https://www.linkedin.com/learning/data-engineering-with-dbt?trk=share_android_course_learning&shareId=%2Fejw28EGQSWFfcwj6qTIUg%3D%3D

What comes after the chaos of self-service analytics?

Embrace the chance to elevate the importance of data and your team

https://medium.com/inthepipeline/what-comes-after-the-chaos-of-self-service-analytics-f0e09c0d7b40

#dbt #DataEngineering #DataOps
#analytics #AnalyticsEngineering #Data

I'm seeking a new challenge in a Data Engineer or Analytics Engineer role.

Yes, I know that I might not have the exact skillset you're looking for & that my background might not be what you expect. However, I can & will learn anything that I'm missing, which is something I'd love to do in a established Data Team.

1/3

#DataEngineering #AnalyticsEngineering #DataEngineer #AnalyticsEngineer #SQL #Data #Database #DataWarehouse #JobSeeking #JobHunt #AvailableForWork #GetFediHired