#dataengineering

135 posts

@thomas_larock The Long Way 'Round #DataEngineering #Analytics #SQL
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/thomas_larock/posts/22460840
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/thomas_larock/posts/22460840
Don't have an account? Find a server at joinmastodon.org
May 11, 2026
@slack_engineering From SSH to REST: A Security-Driven Modernization of Slack's EMR Data Pipelines #DataEngineering #Security #Cloud
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/slack_engineering/posts/22293286
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/slack_engineering/posts/22293286
Don't have an account? Find a server at joinmastodon.org
May 5, 2026
@spotify_engineering Background Coding Agents: Supercharging Downstream Consumer Dataset Migrations (Honk, Part 4) #AI #DataEngineering #SoftwareDevelopment
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/spotify_engineering/posts/22168450
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/spotify_engineering/posts/22168450
Don't have an account? Find a server at joinmastodon.org
Apr 22, 2026
@data_methods Data Role Compression #Data #Analytics #DataEngineering
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414018
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414018
Don't have an account? Find a server at joinmastodon.org
Apr 22, 2026
@nic_crane How I Used Claude Code to Implement {dplyr} Functions in {arrow} #DataEngineering #AI #OpenSource
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/nic_crane/posts/22442030
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/nic_crane/posts/22442030
Don't have an account? Find a server at joinmastodon.org
Apr 9, 2026
@thomas_larock Fail Better, Then Finish 21st #DataEngineering #Analytics #Technology
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/thomas_larock/posts/22460848
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/thomas_larock/posts/22460848
Don't have an account? Find a server at joinmastodon.org
Apr 9, 2026
@philip_heltweg Optimization Opportunities for Cloud-Based Data Pipeline Infrastructures #Cloud #DataEngineering #Optimization
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/philip_heltweg/posts/22426337
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/philip_heltweg/posts/22426337
Don't have an account? Find a server at joinmastodon.org
Apr 3, 2026
@data_methods Learning About Databricks Coming From Snowflake #DataEngineering #CloudData #Analytics
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414029
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414029
Don't have an account? Find a server at joinmastodon.org
Mar 21, 2026
@nic_crane LLM-Assisted Issue Triage for Open Source Maintainers #OpenSource #AI #DataEngineering
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/nic_crane/posts/22442040
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/nic_crane/posts/22442040
Don't have an account? Find a server at joinmastodon.org
Mar 6, 2026
@data_methods DataSpec: Open Source Data Infrastructure #DataEngineering #OpenSource #DataInfrastructure
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414049
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414049
Don't have an account? Find a server at joinmastodon.org
Feb 9, 2026
@rsyslog What rsyslog is today: a high-performance log ingestion and ETL engine #Logging #DataEngineering #SysAdmin
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/rsyslog/posts/22393405
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/rsyslog/posts/22393405
Don't have an account? Find a server at joinmastodon.org
Feb 8, 2026
@sourcegraph_blog Building DataBot: Our always-on data assistant #DataEngineering #AI #SoftwareDevelopment
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/sourcegraph_blog/posts/22376087
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/sourcegraph_blog/posts/22376087
Don't have an account? Find a server at joinmastodon.org
Feb 6, 2026
@nic_crane AI Tooling and Open Source #OpenSource #AI #DataEngineering
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/nic_crane/posts/22442050
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/nic_crane/posts/22442050
Don't have an account? Find a server at joinmastodon.org
Jan 27, 2026
@data_methods Building a Clean API on Top of Texas Railroad Commission Permit Data #DataEngineering #API #SQL
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414058
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414058
Don't have an account? Find a server at joinmastodon.org
Jan 2, 2026
@data_methods Querying 160 GB of Parquet Files with DuckDB in 15 Minutes #DataEngineering #SQL #Analytics
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414066
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414066
Don't have an account? Find a server at joinmastodon.org
Dec 24, 2025
@data_methods DuckPond: Building a Self-Hosted Multi-Tenant Query Layer with DuckDB, DuckLake, FastAPI, and React #DataEngineering #SQL #OpenSource
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414077
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414077
Don't have an account? Find a server at joinmastodon.org
Dec 8, 2025
@supabase_blog Introducing iceberg-js: A JavaScript Client for Apache Iceberg #OpenSource #DataEngineering #JavaScript
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/supabase_blog/posts/22458790
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/supabase_blog/posts/22458790
Don't have an account? Find a server at joinmastodon.org
Dec 8, 2025
@data_methods Testing Out Time Travel With DuckLake #DataEngineering #SQL #Analytics
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414087
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/data_methods/posts/22414087
Don't have an account? Find a server at joinmastodon.org
Nov 21, 2025
@jack_vanlightly How Would You Like Your Iceberg Sir? Stream or Batch Ordered? #DataEngineering #DistributedSystems #Streaming
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/jack_vanlightly/posts/22429749
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/jack_vanlightly/posts/22429749
Don't have an account? Find a server at joinmastodon.org
Nov 5, 2025
@ngrok_news How Rinsed builds data pipelines in car washes: the lessons learned #DataEngineering #Cloud #DevOps
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/ngrok_news/posts/22442204
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/ngrok_news/posts/22442204
Don't have an account? Find a server at joinmastodon.org
Oct 21, 2025
@turbopuffer_blog Economical way of serving vector search workloads #Search #VectorSearch #DataEngineering
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/turbopuffer_blog/posts/22448573
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/turbopuffer_blog/posts/22448573
Don't have an account? Find a server at joinmastodon.org
Sep 18, 2025
@philip_heltweg Open Collaborative Data Engineering With Subject-Matter Experts Using Domain-Specific Languages #DataEngineering #OpenSource #AI
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/philip_heltweg/posts/22426388
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/philip_heltweg/posts/22426388
Don't have an account? Find a server at joinmastodon.org
Sep 16, 2025
@infocaptor_ai MCP vs RAG: A Practical Guide for Data Engineers and AI Builders #DataEngineering #AI #BusinessIntelligence
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/infocaptor_ai/posts/22428427
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/infocaptor_ai/posts/22428427
Don't have an account? Find a server at joinmastodon.org
Sep 3, 2025
@supabase_blog Supabase Analytics Buckets with Iceberg Support #DataEngineering #PostgreSQL #Cloud
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/supabase_blog/posts/22459142
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/supabase_blog/posts/22459142
Don't have an account? Find a server at joinmastodon.org
Jul 15, 2025
@mariano_gappas_blog How We Handle Billion-Row ClickHouse Inserts With UUID Range Bucketing #DataEngineering #ClickHouse #BigData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/mariano_gappas_blog/posts/22437895
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/mariano_gappas_blog/posts/22437895
Don't have an account? Find a server at joinmastodon.org
May 15, 2025
@discord_engineering Overclocking dbt: Discord's Custom Solution in Processing Petabytes of Data #DataEngineering #BigData #Scalability
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/discord_engineering/posts/22292015
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/discord_engineering/posts/22292015
Don't have an account? Find a server at joinmastodon.org
Apr 9, 2025
@ethicalads DuckDB and PostgreSQL Make a Great Pair for Analytical Processing #DataEngineering #Databases #Analytics
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/ethicalads/posts/22418187
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/ethicalads/posts/22418187
Don't have an account? Find a server at joinmastodon.org
Feb 25, 2025
@philip_heltweg An Empirical Study on the Effects of Jayvee, a Domain-Specific Language for Data Engineering, on Understanding Data Pipeline Architectures #DataEngineering #Software #Research
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/philip_heltweg/posts/22426436
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/philip_heltweg/posts/22426436
Don't have an account? Find a server at joinmastodon.org
Feb 12, 2025
@roland_boumans_blog DuckDB Bag of Tricks: Reading JSON, Data Type Detection, and Query Performance #DuckDB #SQL #DataEngineering
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/roland_boumans_blog/posts/22454392
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/roland_boumans_blog/posts/22454392
Don't have an account? Find a server at joinmastodon.org
Dec 19, 2024
@stanislavs_big_data_stream meet your new data lakehouse: S3 Iceberg Tables #DataEngineering #BigData #DataLakehouse
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/stanislavs_big_data_stream/posts/22402975
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/stanislavs_big_data_stream/posts/22402975
Don't have an account? Find a server at joinmastodon.org
Dec 5, 2024
@blake_smith How to use Parquet Java without Hadoop #DataEngineering #Java #BigData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/blake_smith/posts/22403412
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/blake_smith/posts/22403412
Don't have an account? Find a server at joinmastodon.org
Oct 5, 2024
@square_corner_blog Enhancing Data Quality Using Better Designed ETLs #DataEngineering #DataQuality #ETL
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/square_corner_blog/posts/22379685
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/square_corner_blog/posts/22379685
Don't have an account? Find a server at joinmastodon.org
Sep 30, 2024
@thomas_larock Microsoft Fabric is the New Office #DataEngineering #Microsoft #CloudComputing
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/thomas_larock/posts/22460858
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/thomas_larock/posts/22460858
Don't have an account? Find a server at joinmastodon.org
Jul 11, 2024
@thomas_larock Export to CSV in Azure ML Studio #DataEngineering #AzureML #MachineLearning
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/thomas_larock/posts/22460877
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/thomas_larock/posts/22460877
Don't have an account? Find a server at joinmastodon.org
Jan 17, 2024
@thomas_larock Azure ML Studio Sample Data #DataEngineering #AzureML #MachineLearning
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/thomas_larock/posts/22460887
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/thomas_larock/posts/22460887
Don't have an account? Find a server at joinmastodon.org
Jan 8, 2024
@philip_heltweg A Systematic Analysis of Problems in Open Collaborative Data Engineering #DataEngineering #OpenSource #Research
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/philip_heltweg/posts/22426569
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/philip_heltweg/posts/22426569
Don't have an account? Find a server at joinmastodon.org
Oct 30, 2023
@nic_crane Debugging #DataEngineering #OpenSource #RD
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/nic_crane/posts/22442095
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/nic_crane/posts/22442095
Don't have an account? Find a server at joinmastodon.org
Oct 3, 2023
@rory_sawyer Data Pipelines with ClickHouse, dbt, and Superset #DataEngineering #Analytics #Databases
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/rory_sawyer/posts/22454972
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/rory_sawyer/posts/22454972
Don't have an account? Find a server at joinmastodon.org
Jul 12, 2023
@philip_heltweg Challenges to Open Collaborative Data Engineering #DataEngineering #OpenSource #Collaboration
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/philip_heltweg/posts/22426601
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/philip_heltweg/posts/22426601
Don't have an account? Find a server at joinmastodon.org
Jan 3, 2023
@nic_crane Type inference in readr and arrow #DataEngineering #OpenSource #Programming
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/nic_crane/posts/22442107
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/nic_crane/posts/22442107
Don't have an account? Find a server at joinmastodon.org
Nov 21, 2022

« Newer Older »