Is AI Going to Kill Data Engineering Jobs? No — But the Job Is Already Changing
- Jun 28
- 5 min read
What the actual hiring data, platform telemetry, and named industry voices are saying about AI's impact on data engineering work — without the reassurance or the panic.
The debate over whether AI is going to eliminate data engineering jobs has been running for a couple of years now, and it tends to produce one of two confident answers: yes, obviously, look at what these tools can already do; or no, don't worry, AI just removes the boring parts and the job gets more interesting. Neither answer holds up well against the actual data available right now, because the real picture is more specific and more uncomfortable than either side's shorthand suggests.
We looked at what the hiring numbers, the platform-level usage data, and the people who write and ship data pipelines for a living are actually reporting. The honest summary: a meaningful contraction is visible in data-specific hiring, a much larger share of foundational infrastructure work has already shifted to AI agents on major platforms, and broader engineering hiring at the same companies is, somewhat counterintuitively, holding up or growing. These are not contradictory facts. They describe the same shift from two different angles.
What the Hiring Data Actually Shows About Data Engineering Jobs
Indeed Hiring Lab's 2026 labour market data shows data and analytics job postings declined 15.2% year-over-year through October 2025, against an 8.5% decline across overall technology postings in the same period. Taken at face value, that means data-specific roles have absorbed roughly twice the hiring contraction of the broader technology sector over the same window.
That is a genuinely significant number, and it is the strongest evidence available that something specific is happening to data engineering hiring, not just technology hiring in general.
At the same time, research from venture firm SignalFire — covering hiring at twelve major technology companies including Alphabet, Meta, Amazon, Microsoft, and Netflix — found that engineers made up 55% of all new hires across those companies in 2025, up from 46% in 2019. Early-stage startups in the same dataset increased engineering hires by 7% compared to 2019. SignalFire's head of research, Asher Bantock, put it directly to TechCrunch: companies frequently cite AI as the reason for recent layoffs, but the hiring data does not support a story where AI is simply replacing engineering headcount. His read: engineers are more productive now, and there is more work for them to do, not less.
It is important to be precise about what this second dataset does and does not show. It is about software engineering hiring broadly at large technology employers — it is not a data-engineering-specific figure, and the companies in SignalFire's sample are not representative of the broader economy. The two datasets are measuring different populations and reconciling them requires looking at what is actually happening at the infrastructure layer rather than assuming one number cancels out the other.
What's Happening at the Infrastructure Layer
Databricks' 2026 State of AI Agents report, built from aggregated activity across more than 20,000 customer organisations including over 60% of the Fortune 500, puts a specific number on a trend that has been visible anecdotally for a while: more than 80% of new databases created on the platform are now built by AI agents rather than human engineers, and 97% of database testing and development environments are agent-created. Two years earlier, agent-created databases on the underlying infrastructure were close to zero.
This is a specific, measurable, and very fast shift in who performs one category of foundational data engineering work — provisioning, schema design, and environment setup that used to require a data engineer's time and judgement, and increasingly does not. Databricks frames this as connected to "vibe coding" — natural-language-driven development where an agent interprets a requirement and builds the supporting infrastructure directly, with a human reviewing the result rather than writing it from scratch.
The same report found that only 19% of organisations have actually deployed AI agents at meaningful scale, despite 67% reporting some use of AI-powered tools — a substantial gap between experimentation and production use that suggests the infrastructure-layer shift, while real and fast where it has happened, is still concentrated in a minority of organisations rather than universal yet. The report also found that companies using governance and evaluation tooling around their agents put meaningfully more AI projects into production than those that do not, by a factor the report puts at roughly six times for evaluation tooling and over twelve times for governance tooling specifically.
Put together, this suggests the contraction visible in data engineering job postings and the infrastructure-layer shift to agent-built databases are very likely describing the same underlying change: the routine, templated portion of data engineering work — the part that looked most like provisioning and boilerplate construction — is the part disappearing from job postings first, because it is also the part agents have become genuinely capable of doing without close supervision.
What Practitioners Working in the Field Are Saying
This is not only visible in aggregate statistics. It is also the subject of ongoing, detailed analysis from people who build data pipelines professionally and write about it publicly. Zach Wilson, a well-known voice in the data engineering community, has published a detailed breakdown of which specific data engineering tasks and skills are most exposed to AI automation versus which are not. Joe Reis, co-author of Fundamentals of Data Engineering, has argued that treating AI fluency as optional for a working data engineer is no longer a defensible position — not because the discipline is disappearing, but because using AI well has become a baseline expectation of the role, the way version control or SQL fluency already are.
The throughline across this commentary is consistent with what the hiring and platform data suggests: the work that survives and grows in value is judgement-heavy — architecture decisions, governance design, data quality strategy, knowing when an agent's output is subtly wrong — and the work that contracts fastest is execution-heavy and repeatable.
What This Adds Up To
None of the data reviewed here supports a simple "AI is eliminating data engineering" narrative, and none of it supports a simple "don't worry, it's all fine" narrative either. What it supports is a more specific claim: the volume of routine, execution-level data engineering work is contracting measurably, visible in both job posting data and platform-level agent adoption figures, while the demand for people who can govern, architect, and supervise the systems doing that execution work appears to be holding up or growing, consistent with the broader engineering hiring resilience the SignalFire data shows at major technology employers.
That is a genuinely different situation to either "your job is safe" or "your job is gone," and it implies a different question for anyone working in or hiring for data engineering roles right now — not whether the job will exist, but which parts of the current job description are likely to keep being asked for, and which parts already look like they are being automated out from under a job title that has not yet caught up to the change.
Cypher Agency is a boutique data and integration engineering firm helping mid-sized businesses build reliable, governed data and integration environments — without the cost of building an internal team.
References
Indeed Hiring Lab. (2026). 2026 Labour Market Report, cited in Datafold. (2026, March 5). Data Engineering in 2026: 12 Predictions. datafold.com/blog
SignalFire. (2026). Engineering hiring research, cited in Temkin, M. (2026, June 24). AI was supposed to kill engineering jobs, but new data suggests they're the most resilient. TechCrunch. techcrunch.com
Databricks. (2026). 2026 State of AI Agents. databricks.com/resources/ebook/state-of-ai-agents
SiliconANGLE. (2026, January 27). Databricks report finds surge in AI agent adoption despite governance bottlenecks. siliconangle.com
Comments