What Is AI Engineering? A Practical Guide to the Discipline, the Skills, and How to Build the Practice
AI engineering isn't data science, data engineering, or traditional software engineering. It's a distinct discipline sitting at the intersection of all three — and it's genuinely useful to have a clear working definition, whether you're building this capability yourself, hiring for it, or leading a team through it
Job postings for "AI Engineer" have proliferated over the past two years, and a genuine problem sits underneath that growth: the title means something different at almost every organisation using it. Some are describing a data scientist with a new job title. Some mean a data engineer who's added a model call to an existing pipeline. Some mean a backend engineer who's added an LLM API call to an existing product. Some mean someone building agentic workflows from scratch with no established pattern to follow. The role is real and increasingly essential, but the lack of a shared definition makes it genuinely difficult to hire for, to structure a team around, to build a career plan against, or to evaluate whether someone already doing this work has what they actually need.
AI engineering isn't data science, data engineering, or traditional software engineering.
This article sets out a working definition of AI engineering as a discipline sitting at the intersection of data science, data engineering, and traditional software engineering - overlapping meaningfully with each, reducible to none of them, along with the specific technical and software engineering skills it actually requires, and a practical guide for building this capability. It's written as a companion to our earlier piece on what a converged Data Team 2026 should look like - where that article argued AI capability belongs inside the same team as data and integration rather than as a separate, disconnected function, this one goes one level deeper: what does AI engineering actually involve, and what does it take to do it well. This piece also draws the boundary lines explicitly between AI engineering and three disciplines it sits closest to and is most often confused with, data engineering, DevOps, and security. Since where those boundaries actually sit is as important to understand as what AI engineering itself requires.
What AI Engineering Actually Is
AI engineering is the discipline of building, integrating, and operating production systems that incorporate AI models, most commonly large language models and agentic workflows into real business processes. Including the reliability, governance, and maintainability expected of any other production software. This definition is deliberately narrower than "anyone who works with AI," and the narrowing matters.
A data scientist trains and evaluates models, a genuinely different, adjacent discipline focused on model performance, statistical validity, and experimentation.
A data engineer builds and governs the pipelines and platform that make data trustworthy and accessible in the first place. Ingestion, transformation, schema evolution, quality gating, largely independent of whether an AI system ever consumes that data.
A traditional software engineer builds and maintains applications, with AI as one component among many.
An AI engineer sits specifically at the intersection of all three: taking a model, usually not trained in-house, but accessed via an API or a hosted endpoint and building the surrounding system, on top of governed data, that makes that model useful, reliable, governed, and maintainable in production.
The distinction that matters most in practice: AI engineering is fundamentally a systems and integration discipline, not a model-building one, and not a data-platform-building one either. Most AI engineers today are not training models from scratch, and most are not building the data pipelines they consume. They are building the context retrieval, the tool-calling infrastructure, the evaluation harness, and the guardrails that determine whether a call to Claude or GPT. Operating on data someone else has already made trustworthy to produce something correct and useful, repeatedly, in production, under real load, with real governance requirements.
The Technical Skill Set
Model and API Integration
The foundational skill: understanding how to call a hosted model API correctly and efficiently. Prompt construction, context window management, streaming responses, handling rate limits and retries, and understanding the meaningful behavioural differences between model providers rather than treating them as interchangeable. This includes practical familiarity with the major API surfaces (Anthropic's API, OpenAI's API, and increasingly, models hosted directly inside a data platform via services like Databricks Model Serving) and the trade-offs between calling a model directly versus through an abstraction layer or gateway.
Retrieval and Context Engineering
Building the systems that supply a model with the right information at the right time. Retrieval-augmented generation (RAG) architecture, vector search and embedding strategy, chunking strategy for source documents, and increasingly, structured context retrieval directly from a governed data platform rather than an unstructured document store. This is where AI engineering overlaps most directly with data engineering, and where, understanding how the underlying medallion architecture and semantic layer were built pays off most concretely: it positions someone to retrieve genuinely reliable context, rather than working with a black box.
Tool Use and Agentic Workflow Design
Designing the specific capabilities a model is given access to, database queries, API calls, code execution, file operations and the orchestration logic governing how and when a model invokes them. This includes practical fluency with tool-calling patterns (function calling, MCP servers), multi-step agentic loops, and increasingly, the harness-level thinking gaining traction across the industry recently: reliability at scale comes from the surrounding system of tests, gates, and rules that constrain what an agent is allowed to do, not from a more capable model alone.
Evaluation and Testing
Building the harness that determines whether an AI system is actually working not just whether it runs without error, but whether its output is correct, safe, and consistent. This includes designing evaluation datasets, building automated scoring and regression testing for AI-generated output, and critically, given the governance research covered elsewhere on this blog, building the traceability to determine after the fact whether an AI-generated output or action caused a specific production issue.
Prompt and Context Management as Engineering Discipline
Treating prompts, system instructions, and context construction as versioned, tested artefacts rather than ad hoc strings embedded in application code including prompt versioning, systematic A/B testing of prompt variants, and the discipline to change one variable at a time when debugging why an agent's output degraded.
Cost and Performance Engineering
Understanding the cost model of AI-assisted systems specifically, token-based pricing, the cost implications of context window size and retrieval strategy, model selection based on cost-versus-capability trade-offs for a given task, and the budget and governance controls (the same territory covered in our Genie pay-as-you-go pieces) required to keep AI system cost visible and managed rather than discovered at the end of the month.
The Software Engineering Skill Set
This is the part of AI engineering most consistently underweighted in; how the role gets hired for, in how it's taught, and in how people entering the field prioritise their own learning. It's the section most worth sitting with, whether you're evaluating a candidate, assessing your own gaps, or deciding what to learn next.
Solution Design and Architecture
The ability to design a system architecture before writing code. Ddeciding where an AI capability sits relative to existing systems, what data it needs access to and how that access is governed, what the failure modes look like and how the system degrades gracefully when a model call fails or returns something wrong, and how the AI-assisted component fits into a broader application or platform rather than existing as an isolated demo. This is a genuinely distinct skill from prompt engineering or API integration, and it's the skill most likely to be missing where AI capability has grown organically rather than deliberately.
Integration Engineering
Because an AI system is rarely useful in isolation, the ability to integrate it correctly with existing systems; APIs, databases, authentication and identity systems, existing application logic is core to the role, not adjacent to it. This is precisely the overlap with traditional integration engineering discussed in the Data Team 2026 piece, and it's a strong argument for building AI capability alongside existing integration expertise rather than treating it as an entirely separate specialism with no integration depth.
Production Software Engineering Fundamentals
Version control discipline, code review, testing, CI/CD, observability, and incident response. The same fundamentals that apply to any production software, applied to AI-assisted systems specifically. AI engineering work without solid software engineering fundamentals tends to produce systems that work well in a demo and fail unpredictably in production, precisely the pattern described in GitLab's 2026 AI Accountability Report, where the bottleneck moved from writing code to reviewing and validating it, without a corresponding increase in the discipline needed to do that reviewing well.
Data Modelling and Governance Literacy
AI engineering doesn't require deep data engineering expertise, but it does require genuine literacy in how the underlying data is modelled, governed, and access-controlled. Understanding what a semantic layer is and why it matters for retrieval quality, understanding row and column level security well enough to build a system that respects it rather than bypasses it, and understanding schema evolution well enough that a pipeline change doesn't silently break an agent's context retrieval three weeks later.
Security-Conscious Engineering
Given the governance research already covered on this blog, the vibe coding governance gap, the shadow AI findings from Retool's survey. Genuinely capable AI engineering requires security awareness specific to AI systems: prompt injection risk, the danger of granting an agent broader data access than a task actually requires, and the discipline of building systems where an agent's permissions are scoped down from whatever the calling user has access to, rather than inherited wholesale by default.
Where AI Engineering Ends and Adjacent Disciplines Begin
The skill sets above describe what AI engineering requires directly. Just as important, and consistently underspecified in how the role gets described, is where AI engineering's responsibility ends and a genuinely adjacent discipline takes over. Getting this boundary wrong in either direction causes real problem; treated too narrowly, AI engineering becomes disconnected from the platform it depends on; treated too broadly, it becomes an unbounded catch-all role nobody can actually staff or evaluate against.
Data Engineering
Data engineering and AI engineering are adjacent, overlapping, and genuinely distinct. Data engineering owns the ingestion, transformation, and governed modelling of data itself, the medallion architecture, the schema evolution handling, the quality gating that determines whether a table can be trusted. AI engineering consumes that output, building the retrieval, context construction, and tool-use layer on top of it.
The boundary in practice: an AI engineer needs to understand how the underlying data was modelled and governed well enough to retrieve it correctly and know when to distrust it but building and maintaining the pipelines that produce that data in the first place is data engineering's job, not AI engineering's. Converged Data Teams should primarily consist of five distinct roles specifically to make this boundary explicit rather than assumed; a Platform Engineer owning shared infrastructure and governance, an Integration Engineer owning ingestion, an Analytics Engineer owning the semantic layer, an AI/ML Engineer embedded with direct visibility into how the data was built, and a Data Governance Lead owning review and validation across all of it. AI engineering, in that structure, is one role among several sharing a platform, not a discipline that also has to build the platform from scratch.
DevOps
AI engineering does not replace DevOps practice it extends it into a domain DevOps wasn't originally built for. Deployment pipelines, infrastructure-as-code, rollback strategy, and monitoring and alerting are mature DevOps disciplines with established patterns; AI engineering's job is applying those patterns to a new category of workload with its own failure modes. A model call that returns a plausible-but-wrong answer instead of an HTTP error, a context window that silently truncates instead of failing loudly, an agent that takes an unintended action instead of crashing.
In practice, this means an AI engineer needs enough DevOps fluency to deploy and operate an AI-assisted system using the same CI/CD, observability, and incident response infrastructure the rest of the organisation already runs on and not a parallel, AI-specific deployment stack maintained in isolation. Where an organisation has a dedicated platform or DevOps function, the right pattern is AI engineering building on top of that function's existing infrastructure and conventions, with AI-specific monitoring (evaluation scores, hallucination detection, cost tracking) added as an extension of existing observability rather than a separate system nobody else understands.
Security
AI engineering inherits every standard application and data security requirement, secrets management, network security, dependency vulnerability management, identity and access management in full. On top of a set of genuinely AI-specific risks: prompt injection, the danger of an agent inheriting broader data access than a given task requires, and the need to treat an AI system's permissions as something to be actively scoped down rather than passively inherited.
The honest position on ownership: in most organisations, an AI engineer should not be the sole owner of security for the systems they build, the same way a backend engineer typically isn't the sole owner of network security for the application they ship. The realistic model is an AI engineer who understands AI-specific risk well enough to build securely by default and to work effectively with a dedicated security function — rather than either extreme of assuming security is entirely someone else's problem, or assuming AI engineering needs to independently reinvent an organisation's entire security practice.
Building the Practice
Don't treat AI capability as a separate function disconnected from data and integration
The strongest AI engineering capability tends to emerge from people who already understand the underlying data platform, not from a standalone AI specialist hired or self-taught in isolation, reverse-engineering that understanding from the outside.
Distinguish AI engineering from data science explicitly
These are adjacent but genuinely distinct disciplines, requiring different primary skills. Model evaluation and statistical rigour for data science, systems integration and production engineering for AI engineering. Conflating the two, whether in a job description or in a personal learning plan, tends to produce strength in one dimension and unexpected weakness in the other.
Invest in the harness before the scale
The strongest AI engineering outcomes come from building evaluation infrastructure, guardrails, and review capacity before scaling up the number of AI-assisted systems in production, not after.
Treat AI engineering literacy as a baseline skill, not a narrow specialism
The stronger long-term pattern, is building AI engineering literacy as an expected skill across an existing data and integration team; the way SQL fluency became a baseline expectation a decade ago rather than treating it as a specialism held by only one or two people.
Assign explicit ownership of evaluation and governance
Someone needs to own the evaluation harness, the cost governance, and the review discipline for AI-generated output and agent actions specifically not as an assumed byproduct of good engineering practice generally, but as a named, accountable responsibility, consistent with the Data Governance Lead role.
A Diagnostic, However You're Approaching This
The useful question isn't "do we have an AI engineer," or "am I an AI engineer yet." It's a more specific set of questions, useful from any vantage point: does this cover the model-integration side (retrieval, tool use, evaluation), the data literacy side (understanding how the underlying platform was built and governed), and the software engineering side (solution design, integration, production discipline) well enough to build something that survives contact with real production load? Is this capability connected to genuine understanding of the underlying data platform, or built in isolation from it? Are the boundaries to data engineering, DevOps, and security actually clear, or is everything simply assumed to be "AI engineering's problem" by default? And is there explicit, named ownership of evaluation and governance, or is that responsibility being assumed rather than assigned?
Where the answer to any of these is genuinely uncertain; whether you're assessing a team, a hire, or your own skill set that uncertainty is the actual starting point. Not a headcount number or a course to enroll in, but a clear map of which of these skills already exist, and which need to be deliberately developed.
This article was written by Keith Jenneke, Principal Consultant & Practice Lead at Cypher Agency. Keith leads our data platform and AI engineering practice, building on Azure Databricks across professional services, resources, and government.
References
GitLab Inc. (2026, June 23). GitLab Research Reveals Organizations Are Generating AI Code Faster Than They Can Control It. ir.gitlab.com
WRITER. (2026). Enterprise AI Adoption in 2026: Why 79% Face Challenges Despite High Investment. writer.com/blog
Publicis Sapient. (2026, June 17). 2026 Global Enterprise AI Report Reveals Gap Between AI Adoption and Enterprise Readiness. publicissapient.com
Comments