All Posts

Stop searching for AI use cases. Find practical AI opportunities by identifying workflow friction, repetition and manual work.

LLM cost optimisation starts before the API call. Learn how caching, deterministic logic, retrieval and precomputation can eliminate unnecessary model requests.

Shadow AI may already be operating in your business. Learn how to discover which AI tools your team uses, what data they share and where the opportunities and risks are.

LLMs are powerful, but they should not be the default processing layer for every task. A practical framework for deciding what belongs in code, retrieval, algorithms and AI reasoning.

AI coding agents have had a difficult few months for security. In July, researchers demonstrated that a public GitHub issue could manipulate an agent into retrieving information from a private repository. Another piece of research showed how symbolic links could undermine the approval controls used by six different coding assistants. Separately, developers have reported coding agents executing destructive database operations without adequately signaling the consequences. Thes

Most organisations know exactly which data centre region their core platform lives in. Almost none can answer the same question for AI. Here's where your data actually goes the moment it touches an AI model, and why the answer matters more than most governance frameworks currently account for.

AI engineering isn't data science, data engineering, or traditional software engineering — it's a distinct discipline sitting at the intersection of all three. Whether you're building the capability yourself, hiring for it, or leading a team through it, here's a working definition

Databricks' community connector framework got us part of the way to a Xero-to-Lakehouse integration, but not all the way. Here's what it took to finish the job properly — four platform-internal bugs, a decision to rewrite rather than fork someone else's code, and why we're open-sourcing what came out of it.

A $600,000 Salesforce contract, cancelled in two months. Enterprise transformations that run for years, weighed down by governance nobody could shortcut. Having worked inside both, I know why these aren't really the same debate — and that gap is the part worth talking about.

Most organisations still run separate data, integration, and AI teams — three roadmaps, three toolchains, three reporting lines. That separation made sense when each function had genuinely distinct skills. It doesn't hold up as well now. Here's the case for a converged data team, and what it should actually look like.

Three genuinely different mechanisms exist for row and column level security in Unity Catalog. A practitioner's guide to what each one is actually for — and the one mistake that trips up more rollouts than any misconfigured filter

GitLab's 2026 AI Accountability Report surveyed 1,528 developers and technology buyers. The finding worth sitting with: 78% code faster with AI, but overall software delivery hasn't accelerated. The bottleneck didn't disappear — it moved. What this means for Data Integration, Analytics, and AI platform teams specifically

Databricks Genie moved to pay-as-you-go billing on 6 July 2026. This article sets out what the change requires of data team leaders, and what it requires of data engineers, analysts, and data scientists who use Genie directly, within a governed Data Integration, Analytics, and AI capability

A CDF history table captures every Delta Lake change permanently — independent of VACUUM retention. Here's the pattern, the schema evolution handling, and the partitioning strategy that scales

Data and analytics job postings are down twice as much as the broader tech market. At the same time, engineering hiring overall is resilient, and AI agents now build the majority of new databases on major platforms. Here's what the actual data says is happening

SCD Type 2 Databricks implementation compared — manual MERGE logic versus Lakeflow's APPLY CHANGES INTO. Working code, gotchas, and migration lessons from real production pipelines

Vendors are already converging on multi-model routing. We explain the five questions that actually determine whether AI agents create value for your business — and why they matter more than which platform you standardise on.

Change Data Feed in Databricks looks simple in the documentation. In production, it breaks in six distinct ways — and each one is silent until something downstream stops working. This is what I learned building CDF pipelines on Azure Databricks.

Tokenmaxxing — the practice of treating AI token consumption as a measure of productivity — emerged from Silicon Valley in early 2026. It is now showing up in data engineering teams across industries, and the costs are less visible but no less real

Anthropic has published an official integration between Claude and Databricks — giving teams the ability to ask questions of their data in plain English. Here is what your environment needs to look like before that capability actually works.