Unleashing the Power of Enterprise AI With SAP Databricks
Author: Andrew McQueen
20 Feb, 2025
In case you haven’t seen the news all over LinkedIn, Databricks and SAP have announced a groundbreaking partnership. SAP Databricks is provided by SAP as a part of SAP Business Data Cloud on Azure, AWS, and GCP – to make SAP data available in Databricks. This allows valuable enterprise data in SAP to seamlessly be read into Databricks to power AI applications.
What is SAP Databricks, and why should you care? I will explain everything you need to know about this huge announcement, and discuss some cool things SAP Databricks will enable. Think of SAP Databricks like a bridge – SAP provides the valuable enterprise data on one side, and Databricks provides the cutting-edge AI tools on the other side. SAP Databricks spans the gap between them to create huge value.
In this article, I’ll provide an overview of what SAP Business Cloud and SAP Databricks are, explain the key problem solved by SAP Databricks, and share my thoughts on the new partnership. Whether you are a SAP user or a Databricks partner, you should be excited about this announcement!
What are SAP Business Cloud & SAP Databricks?
SAP is a leading provider of enterprise software, with 90% of the Fortune 500 leveraging SAP applications like ERP, Ariba, SuccessFactors, and Concur. For many organizations, their most valuable data resides in SAP systems. Databricks, on the other hand, is a unified data analytics platform that provides powerful tools for data engineering, machine learning, and AI workloads.
According to SAP’s announcement, SAP Business Data Cloud is a “groundbreaking solution that unifies all SAP and third-party data throughout an organization, providing the trusted data foundation organizations need to make more impactful decisions and foster reliable AI.”
SAP Business Data Cloud
SAP Databricks is a key component of SAP Business Data Cloud. As Databricks explains in their recent blog post, “SAP Databricks provides customers artificial intelligence, data warehousing, and data engineering capabilities, all governed by Unity Catalog.” It enables bi-directional data sharing between SAP and Databricks while preserving the semantics and business context of the SAP data.
Basically, SAP Databricks is a fully managed SaaS solution to get your SAP data into Databricks so it can be used to build GenAI apps, models, and more.
The Problem Solved by SAP Databricks
In the past, connecting SAP data to Databricks for AI and analytics use cases was challenging. At my company Xorbix Technologies, we had a prospective client who wanted to use tools like Databricks AI/BI Genie on their SAP data to ask questions and generate visualizations & answers. However, there was not a simple, straightforward way to make that SAP data available in Databricks.
AI/BI Genie Chat
We considered various approaches, each with their own drawbacks:
- Connecting Databricks directly to the underlying SAP database raised security concerns and went against best practices by bypassing SAP’s security model. It was too time-consuming and error-prone for a small proof-of-concept. We needed a faster way where we didn’t have to build the connection and implement all the security ourselves.
- Using APIs to read SAP data would incur licensing costs. While it avoided some of the technical challenges of backdoor database access, it still would have required building and maintaining a custom ingestion process to copy the data into Databricks. Not only that, changes to the underlying source data in SAP would need to be synced with the copy in Databricks, to make sure Genie results were fresh and up to date. This would require a truncate & reload process (inefficient), or finding another way to read source system changes into Databricks. This kind of solution is not at all straightforward to implement for a quick POC.
- Leveraging third-party connectors like SAP CDC Connector in Azure Data Factory solved the issues of handling security and change data processing. However, it required licensing another tool and introduced yet another platform to manage and secure. We ultimately determined a third party connector wouldn’t be suitable for the POC, either due to prohibitive cost, or the added complexity.
In the end, although there were several options to get SAP data into Databricks, each had significant drawbacks that prevented us from moving forward with the POC. The client missed out on experiencing the power of Databricks for their SAP data.
SAP Databricks solves this problem by providing a seamless, fully-managed way to connect SAP and Databricks. According to the SAP Business Data Cloud solution brief, it allows you to “discover, share, and model data with ease” and access “all your data with zero copies.” Hence, all the problems I outlined would have been resolved by SAP Databricks. Now that SAP Databricks will soon be available, we will be following up with that organization to hopefully turn a past miss into the first of many Databricks+SAP success stories.
The Future is Now: How SAP Databricks Will Revolutionize Enterprise AI
Before SAP Databricks, connecting SAP data to Databricks for AI and analytics required costly, time-consuming solutions that were difficult to implement and maintain. But now, with their exclusive partnership, SAP and Databricks are offering a seamless, fully-managed way to unlock the value of SAP data using Databricks’ cutting-edge tools.
Whether you are a customer or a consulting partner, this is an announcement to celebrate. With SAP Databricks, I can help Xorbix’s clients drive huge value by applying Databricks’ AI and analytics capabilities to their SAP data. It’s a win-win for everyone.
I’m excited to see what the future holds for SAP Databricks. It’s the perfect partnership to bridge the gap between valuable enterprise data in SAP and the innovative AI and analytics tools in Databricks. The potential is enormous, and I can’t wait to see what customers will achieve with SAP Databricks.
Read more on our related services:
