Integration of Google BigQuery with SAP Data Warehouse Cloud




Analyzing business data often requires the consumption of additional business-critical data from third-party sources. It is counterproductive to have to copy or move all this data into DWC for such analysis. Instead, DWC has the provision to integrate data from multiple sources, one such example being Google BigQuery.


Data federation between Google BigQuery and DWC has been explored in the following blogs:

  • Google BigQuery to SAP Data Warehouse Cloud DWC
  • Data Federation Between SAP Data Warehouse Cloud (DWC) and Google BigQuery

Here, delving deeper into this usecase, we create a Google BigQuery connection based on a specific GCP Service account in DWC and access data from a different third-party account in BigQuery. Read on to find out how this is done.

SAP Data Warehouse Cloud (DWC) is a cloud-based data warehousing solution that combines both efficient data management and advanced analytics. Built on the SAP HANA Cloud database, DWC combines data from several sources (SAP and third-party data, across clouds and on-premise repositories), enabling highly efficient real-time analysis.

Architecture of SAP Data Warehouse Cloud
Figure 1. Architecture of SAP Data Warehouse Cloud (Refer link)

Google BigQuery is part of the Google Cloud Platform (GCP) – It is a serverless, scalable, multicloud data warehousing solution that enables data analysis at a petabyte scale.

Once integrated with DWC, data from Google BigQuery can be queried directly by means of virtual tables in DWC. Live connectivity ensures that queries are federated through virtual tables with fresh data and there is never any need for caching or to replicate the data from the source.

SAP DWC and GCP Integration
Figure 2. SAP DWC and GCP Integration (Refer link)

DWC can connect to GBQ using two mechanisms:

  • Through SDI with the use of the Google BigQuery adapter
  • Directly from DWC through the connection type 'Google BigQuery'

As of May 2021, the first connection type (through SDI) is not available. It requires a 3rd party driver, but this has been blocked by SAP for security reasons.

This blog will focus on the second method, i.e., establishing a connection directly from DWC using the ‘Google BigQuery’ connection type.

Following are the prerequisites to integrating Google BigQuery data with SAP DWC

  • Google Cloud Platform Account
  • User with relevant authorization to use Google BigQuery
  • Google BigQuery Service Instance with data
  • SAP DWC instance

Comments

Post a Comment

Popular posts from this blog

Efficient SAP Data Integration Using Ingestion Frameworks

Image
With exponential technological advancements in play, businesses need efficient and reliable ways to manage and integrate large volumes of data. Complex and comprehensive systems require robust data ingestion frameworks to ensure smooth data processing, compliance with regulatory standards, and maintenance of data integrity. This blog covers the critical aspects of data integration using ingestion frameworks, highlighting the challenges businesses face—such as handling diverse data sources and formats, ensuring data quality, and maintaining scalability—and pointers on how to overcome them. What is a Data Ingestion Framework? A data ingestion framework encompasses a set of tools and methodologies designed to efficiently collect, process, and load data from various sources into a central repository. It is fundamental in  modern data management , enabling systematic handling of vast amounts of data. Key steps include data collection, transformation, and loading, ensuring data integrity...
Read more

How Hyper-personalization Can Drive Customer Engagement, Conversions, and Loyalty

Image
In the previous   blog , we explored the different areas of business where hyper-personalization can play a vital role. Today we will look at the different benefits of hyper-personalization and how implementing a strategic initiative around a personal customer focus can lead to concrete value realization and give you large returns in the long run. 76% of consumers are more likely to consider purchasing from organizations that personalize -   Mckinsey [1] Hyper-personalization is all about  NOT  taking your customers for granted. An intelligent and personalized campaign can be more impactful, than a generic one that hits a broader customer base - with little to do with a single customer’s preferences. A segmented campaign with targeted messaging is more likely to persuade users and lead to healthy conversions. Even campaigns focused on the correct gender are a step in the right direction. I am sure everybody has been frustrated by a message addressed to you, the indiv...
Read more

Strategic Approach to Driving User Adoption in Business Transformations – 5 key steps

Image
  Despite investing in various digital transformation projects, many organizations overlook ensuring the success of each project before moving on to the next. This haste creates a digital adoption gap, where the promised benefits of technology capabilities remain unrealized due to limited user engagement. To bridge this gap and unlock the full value of software investments, leaders should prioritize and ensure effective user adoption before diving into the next project. After the implementation of digital transformation projects, business leaders often move on to the next initiative, with the deployed technology becoming a part of the tech stack. This challenge is intensified by diverse workforces across generations, from baby boomers to Gen Z, necessitating innovative training approaches. Success in business transformations relies on user adoption – engaging users, tailoring training, and addressing concerns. High adoption rates unlock added value, paving the way for growth and pr...
Read more