Snowflake OpenFlow in Action – Part 6 – Real Time Data Ingestion from SharePoint into Snowflake

Introduction:

Continuing our Snowflake Openflow blog series, this post dives into the Openflow Connector for SharePoint—a powerful tool for ingesting data from SharePoint lists, files, and document libraries directly into Snowflake. Whether you’re centralizing business data for reporting or enabling downstream analytics, this connector makes it easier than ever to integrate SharePoint into your modern data stack. By leveraging the connector, you can ingest Sharepoint content for custom processing with Snowflake and prepare the ingested files for conversational analysis using AI assistants via snowflake cortex.

For organizations aiming to integrate SharePoint data into their Snowflake environment, this connector provides a robust and efficient pathway, enabling reliable, automated pipelines from SharePoint to Snowflake.

Prerequisites:

• • Ensure that Openflow is setup on your snowflake account and a deployment is active. You can refer to previous blog on how to setup openflow in your snowflake account
  • Microsoft Entra ID (Azure AD) Setup
    • Setup oAuth based authentication between snowflake and Microsoft 365. This identifies the connector as a trusted client that can access your organization’s sharepoint environment securely.
    • Generate Client ID, Client Secret and Tenant ID.
    • Assign API Permissions and generate Client Certificates which are required for the connector’s parameters.
  • Snowflake Setup
    • Create a user with type = ‘SERVICE’ and grant OPENFLOW_ADMIN_ROLE created during Openflow setup to this user.
    • Generate Key-pair – public and private keys for this service user
    • Create a Database and schema and give usage and appropriate privileges to the OPENFLOW_ADMIN_ROLE
    • Create a warehouse for the connector to use, with the smallest size and scale up if required. Grant Usage on this warehouse to OPENFLOW_ADMIN_ROLE.
      

Steps to setup the connector:

• Launch Openflow from Data -> Openflow. Create a new Runtime using the same Deployment that is created earlier (Refer previous blog to know how to create a Deployment). Once the Runtime is Active, navigate to the Openflow Overview page. Go to Featured Connections section and click on View more Connectors. Select Microsoft SharePoint (simple ingest) connector and click on Add to Runtime 

   

• Select your runtime from the Available runtimes drop-down list. Click Add. 

• Authenticate using your snowflake credentials. 

   

• Once the connector installation is complete, Openflow Canvas will appear with the connector process group added to it. 

   

• Now right-click on open area of the canvas and select Enable All Controller Services. 

   

• Now again right-click on Microsoft Sharepoint(Simple Ingest) -connector and select Parameters to set the flow parameters. 

   

• There are few parameters context to be set to ingest sharepoint data. Source Database Parameters: 

   

  Parameter Name	Parameter Value	Description
  SharePoint Application Certificate		An application certificate in PEM format. Example: —–BEGIN CERTIFICATE—– certificate value —–END CERTIFICATE—–
  SharePoint Client ID		These are credentials that Snowflake uses to request access tokens from Microsoft Entra ID. Think of them as the username/password (securely stored) for your app.
  SharePoint Client Secret
  SharePoint Tenant ID
  SharePoint Site Domain	www.domainname.sharepoint.com	Give the domain name of the synchronized SharePoint site.
  SharePoint Site URL	https://domainname.sharepoint.com/sites/sitename	Give the SharePoint site from which the connector will ingest content.

  

  Snowflake Parameters:

  Parameter Name	Parameter Value	Description
  Snowflake Account	<Organization_Name>-<Account_Name>	Give your Snowflake Account Identifier copied from your Account Details
  Snowflake Database	OPENFLOW_SHAREPOINT	Give the Snowflake Database where the data should be stored
  Snowflake Private Key		RSA Private key Value with header and footer. Note that either Snowflake Private Key or Snowflake Private Key File should be defined.
  Snowflake Private Key File		Upload the RSA Private key file by checking the Reference Assets box
  Snowflake Private Key Password		Give the password if you have the encrypted RSA Private Key file
  Snowflake User Role	OPENFLOW_ADMIN_ROLE	Give the Snowflake Role to be used during execution
  Snowflake Username	OPENFLOW_SHAREPOINT_USER	Give the Snowflake Service username created earlier
  Snowflake Warehouse	OPENFLOW_WH_SHAREPOINT	Give the Snowflake Virtual warehouse to be used for execution

• Run the workflow by right-clicking on the connector and selecting Start.

   

  • Once the workflow is completed, you will be able to find new objects created in Snowflake.
    This concludes the Microsoft Sharepoint (simple ingest) connector ‘s role of ingesting data from Sharepoint to Snowflake. We would similarly install the next connector Microsoft SharePoint (Cortex connect)  to Ingest Sharepoint content and make it ready for chat in your AI Assistants with Snowflake Cortex.
    
    
  • Follow the similar process of  Enabling all controller services. Parameters would be automatically defaulted. Verify them once and then Start the Process Group.  This connector would be chunking the word documents that were fetched from the previous connector and would be storing them in snowflake tables. Once the workflow is completed, you will be able to find new object DOCS_CHUNKS tables created and populated.

    The Microsoft Sharepoint(Cortex connect) basically does the following activities. 

  • Breaks full documents into smaller, manageable “chunks” of text (usually paragraphs or sections).

  • These chunks are then indexed or stored separately for search, retrieval, or AI model consumption.

  • Each chunk can be linked back to its source document using metadata (like document ID, file name, URL, etc.).

    Conclusion:
    
     Unlocking SharePoint for Snowflake—No Code, Just Flow Bringing SharePoint data into Snowflake doesn’t have to be complex or code-heavy. With OpenFlow, you get a streamlined, low-code pipeline that connects the dots between Microsoft’s content-rich collaboration platform and Snowflake’s powerful analytics engine. From handling authentication and chunking documents to enabling AI-powered search and analysis, OpenFlow abstracts away the heavy lifting—so you can focus on delivering insights, not debugging pipelines. With secure integration, incremental sync, and minimal setup, turning your SharePoint content into trusted, query-ready Snowflake data has never been more straightforward.

    Referral Link: https://docs.snowflake.com/en/user-guide/data-integration/openflow/connectors/sharepoint/setup

     

    Please feel free to reach out to us for your Snowflake solution needs. Cittabase is a Premier  partner with Snowflake.