Customer Engagement & Dynamics CRM Forum

 View Only
  • 1.  Data Export Service (DES) Alternatives

    Posted Mar 14, 2022 05:08 PM
    We are looking for peers that currently use Microsoft Dynamics Customer Experience (CE) (aka Dynamics CRM) and its "Data Export Service" (DES) functionality. DES is used to create a near-real-time replica database (ODS) which we use for reporting. DES is being deprecated in November 2022, and we are trying multiple POCs to determine a best replacement path.

    We would appreciate any thoughts others have on their approach.


    ------------------------------
    Lauren Palsgrove
    ------------------------------


  • 2.  RE: Data Export Service (DES) Alternatives

    Posted Mar 15, 2022 01:44 AM
    Hi Lauren

    Microsoft documentation for Azure Synapse Link for Dataverse can be found at:
    • https://aka.ms/DESDeprecationBlog
    • https://aka.ms/DESDeprecationPlaybook
    • https://docs.microsoft.com/powerapps/maker/data-platform/export-to-data-lake


    ------------------------------
    Robert Ward
    Chubb Australia
    ------------------------------



  • 3.  RE: Data Export Service (DES) Alternatives

    Posted Mar 15, 2022 09:07 AM
    ​Hi Robert,

    Thanks for the reply!  We have been using the materials above, but we are curious what others are doing to replicate the near real-time load of Data Export Services using Azure Synapse Link.  We have tried a few methods such as Append Only, In-place, Watermark, etc., but we would love to hear what has been successful for others in producing the near real-time results of DES.

    Thank you!

    Lauren

    ------------------------------
    Lauren Palsgrove
    ------------------------------



  • 4.  RE: Data Export Service (DES) Alternatives

    SILVER CONTRIBUTOR
    Posted Mar 15, 2022 01:50 AM
    We are looking at this ourselves. We are using Dynamics online. We plan to move to Azure Synapse Link for Dataverse. We are currently looking at whether we use Azure Synapse Link to export data from Dataverse to Azure Synapse Analytics or Dataverse to Azure Data Lake Storage Gen2 or both.

    We originally had a need for SQL DB which allowed us to export our data to on-premise data warehouse but this change has highlighted that these older on-premise technologies have no roadmap. So have decided to embrace the cloud technologies and will be moving our data warehouse and lake to the cloud. 

    If you haven't found it already here is Microsoft's document on this https://docs.microsoft.com/en-us/powerapps/maker/data-platform/export-to-data-lake.

    ------------------------------
    Mark Westerman
    Charles Sturt University
    Thurgoona
    ------------------------------



  • 5.  RE: Data Export Service (DES) Alternatives

    TOP CONTRIBUTOR
    Posted Mar 15, 2022 08:25 AM

    We are also using Dynamics Online and configured Azure Synapse to dump Dynamics data to synapse data lake and then using SSIS bring data to On-premise DB. Which is working great so far. In the past there was some differences on how many entity available through synapse and such ...but slowly Microsoft is bringing more and more entity available. Still tables like StringMap and such metadata is not available, but using WebAPI, it is available. We were able to replace most on-premise PowerBI-integration with on-premise to cloud successfully.  Initially, when you configure Synapse, there is window of time period to let everything sync, but later on it is smooth ride. Also there is no troubleshooting ways for synapse links, so if you wants to Resync then basically, unlink and relink synapse. 

    Configuring Synapse Link is very plug and play model....usually Admin with all Azure Tenant Global Administrator rights can configure most part. 

    Bringing your Dataverse data to Azure Synapse – Allan De Castro's Power Platform Notebook
    Create an Azure Synapse Link for Dataverse with Azure Data Lake - Power Apps | Microsoft Docs
    Create an Azure Synapse Link for Dataverse with your Azure Synapse Workspace - Power Apps | Microsoft Docs




    ------------------------------
    Nirav Patel
    Software Engineer
    Spraying Systems Co.
    Wheaton IL
    ------------------------------



  • 6.  RE: Data Export Service (DES) Alternatives

    Posted Mar 15, 2022 09:34 AM
    Hey Nirav,

    Thanks for the Response. I assume you are setting up Data Flows to move data from synapse to sql database. In our case we need to replicate more than 300 tables in which few of them will have more than half a million upserts everyday, Can we achieve this with SSIS? currently we are thinking to setup synapse pipelines to achieve this, so looking for best ways to move forward.

    ------------------------------
    Vikas Gangishetti
    CAPFinancial Partners, LLC
    Raleigh NC
    ------------------------------



  • 7.  RE: Data Export Service (DES) Alternatives

    TOP CONTRIBUTOR
    Posted Mar 15, 2022 09:42 AM
    Edited by Nirav Patel Mar 15, 2022 09:43 AM
    @Vikas Gangishetti
    • Yes we also use SSIS and that is best way so far. There are some data timestamp available in synpase which is utilized by Azure Synpase Sync architecture only (they are not available in Dataverse tables) which we use for Sync.
    • Your number of tables are many, but still should be achievable,.. the amount of processing/power might need some attention.


    ------------------------------
    Nirav Patel
    Software Engineer
    Spraying Systems Co.
    Wheaton IL
    ------------------------------



If you've found this thread useful, dive deeper into User Group community content by role