Customer Engagement & Dynamics CRM Forum

Expand all | Collapse all

Making the leap from On Prem to Online... some questions

  • 1.  Making the leap from On Prem to Online... some questions

    Posted 18 days ago
    Hello all.  So we've been On Prem since rolling out MS CRM about 6 years ago, but  just got word that we'll be moving online at some point in the near future.  My biggest question is with integration into other systems.  I have several processes that sync our data in CRM (yeah I still call it that...) to other systems and databases, and I do this by copying the data to a Working database on the CRM SQL Server and comparing the data, then from this I can determine what needs to be created, updated, or marked as inactive.  Since moving online I will not have access to our data like this can someone give any suggestions on how to do this?  I'm still not sure what capabilities we'll have, but I am requesting that we have an Azure Dataverse or Data Lake or whatever it's called so we can view our data in an Azure database.  Also we generally use a hash of the data to compare. Another option I'm testing, we create a checksum hash of the data that could change and just compare this instead of every field, so I'm thinking of keeping this hash along with the entity Guid and business key locally in SQL and using this to compare.  As it creates new entity records in CRM it'll update the local SQL table with the new GUID for future compares.

    Anyway just trying to see how others do this.  We can often have 50K to 100K changes each day, and with all this now going over the wire and not just to a local database i'm trying to find the best way to do this.  Thanks for any advise.

    ------------------------------
    Sam Alexander
    Lead DBA Applications Analyst
    Texas Farm Bureau Insurance
    Waco TX
    ------------------------------


  • 2.  RE: Making the leap from On Prem to Online... some questions

    TOP CONTRIBUTOR
    Posted 15 days ago
    Sam,

    The Data Export Service described here: https://docs.microsoft.com/en-us/power-platform/admin/replicate-data-microsoft-azure-sql-database may be able to fill your need. It describes a one-time export and then continual exports of changes. I haven't used this feature, but it looks promising.

    Also, there is a preview feature available to connect to the Dataverse database using SQL Server Management Studio v18.4 or greater by enabling the TDS protocol endpoint. Info here: https://docs.microsoft.com/en-us/powerapps/developer/data-platform/dataverse-sql-query 

    The SSIS Kingswaysoft connector is a good tool for doing DTS operations.

    I initially really missed having direct access to our database when we went online, but over time the tools and options have improved.

    ------------------------------
    Andy Arndt
    Minitab, Inc.
    State College PA
    ------------------------------



  • 3.  RE: Making the leap from On Prem to Online... some questions

    SILVER CONTRIBUTOR
    Posted 14 days ago
    Edited by Kevin Harrington 14 days ago
    Hi Sam,

    With that many daily updates, keep in mind the number of API calls you'll have to make. Also, would there be an option to possibly use virtual entities? As db storage is particularly expensive online.

    We currently use kingswaysoft but we need to redesign this so it's a very good topic. I've read of those using Azure data factory (online version of ssis) and being happy with it. We also have the complication of users wanting near real time data for certain datasets.

    ------------------------------
    Kevin Harrington
    ------------------------------



If you've found this thread useful, dive deeper into User Group community content by role