Constant and synchronous data along the lines of standard solutions for SMEs

Companies often face the problem of using a standard software solution for various departments, which is well-suited to their needs. However, as the company grows or undergoes organizational changes, the manual exchange of data between these standard software solutions becomes increasingly cumbersome, and the potential for errors grows. Errors and inconsistent data can reduce or even jeopardize efficiency and customer service, and consistent data management is no longer guaranteed in the long run, leading to increased reporting efforts. A customized tool for synchronizing these different software solutions can provide a solution, ensuring that customer information and other data are consistent across various systems.

What is the best course of action?

To make synchronization of standard solutions meaningful, a few clarifications must first be made to determine the scope and an appropriate implementation:

• What type and quantity of data should be synchronized?
• Which systems are involved?
• How quickly do the data need to be synchronized?
• What can be used to identify or compare the data together?
• Why should the data be synchronized and what are the potential risks?
• Which is the leading system?

Once these six questions have been answered, you can begin clarifying the technical details. The analysis will focus on data access and the software's API (interface). Each question is important to ensure that the synchronization interval, the amount of data and changes, and the technologies used are optimally aligned with the customer's needs. The objective, such as saving time or reducing duplicate data entry, should always be considered during the analysis.

Which technologies should be considered/used?

Once all necessary clarifications regarding the interface of standard solutions have been made and an overview of the data has been obtained, it is important to evaluate which technologies should be used. Whether a webhook or a console application that accesses an API is used always depends on various factors.

Excursion & Example

A webhook means that the recipient receives updates directly from the source. You "hook" me as the recipient and receive updates from the source. In a console application, the target system retrieves the data from the source.

If there is a large amount of data and the changes cannot be sent via a webhook, further measures must be taken to read the changes quickly enough. This can be achieved through asynchronous programming, parallelization, splitting the synchronization, or limiting the data to be checked. This only works with good communication and an understanding of the customer's individual goal.

What should be taken into account and where should special attention be paid?

Especially when large amounts of data need to be synchronized, the interfaces of standard tools like elastic.io, mulesoft, dell boomi, etc., can reach their limits. This is particularly true when data records need to be queried or validated individually, requiring thousands of queries. It's crucial to ensure that the software solutions aren't overloaded during synchronization, which could hinder or prevent the user from working effectively. We offer a solution to this problem by reducing the number of queries to a single subsystem, ensuring that work remains uninterrupted during synchronization.

Similarly, if a large amount of data is being synchronized, it can be a hurdle if individual queries fail to synchronize. This can happen due to a network error, temporary server overload, or other server-side errors. Since this leads to an application error, conventional tools usually simply skip it, and the data is not synchronized. However, it is better to define a "retry policy" and specify that queries are resent after a defined interval. This ensures that synchronization occurs or that you are notified if synchronization is interrupted due to server errors. 

In contrast, with a custom solution, a suitable implementation path can be found, whether it involves connecting APIs on both sides or, for older applications, exchanging data via files. While standard sync tools like MuleSoft, Dell Boomi, etc., generally achieve good results and can build upon existing infrastructure, they incur ongoing costs and create a high degree of dependency on the solution. Therefore, careful planning and analysis should be considered beforehand.

We would be happy to present the advantages and feasibility of data synchronization with your existing standard software solution and explain our detailed procedure in a personal consultation. Contact with us.