Data to Dashboard with the Integration Platform

On September 27, 2016, Syncfusion Product Manager Chad Church presented Data to Dashboard with the Syncfusion Integration Platform, a webinar showing how to employ all the power of the Syncfusion Integration Platform from data integration to data exporting, which can be delivered to the Syncfusion Dashboard Platform for data visualization.

The end of the webinar was open to audience questions, which are presented here with answers.

Can I create a REST service to receive JSON data so that I can process it and store in a SQL database?

Yes. This is quite simple to do. You can create a REST service using any technology that meets your needs, and you can have the JSON data automatically processed by a workflow running inside the Syncfusion Data Preparation Platform. You can also use GetHTTP as a processor. This can poll a URL and write results to a file for further processing.

Can one create forks and joins in the workflow?

Yes, absolutely. Most things you can do in code can instead be accomplished using the configuration system. This makes the system powerful and easy to maintain.

Do we have the capability to be a “sort of” database source from any of these processes that can be read using SQL in a subsequent process?

Yes. You can merge from any source and process into tabular format. You can then store into a SQL store for exposure downstream.

Can I create a process to read a SQL database and post JSON to AWS?

Yes. Your workflow will do the conversion inside the Syncfusion Data Preparation Platform. The final step will call InvokeHTTP, or something similar, to post to AWS.

Can you call any custom service to post data without a problem? It is possible to store it in an intermediate SQL store, but I would like to know if the process itself can mimic a SQL store.

Not that we know of. It will be possible to have a custom SQL processor implemented, which will make the caching transparent. We can implement internally with SQLite or something similar. Alternatively, you can also push into a Hive table store to process SQL that way.

With regards to reading SQL databases, should I create a cycle—for example, reading the database each five minutes—or is there another flow to start the process?

Yes. This is configurable.

Can you read or write SQL Azure?

Yes.

How do you add your own processes?

Typically, you implement a custom processor using Java. You can also use a Java template and invoke C# to do the actual work. Syncfusion can help through our Direct-Trac support system.

What’s the next step you recommend in learning how to design with the Data Integration Platform?

We recommend that you download the beta version and play with the samples to get a feel for it. It is pretty straight forward to work with and start using for even complex ETL.

What limitations does the free license have?

Our free Community License is for companies with less than a million dollars (US) in annual revenue.

As a developer, can I get it free?

Yes, absolutely.

When would you use this tool versus SSIS?

Both are comparable. Licensing is the main difference.

Is it possible to define a data model that is filled using different processes—data flow metaphor versus process flow?

Yes.

What are some things to take care of during installation?

It installs with MSI. No configuration or special requirements are needed.

Can I use DB2 instead of SQL Server?

Yes. All JDBC sources are supported.

What are the costs for the production environment when a developer has a free license?

There are no production fees. If you qualify to use it, you qualify for production too. There are no run-time fees or distribution fees either.

Can the processes be triggered by request (e.g., from a dashboard)?

It is completely configurable.

Is SharePoint supported?

Yes. It is supported through standard API points.

How we can manage different types of data in a database? For example, if I’m working on a website that will contain different game scores—like football, cricket, etc.—the games will have different styles for scoring. How do I save them in one database?

Typically, the OLAP data source will be custom-designed to accommodate all your OLTP data in a form that can be easily consumed.

The entire webinar can be viewed at https://youtu.be/ST0u5QyDFmc.