Neste Engineering Solutions (NES)
NES switched to modern cloud-based data warehouse – load chains run smoothly with ADA
Neste Engineering Solutions (NES) decided to move its on-premise data warehouse to cloud, and chose its long-time partner Epical to carry out this transformation. At the same time, NES upgraded its tool for managing load chains to Qivada’s Analytics Development Accelerator (ADA).
The support for NES’s data warehouse solution – an on-premise system built around SQL server – was ending, and the company decided to move the system into Azure infrastructure.
Part of this transition had to do with managing work flow. Previously, NES was using Epical’s on-premise solution, Execution Manager, for this purpose. Now it needed a modern tool for that as well.
“Execution Manager had become slow and caused some trouble in reporting. To solve this, NES opted for ADA, which did not have these problems”, says Mika Koivisto, Epical’s developer at NES.
Successful migration project
The migration from Execution Manager to ADA was a smooth process, with the help of a migration tool created for this purpose.
“The migration tool saved us a lot of work. We used it to copy the dependency structure directly from the Execution Manager, including all package definitions. During this process, we also updated the dependencies a little. The monotonous definitions were done automatically, and only the load chains were created manually”, Koivisto explains.
The migration included NES’s SQL and SSIS servers. “The servers were slightly modernized and they are now running in Azure’s cloud. ADA enables turning them on and off according to the user’s needs. NES hasn’t started to use this functionality yet, but it is on our to-do list and could bring some savings”, says Epical’s Senior Consultant Tomi Alanappa.
Before granting a permit for ADA, NES did a technical audit to assure the security of this tool, among other things. Once the permit was given, the migration project was carried out fast and successfully.
Ideal tool for managing load chains and deployments
ADA enables defining dependencies for each data load individually, instead of building load chains one level at the time. For the user, this is a meaningful way of managing load chains.
“ADA offers vast possibilities for managing load chains and doing exceptions. The entire network of dependencies can be visualized, and it’s easy to monitor the loadings and get an overall understanding. You can also search any loading easily or eliminate a certain loading from this network”, Alanappa explains.
“One great feature is that if a load chain collapses, you can continue directly from the point of collapse after the problem is solved. This enables fast recovery”, Koivisto adds.
NES is using ADA for deployments from test environment to production as well. “This is easy with ADA, because you only have one place where you do the specific definitions for each environment. Managing the different environments is straight-forward”, Alanappa says.
Improved user experience
The goal of this migration project was to improve the user experience at NES. ADA is fast to use, and it offers the end-users a simplified user interface.
“The feedback from the users has been positive. ADA works much better in the Azure world than Execution Manager, and the users can easily find the functionalities and run the loads they need”, Koivisto says.
“With ADA, it is easy to handle the access rights. We can provide a view where the user only sees the necessary loadings. This eliminates extra information from the background, which would only disturb the user”, Alanappa points out.
Possibilities for further development
At the moment, NES is using just one of ADA’s three features, Work Flow. ADA’s Data Flow or Data Editor are not in use, but this might change in the future – for example by using ADA for data loadings in Azure’s Data Factory.
“The basic use is running well and the customer sees the improvement in the user interface. The loadings are carried out with the same efficiency as before, but with less processor cores. This brings the license fees down also. The door to the Azure world is now open for further development”, Koivisto finishes up.