Skip to main content
System StatusContact Support
VersionOne Community

How to run the Datamart ETL/Dataloader

This article is for local installs only. It does NOT apply to hosted/SaaS instance. If you have an issue with data in a hosted/SaaS instance, then please contact Support right away! 

 

The Datamart dataloader, or ETL, is the part of Analytics that reads the Lifecycle database and updates the Datamart so that you have current data in your reporting database. Some things that are updated when the dataloader is run:

  • Users
  • user permissions/roles
  • new projects added
  • projects updated
  • themes, features, backlog groups, backlog goals
  • custom fields
  • well, let's just say EVERYTHING that is in lifecycle...

So with that said, it is important that the dataloader/ETL is run at least daily on production systems or the data will quickly fall behind, and then you are looking at reports that are no longer relevant.

To run the dataloader/ETL, go to the Datamart folder on the Datamart webserver. The folder will be named after your datamart instance like this:

{VersionOne instance name} to {VersionOne instance name}-datamart

So if your instance name is VersionOneProd you would have this folder: VersionOneProd to VersionOneProd -datamart

Go to that folder and look for VersionOne.DatamartLoader.exe

One thing to note is that there is ALSO a file in there named VersionOne.DatamartLoader.exe.config. If your file manager is hiding the file extensions, then you would see that config file as "VersionOne.DatamartLoader.exe" and think that is the one to run. The only way to tell is that you would have a file right there above it named "VersionOne.DatamartLoader". See? The file extension is hidden. To get your file manager to show file extensions, use a search engine such as Google to find out. Just Google "Show file extensions".

Now, once you have the file, you can do 1 of 3 things.

  1. You can double-click that file right there and it will run.
  2. You can open a command prompt at that location, and then type in the name and let it run.
  3. You can set up Windows Task Scheduler to run it automatically every day.

Obviously we recommend using Option #3.

About Windows Task Scheduler

However, you should NOT tell the task Scheduler to run that EXE directly. This is a known bugaboo with Windows task scheduler. What you SHOULD do is use a batch file that calls the executable. So you have task scheduler called that batch file which calls the executable. That way the dataloader/etl is run as its own process and not as a child process of the task scheduler, which has been known to cause problems with our dataloader. The batch file will look like this:

 

@echo off
rem ***************************************************************************
rem 
rem batch file to launch the VersionOne ETL 
rem 
rem ***************************************************************************

setlocal

rem ***************************************************************************
rem You need to set these two environment variables based on the ETL location
rem ***************************************************************************
set ETL_DRIVE=C:
set ETL_HOME="\Program Files\VersionOne\VersionOne to Version-datamart"

rem ***************************************************************************
rem this is where all the action happens :)
rem ***************************************************************************
%ETL_DRIVE%
cd %ETL_HOME%
VersionOne.DatamartLoader.exe

endlocal

Use THIS LINK for a downloadable copy of this batch file. You will need to remove the .txt file extension prior to using it.

After that, the only thing you need to edit is the ETL_HOME folder location. You would use the location of the VersionOne.DatamartLoader.exe  file. 

Then set up the task scheduler to call the batch file name.

To monitor the dataloader activity and make sure it is running, you can look at the dataloader log file that will be in that same folder location. The file is called dataloader.log. Each run will append to that file, so you can scroll to the bottom of that file to see the last run. If you see anything other than a message that looks like this:

2016-07-28 06:04:12,916 [1] INFO  - Total time: 00:53:15.0410484

Then please log a support ticket and send us that file.

 

 

     

 

 

 

 

 

 

  • Was this article helpful?