Hi, I'm using Pentaho BA server 5.0.1 CE and I have nightly scheduled Kettle jobs to move data from our production database to a data mart for reporting. Today I was making changes to one job, an incremental ETL that always loads data for the previous day, and I noticed that it didn't trigger for the last two days. Upon examining the data in the reporting database further, I found out that there are two days worth of data missing.
Although the xaction that runs the Kettle job is scheduled to run daily, Scheduler Admin tells me that it was last run two days ago. I checked the logs and the server was up during the night and other scheduled jobs ran OK. There's no error in the log, no anything, because the scheduled xaction simply didn't trigger at all.
There's not much I can do about it now except fix the recent missing data ad-hoc and live with the fact that older data will be lost (as they get periodically deleted from production DB). I guess the blame is partly on me for making the ETL "naive" and don't check what was imported and what wasn't. However, fixing this doesn't really solve the bigger problem - users expect their reporting data to be up-to-date and not three days old when Pentaho randomly decides not to trigger its scheduled jobs. Any ideas will be appreciated!
Although the xaction that runs the Kettle job is scheduled to run daily, Scheduler Admin tells me that it was last run two days ago. I checked the logs and the server was up during the night and other scheduled jobs ran OK. There's no error in the log, no anything, because the scheduled xaction simply didn't trigger at all.
There's not much I can do about it now except fix the recent missing data ad-hoc and live with the fact that older data will be lost (as they get periodically deleted from production DB). I guess the blame is partly on me for making the ETL "naive" and don't check what was imported and what wasn't. However, fixing this doesn't really solve the bigger problem - users expect their reporting data to be up-to-date and not three days old when Pentaho randomly decides not to trigger its scheduled jobs. Any ideas will be appreciated!