Hi,
I have a few questions regards to Pentaho configurations:
1. I see there are -Xms, -Xmx and -MaxPermSize in Spoon.bat/Spoon.sh. I understand that Xms and Xmx is minimum and maximum memory allocation to Spoon. But I am not sure what MaxPermSize is and what it doing. We have set the xms and MaxPermSize to 8GB and xmx to 32GB. Does this help to boost Spoon's performance? We have 256GB in the server and we split to 4 Spoon services, allocate 32GB each.
2. Carte service is down when Pentaho process large files (10-12million records text files) without any indication and we would need to restart the carte service. How can we improve this? Is Carte responsible to invoke Spoon to run transformations?
3. How can we store the details of the log file into database?
4. The service crashed after processed a large file and not able to process the next file, and we needed to restart the services again. I believe that is because the memory is not fully being release after processed the large file. How can we flush/release the machine memory if the memory has hit the maximum after processed the large file?
5. How can we improve the Spoon performance if the file is too large (10-12million records text files). The duration taken now is around 4 hours. How can we shorten by 40-50% of the duration?
Pentaho exact version: 4.0.1
Appreciate your help on my queries.
Best Regards,
Kar Ho
I have a few questions regards to Pentaho configurations:
1. I see there are -Xms, -Xmx and -MaxPermSize in Spoon.bat/Spoon.sh. I understand that Xms and Xmx is minimum and maximum memory allocation to Spoon. But I am not sure what MaxPermSize is and what it doing. We have set the xms and MaxPermSize to 8GB and xmx to 32GB. Does this help to boost Spoon's performance? We have 256GB in the server and we split to 4 Spoon services, allocate 32GB each.
2. Carte service is down when Pentaho process large files (10-12million records text files) without any indication and we would need to restart the carte service. How can we improve this? Is Carte responsible to invoke Spoon to run transformations?
3. How can we store the details of the log file into database?
4. The service crashed after processed a large file and not able to process the next file, and we needed to restart the services again. I believe that is because the memory is not fully being release after processed the large file. How can we flush/release the machine memory if the memory has hit the maximum after processed the large file?
5. How can we improve the Spoon performance if the file is too large (10-12million records text files). The duration taken now is around 4 hours. How can we shorten by 40-50% of the duration?
Pentaho exact version: 4.0.1
Appreciate your help on my queries.
Best Regards,
Kar Ho