MergeDocument performance

Message Topic Search Topic Options Post Reply Create New Topic Printable Version Translate Topic

   Hi.
I'm loading a 2 page PDF as a form (template) filling it with data and then adding it to a main document using MergeDocument (unless it's the first iteration in which it's stored as the main document). This is repeated until all data has been handled. This has been working fine but now as the amount of data is growing, performance is becoming an issue.

I tried a 71 entry data block, generating a 142 page PDF and for each "page" the MergeDocument took longer and longer time, ending up taking 4.56 seconds. The first merge took about 1/10 of a second.

Any input on how to speed things up would be greatly appreciated.

Thanks in advance,
Clas

PS. To get a basic idea of the code, see my post http://www.quickpdf.org/forum/topic2647.html.

Author	Message Topic Search Topic Options Post Reply Create New Topic Printable Version Translate Topic
ClasG Members Profile Find Members Posts Beginner Joined: 21 May 13 Location: Sverige Status: Offline Points: 7	Post Options Post Reply Quote ClasG Report Post Thanks(0) Quote Reply Topic: MergeDocument performance Posted: 13 Aug 13 at 1:44PM
	Hi. I'm loading a 2 page PDF as a form (template) filling it with data and then adding it to a main document using MergeDocument (unless it's the first iteration in which it's stored as the main document). This is repeated until all data has been handled. This has been working fine but now as the amount of data is growing, performance is becoming an issue. I tried a 71 entry data block, generating a 142 page PDF and for each "page" the MergeDocument took longer and longer time, ending up taking 4.56 seconds. The first merge took about 1/10 of a second. Any input on how to speed things up would be greatly appreciated. Thanks in advance, Clas PS. To get a basic idea of the code, see my post http://www.quickpdf.org/forum/topic2647.html.

AndrewC Members Profile Find Members Posts Moderator Group Joined: 08 Dec 10 Location: Geelong, Aust Status: Offline Points: 841	Post Options Post Reply Quote AndrewC Report Post Thanks(0) Quote Reply Posted: 14 Aug 13 at 2:45AM
	Clas, Which version of QPL are you using ? The filedropper link to your file is no longer working so I have no idea how large/complex your PDF file is. It would be helpful for me to be able to test with the actual file so it would be good if you could post an updated link in this thread. Andrew.

ClasG Members Profile Find Members Posts Beginner Joined: 21 May 13 Location: Sverige Status: Offline Points: 7	Post Options Post Reply Quote ClasG Report Post Thanks(0) Quote Reply Posted: 14 Aug 13 at 10:37AM
	Hi Andrew. We're using 9.14 at the moment. I've re-uploaded the file. It got the same ID so the link works again. The PDF's got some 200+ fields. Thanks for your effort, Clas

AndrewC Members Profile Find Members Posts Moderator Group Joined: 08 Dec 10 Location: Geelong, Aust Status: Offline Points: 841	Post Options Post Reply Quote AndrewC Report Post Thanks(1) Quote Reply Posted: 14 Aug 13 at 1:39PM
	Clas, I have optimised your PDF file a little by removing some of the fonts. Many of these fonts were not being used. Some of the formfields were using Verdana font but I have changed this to the embedded Helvetica version so the fields will look slightly different but the file size has been reduced considerably. If you prefer to use Verdana then all the t formfields would all need to be changed to Verdana. This can be done quite quickly in Acrobat Pro. http://www.filedropper.com/fk3059stripped I would be interested to see how much improvement this will make. Also - Are yo loading the PDF's from a file or from a stream ? Are you merging the documents in memory ? If you are using the disk then you could use MergeFileListFast to merger 50 or so files together in one operation. Andrew.

ClasG Members Profile Find Members Posts Beginner Joined: 21 May 13 Location: Sverige Status: Offline Points: 7	Post Options Post Reply Quote ClasG Report Post Thanks(0) Quote Reply Posted: 27 Aug 13 at 1:40PM
	Thank you so much for your help. The modified file cut the time for a 162 page document from 4 min 34 sec to 2 min 44 sec . A great improvement. Unfortunately this is still too long. We've been using Crystal Reports to generate the reports before, but when the issuer of the form to fill decided to change the layout again, we decided to try to use their original (PDF) directly instead of re-creating it in CR, and therefore have started using QuickPDF. The problem is that the customers are used to the time to generate the reports that CR needed, and that's under 20 sec for (roughly) the same data. It also gives us trouble because the customers web-servers normally time out after 2 minutes. So I've been working on implementing a asynchronous asp page to solve the time out problem and that now works. Problem is that the customers still can't understand why reports take almost ten time as long to generate. Thanks again for your big help Andrew. But we're still hoping there is some way of improving performance additionally. Either by optimizing our code or if the MergeDocument could be optimized some how...

ClasG Members Profile Find Members Posts Beginner Joined: 21 May 13 Location: Sverige Status: Offline Points: 7	Post Options Post Reply Quote ClasG Report Post Thanks(0) Quote Reply Posted: 02 Sep 13 at 9:32AM
	I happened to see that there had been performance enhancement to MergeDocument in the 9.15 version. I upgraded and tried again, and it almost cut the time in half again . This is still to slow though (I know - whine, whine, whine... ). So I rewrote the code and used MergeFileListFast as Andrew suggested. It's not ideal since the code runs on a web-server and you never know about security rights... Is the w3wp process allowed to create files...? It works on the machines I tried so far at least. Now the original 4 min 32 sec is down to... Wait for it... Wait for it... less than 20 seconds . Hurrah for MergeFileListFast . It would be great to have the same speed (or at least close to it) in MergeDocument. Or the possibility to do the same with variants - MergeVariantListFast. That couldn't be to hard to implement . Thanks again Andrew for all your help. Edited by ClasG - 02 Sep 13 at 9:34AM