DASetTextExtractionWordGap

I'm using library version 8.13 and I'm extracting text from pdf's using DAExtractPageText.  I've run into a few documents that aren't extracting quite the way I'd like using using an "Options" setting of 3 (3 = Return a CSV string for each piece of text on the page with the following format:  Font Name, Text Color, Text Size, X1, Y1, X2, Y2, X3, Y3, X4, Y4, Text).  It's putting some "words" together that shouldn be separate.  In order to address this issue, I figured I'd use the fairly new DASetTextExtractionWordGap function to try and clean things up a bit.

Unfortunately so far, I haven't been able to get this function to have any impact on what's being extracted at all.  Has anybody had any success using this command and if so, what sort of wordgap values were you using?  By default I've been using 0.7, which I _think_ is what the default is, but I'm not 100% certain of that anymore.  Adjusting that value both higher and lower seems to have no impact.  What's the trick to getting this to work?

Thanks!

AndrewC wrote: There are some new PDF's that have become more common lately that have no space character defined when the text is drawn.  The PDF is just placing the word in their correct location and is not relying on the space character.  This forces Quick PDF Library to guess where the spaces are and this gets quite tricky.

Trust me, I know this quite well. :)

I went ahead and redid the documents I was working with using the word-based extraction and put all the "phrases" together myself, so I have some idea what a pain it is.  My advantage though was that I could tailor the spacing for my specific need, so it was less likely to put spaces where it shouldn't or vice versa.  Still, I prefer to use the built-in version when I can as it makes things easier on my end.

We have added some improved code for this into the 8.14 betas and it fixes most of these problem PDFs.  Quick PDF Beta 4 can be downloaded from  - http://www.quickpdflibrary.com/blog/2012/02/quick-pdf-library-8-14-beta-4-released/ I will be interested to see if this fixes your problem.

Thanks!  I'm doing a new set of documents that are having similar issues, and I'll give it a try.


Edited by Papajin - 11 Feb 12 at 8:42PM

I am decoding some pretty complex, high-density text pages.  Library version 8.13 did a pretty good job, but not perfect, so I upgraded to 8.15 today.  8.15 is definitely better, but there are still some puzzling aspects to the ones that didn't come out clean.

I can't get any change in the output by using SetTextExtractionWordGap or the DA version, thinking that there might be some difference in internal processing.  No matter what I set the wordgap to, it gives the same results.  I see two likely interpretations.  First would be that the processing never got to a point where it needed to examine a string based on word gap.  The other would be that it is getting tested, but not correctly.

I can certainly provide the test file, but I can describe the problem areas as ones (in a pretty small font) that appear to Adobe Reader (and also QPDF rendering) as:

"             M  J  J  A  S  O  N  D  J"        
"to Buy   0  0  0  0  0  0  0  1  0"
"Options 0  0  1  1  0  5  3 10 1"
"to Sell    0  0  0 1  0   0  2  0  1"

There are some minor differences because of trying to show this in the Forum font, but you get the picture.  This gets extracted as:

"M J JASONDJ"         - as a single string entry.  The related lines below it are interpreted as
"to Buy 000000010"
"Options0 0 1 1 0 5 3101"
"to Sell 000100201"

Because of the interesting interpretations of these as strings, it looks to me as if it is trying to do something with a word-gap type of processing.

Another oddity is that the string returns are very slightly different between mode 0,3 and 4.

mode 0, the string comes back as "Options 0 1 1 0 5 3101"
mode 3  returns two strings: "Options" and "0011053 10 1"
mode 4 returns four "Options" "0011053" "10" and "1", and these stay the same no matter what I set the word gap to.

Interestingly, "to Buy" gets correctly broken up by mode 4 into "to" and "Buy", but the gap between those two word is visibly quite a bit less than the gaps between the digits on the rest of the line.

One possibility is that the processing doesn't like single-character "words" and assumes they have been kerned, or somehow spaced out, but still belong together.

I'm not sure what all this means, but it is one of very few flaws in otherwise very impressive processing.

Thanks - Howard

I gets very difficult to try and determine what the correct width of a space character.

Hi Andrew - I really hate to ask, but Google searches on pdf internal structure are turning up nothing useful.  I just need some sort of clue as to how to parse ExtractFilePageContentToString results.  Is this pretty much the internal pdf structure, or is this more of an intermediate level of QPL processing? 

 It looks a lot like each line is a separate command element and the last two characters on the line are a "command".  The rest of the line is probably decoded based on the "command"

I don't know if you recall "magic pictures" of the 1980s or 90s which are a page of various dots, and if you stare at them a certain way, you see a picture.  That is what this seems like.  I can almost see the structure, but not quite.

If I am on the right track, please just reply "yes".  If you have a handy link, I would appreciate it, but this is way beyond anything you should have to spend any time on.  I am pursuing this to keep my brain exercised - it just "feels" like something useful.

Thanks - Howard

PS. I did locate some of the strings in question and I see the difficulty in interpreting them.