Textextraction: Determine the used codepage

Hi Ingo,

the PDF-Spec gives no Results if I search for "codepage". Therefore it is not so simple. But there is an explanation in 5.5.5 Character Encoding. I hope this is the key.

To verify this, it is necessary to have a longer look in an inflated encoding dictionary and an entry in it. For a such test a PDF with codepage 1251 is needed.

This means a longer way thru some objects. Can you mail me a such PDF?

Erläuterungen natürlich besser für mich auf deutsch.

Cheers,

Werner

Extracting text is not an easy process.   Each font can either be defined as single byte or multi byte font and they also contain an Encoding entry in the CMap which is similar to a code page.  To make things more complex a font can also contain a ToUnicode mapping array to help with text extraction routines.  On top of that a font can contain a Differences array which can remap any character code to a new code.

Hi Ingo, Hi Andrew,

the goal is not to paint a glyph correctly, therefore it is enough to figure out  the use of the codepage 1251 with its charset.

Here a quick&dirty solution without to much work:

function IsQP1251(fn: string): boolean;
var
  QP: TQuickPDF;
  obj: string;
  i, n, glyphno, error, p: integer;
begin
  Result := false;
  QP := TQuickPDF.Create;
  if QP.UnlockKey({$I PDFkey.inc}) = 1 // 8.xx
  then begin
    QP.LoadFromFile(fn, '');
    n := QP.GetObjectCount;
    i := 1;
    repeat    // search for an PDF-object /Encoding with afii-codes for cyrillic inside  
      obj := QP.GetObjectToString(i);  // afii10017-afii10846
      if (Pos('/Encoding', obj)>0) and (Pos('/Differences', obj)>0)
      then begin
        p := Pos('afii10', obj);
        if (p>0)
        then begin       // 17..846
          Val(copy(obj, p+6, 3), glyphno, error);
          Result := (error=0) and (glyphno>=17) and (glyphno<=84 6);
        end;
      end;
      inc(i);
    until (i>n) or Result;
  end;
end;

This works under Delphi7 with QuickPDF 8.13

The basic idea is:

inside the PDF is an encoding-object, which may start so:
/Type /Encoding /BaseEncoding /WinAnsiEncoding /Differences
So I go over all objects and look for /Encoding and /Differences.

The /Differences-array is the key. There are strings starting with "afii" followed by a number. They denote a special interpretation of the value of a byte.

It is dirty, because I am not sure about is there ever a such /Encoding-entry in the PDF.

It is dirty, because I go the short way directly to an object. Better to go along through the tree (/Procset => /Font => /Encoding), but the afii-codes are rather unique in conjunction with /Differences.

I dont know, may this Encoding-object be compressed too? In this case it would be more work, another reason for my wish (case 9605) for a GetInflatedObjectTo... as great brother of GetObjectTo... used in the solution above.

An overview you may get here
http://partners.adobe.com/public/developer/en/opentype/aglfn13.txt

A good source is
http://www.adobe.com/content/dam/Adobe/en/devnet/font/pdfs/5013.Cyrillic_Font_Spec.pdf
too.

The solution works with the two examples I got from Ingo. Maybe there is more to do. But it is a first entry.


Cheers and merry christmas

Werner


Edited by edvoigt - 23 Dec 11 at 12:42PM

Hi,

it seems to be a good source here:

http://www.science.co.il/language/locale-codes.asp?s=codepage

There are some other forwarding links.

And you need the relation between afii-codes and copepage.


May it helps.

Werner