Can I trust PDF validation software?

If I use validation software from different manufacturers I sometimes get different results. Why can this happen? Does it mean that I can't trust the software? What can I do about it? I hear these and more questions very often and I can understand the user's concerns. In this article I try to shed some light on the mysteries of PDF validation.

What is a PDF Validator? That's a software which checks the conformance of a PDF document with a particular specification such as PDF-1.4 or ISO-32000 (PDF-1.7) or ISO-19001 (PDF/A) etc. There are a number of independent PDF Validator suppliers, including Adobe with its Preflight function as part of its Acrobat Professional Edition.

Which checks does a PDF Validator perform? A PDF Validation software usually offers a number of rule sets (sometimes called profiles). The user can select a rule set and check his document for conformance with the particular rule set. The number of rules and the nature of the checks depend on the software. If for example a rule set is named PDF/A-1b this does not mean that all validation products use the same set of rules. There are various reasons for this:
  • Some validators concentrate on the rules given by the ISO standard and neglect the rules of the related documentation (e.g. PDF-1.4 in case of PDF/A-1b) whereas others are going further.
  • The interpretation of an ISO standard is not as easy as it seems and experts may sometime have a different opinion.
  • Some rules are not part of a published ISO standard (e.g. the format of an embedded TrueType font).
  • Some rules might have been forgotten or misinterpreted.
How can I check the quality of a PDF Validator? There are a number of test suites on the market to test the validation software. As an example, the PDF Association released the Isartor Test Suite which is intended to test a PDF/A-1 validation software. The PDF Association is currently developing a test suite to cover PDF/A-2 und PDF/A-3 as well.

If I still have different results even if both validators pass the corresponding test suite, what does this mean? As with validation software a test suite cannot cover all real world cases and is undergoing continuous maintenance to improve it.

What can I do if I experience different results? In this case please send a support request to the involved manufacturers and mention which products you used. This enables the manufacturers to solve the issue in a bilateral way. If the case is more complex and if the manufacturers are members of the PDF Association then the case will most likely be submitted to a technical working group to be solved.

And, to let this article not grow too long, here's my conclusion: 
  • PDF validation software which conforms to the test suites of the PDF Association have good quality and are continuously improving.
  • It is in the interest of all serious manufacturers, especially those that are PDF Association members, to harmonize the validation results.
  • There are still differences in the validation results but these differences are often related to a harmless cause.
  • A user can help to improve the situation by following the process described in this article.
Did this article help? Please let me know and post a comment!