Focus on Standards

As pointed out in LISA's review of Umberto Eco's Mouse or Rat? Translation as Negotiation, most "quality problems" in localization have little to do with mistakes, and more with a mismatch of assumptions and goals between the people requesting a translation and the people supplying the translation. Eco's book holds that the key to successful translation is really in the ability to successfully negotiate between competing demands to find the translation that fits a particular situation and represents the best tradeoff between factors that cannot all be simultaneously met. In the localization industry, we tend to ignore this reality, until we find that our assumptions do not match, and that project quality has suffered as a result. ASTM WK2953, by focusing on the earliest stages of localization, attempts to address the issues of requirements and mismatches of expectations before they become problems.

Most quality problems in translation and localization have little to do with real errors, and more to do with clashing assumptions.

Before looking at ASTM's contribution to localization quality, let us look at OSCAR's GILT Metrics eXchange (GMX). Then I will suggest how GMX and the ASTM translation project might fit nicely together.

One of the newer OSCAR initiatives is GMX (short for GILT Metrics eXchange), led by Andrzej Zydroń. Those of you who have been diligently reading the Globalization Insider are aware of one of the three aspects of GMX: GMX-V, which standardizes word and character counts (that is, Volume) for source and target texts and provides a place to record these counts. GMX-Q, on the other hand, provides a place to identify a quality metric for the particular translation/localization project at hand. A metric provides a precise definition of quality and a method of quantifying it. Of course, there is no universal metric of translation quality. The question explored in this article is how to define a general framework for metrics. The third aspect of GMX (GMX-C) provides a place to quantify the complexity of a translation/localization project. GMX-C should be considered in parallel with GMX-Q, since the complexity of a project is intertwined with customer requirements.

SAE J2450 was developed for the automotive industry. But suppose you are in an industry other than automobiles. Where do you find a suitable metric?

One example of a quality metric in translation is SAE J2450 (see Mission Impossible: Improve Quality, Time and Speed At the Same Time by GM’s Don Sirena for one view of SAE J2450). This metric, which has been under development since 1997, is targeted at the automotive industry and applies only to documents of one text type (service information), one particular automotive audience (service technicians, not consumers), and one specific purpose (to allow technicians to perform their duties accurately and efficiently). In the J2450 metric, specific aspects of a target text are categorized as errors of varying degrees of severity and given points. The accumulated points for a translation provide a numeric indicator of its quality.

But suppose you are in an industry other than automobiles. Where do you find a suitable metric? For software localization, the LISA QA Model 3.0 is an excellent tool for creating metrics. Various categories of errors in the software itself (that is, in the messages, menus and prompts) and in the accompanying documentation (on-line or printed) are assigned point values corresponding to severity, and an overall error threshold is established. Once the LISA QA Model has been tailored for a particular project or set of projects, it becomes a metric.

There is a need for a general framework within which quality metrics can be defined for multiple audiences, purposes and text types

While the LISA QA Model can be used to generate specific localization-related metrics, there is a need for a more general framework within which metrics can be defined for multiple audiences, purposes and text types. Fortunately, such a framework is under development within ASTM International. And there is still time to make your voice heard to influence the final draft.

Before describing the ASTM framework, I note the current effort within CEN (The European Committee for Standardization) to develop a standard for translation services ( CEN/BTTF 138 ). The scope of the current draft of CEN/BTTF 138 is to "specify the requirements for translation service providers." One of those requirements is "documented quality management," but no particular translation quality metrics are included.

ASTM work item WK2953 aims at developing a guide "designed to provide a framework within which the participants in a services agreement can define the processes necessary to arrive at a product of desired quality to serve the needs and expectations of the consumer." The ASTM translation guide is not a metric, but it is sufficiently flexible that many metrics could be developed using it, each one corresponding to what is called a set of specifications. For those familiar with the Skopos approach to translation and its recent elaboration by Christiane Nord (Translating as a Purposeful Activity: Functionalist Approaches Explained. St. Jerome Publishing: Manchester, UK. 1997), the specifications for a translation project correspond to what Nord calls a translation brief Specifications define customer satisfaction requirements..

It might even be possible to partially automate the process of developing metrics by generalizing on the LISA QA Model.

In the current draft of the ASTM translation guide, there is a section on selecting a translation service provider that overlaps with the current draft of CEN/BTTF 138; however, the focus of the ASTM guide is on the items that need to be addressed in developing a set of specifications. These items are grouped into three sections:

• information about the source text, including its subject field and text type;
• decisions to be made about the target text, including what to do with dates, monetary values, proper names and how to format the text; and
• procedural specifications such as what termbase and translation memory to conform to.

No absolutes are stated. Instead, each item involves a choice among potential specifications. Once all the specifications for a project or set of projects have been agreed on and documented, a matching translation metric can be created or retrieved. The beauty of using a framework such as the ASTM guide is that all the metrics based on it will share a common terminology and organization. It might even be possible to partially automate the process of developing metrics by generalizing on the LISA QA Model.

There seem to be only three options for those who want to pursue the GMX-Q project:

• reduce GMX-Q to a place for recording an unambiguous link (such as a webpage address) to a translation quality metric, with no attempt at coherence among metrics;
• adopt the ASTM translation guide as the GMX-Q framework for metrics; or
• develop a new framework for metrics.

Note that both GMX-V and TBX are examples of OSCAR standards that depend on non-OSCAR standards. TBX depends on ISO 16642, and GMX-V depends on XLIFF. Standards are cooperative endeavors.

Before deciding among these three options, I strongly encourage you to consider (1) joining ASTM F15.48 and/or OSCAR or (2) contacting the author about joining the BYU Translation Research Group and providing input through that channel. Why re-invent a framework for metrics when the ASTM guide might be sufficient with a little refinement?

Standards are cooperative endeavors.

ASTM influence extends far beyond the borders of the United States: Here is a quote from the ASTM website (Under About ASTM): "Standards developed at ASTM are the work of over 30,000 ASTM members. These technical experts represent producers, users, consumers, government and academia from over 100 countries. Participation in ASTM International is open to all with a material interest, anywhere in the world."

The next major review of the ASTM guide will be conducted during the second half of March 2005, so please act now.

To join ASTM: Contact Kathy Morgan (kmorgan@astm.org) about joining ASTM F15.48, which involves becoming a dues-paying member of ASTM.

To join OSCAR: Contact Arle Lommel (arle@lisa.org).

To join the BYU Translation Research Group at no cost: Contact Alan K. Melby (akmttt@byu.edu).

Alan K. Melby is Director of the Brigham Young University Translation Research Group, a Provo, Utah-based think-tank dealing with translation-related issues. A former chair of the LISA OSCAR Committee, Melby is a professor of Linguistics at Brigham Young University. He is currently serving as chair of the Translation and Computers committee of the American Translators’ Association (ATA), and is involved in research on the use of feature films for language training.