Buyer beware

When it comes to tools, we localization service providers and software publishers are generally guilty of the tendency to leap before we look. This is evident in the ambivalent feed-back on tool performance seen in LISA Industry Survey Report statistics over the last three years.

While tool use overall seems to be increasing, there also seems to be a decline in the number of users who believe that tools make the process faster and better. Responses indicating that tools improve the quality of the process went from 87.5% in 1996 to 93% in 1997, and to 77% in 1998. Responses indicating that tools help shorten the localization cycle (time-to-market) appear to have fallen as well, from 95.5% in 1996 to 93% in 1997 and to just 68% in 1998 ¹ . Regarding the impact of tools on the cost of localization, we will be waiting to see how 1999 statistics compare to the 65.7% of respondents in 1998 who indicated that tools indeed reduced process costs.

For now, it seems that the more inroads translation technologies make into the localization process, the more demanding users will become that the tools meet their needs.

The black box

As one of the higher-profile tool categories, Translation Memory (TM) bears a heavy burden. Commercially available TM tools have matured to include stable double-byte and bi-di support, greater file format coverage and brand loyalty. Occasionally TM is indeed a miracle-making black box of budget- and schedule-cutting possibilities. Occasionally it is the straw that breaks the back of profit margins and cripples the localization process beyond belief. The most prudent statement tool pundits can make is that TM is a conditionally successful leveraging solution with a long list of "ifs", "ands" and "buts".

The message to TM vendors

Originally developed and marketed as a translator tool, TM is undeniably moving toward enterprise use by localization service providers. Desktop publishing, engineering, estimation and project management resources are using TM technology extensively. The message to tool vendors is exploit this! Re-engineer your tools to accommodate large-scale production processes, which have a hard time with applications designed for small, non-distributed workgroups. Overhead is the enemy, and the more difficult it is to integrate a tool into an existing process, the more resistance there will be and the fewer licenses you will sell. Start with the process, not the feature set.

User or used?

The sad reality is that commercial TM development is not currently process-oriented, and as TM demand increases, workflow yields to technology. As a user, do your tools serve you in the end, or vice versa? With many tools, you have a prep step that you didn't have before you started using them, and this prep step in turn implies a post-processing step. Both preparation and post-processing introduce an additional trouble-shooting requirement that wasn't there before, but one that you may observe, over time, delaying your schedule, increasing your burden and eating into your bottom line. Assess whether your tools work in your process or whether you are constantly redefining your process to accommodate the tools. Understand what this costs.

Process: that ugly word

For lack of anything remotely resembling statistical process control in localization, the success, failure, improvement, or deficiency of the process is based on metrics such as more words or pages processed over time. These types of outcome-based assessments are notorious in quality circles for their inefficiency, as the root causes of a defective process can't be identified, assessed and fixed after the fact. Even if results don't appear to be sub-par, how can you really measure whether any trend in outcome is tool-related, when so many other variables enter into the process?

For instance, factor in the material to be processed. File format plays a huge role in whether a tool yields satisfactory results. Proprietary twists on standard formats (e.g., highly customized HTML) can make a high-performance tool less efficient or entirely useless. Break-even for TM ROI is reached and breached quickly when too much work is needed to make a file format work with a given tool.

The key is benchmarking tool performance by measuring the impact of a tool at every step in your process. To assess turnaround, quantify the speed of work in every phase with and without tools, with different tools and with different processes. For quality, quantify the cost of rework due to tool failure, as well as process improvement due to tool success.

Our dirty laundry

Everyone has horror stories to share about TM tools, so why don't we share them as eagerly as we celebrate and document our success stories? Can't we agree that if there are fundamental flaws in the currently available crop of tools, we will all experience the same pitfalls sooner or later? Continuous improvement of TM technology can only come through industry pressure on tool vendors to produce enterprise leveraging solutions, not just solutions for the lone translator, the isolated file format or the single platform. How many times have you been told that no other customers are currently reporting that problem, or that this is not a defect but an enhancement request? With that type of attitude among tool vendors, is it any wonder that we see an increase in the number of service providers and clients developing proprietary leveraging solutions?

Message to TM users

Take action! Whether we develop our own tools, use commercial tools or both, we can all benefit from:

• publishing our implementation failures
• collecting and sharing our tool performance data
• lobbying tool vendors for enterprise solutions.

Clearly a moderated forum for constructive discussion on tool issues is needed. One venue for this might be the LISA Benchmarking SIG (see also the news item on p. 20), which is coming to life again. One way or another, the dialog on TM tools in particular has to move to the next level, from the anecdotal to the empirical. If the burden is on tool vendors to produce better tools, they need better data from their users.

¹ As stated in the 1998 Industry Survey Report, the quality and time-to-market tool numbers for 1998 may also be attributed to the new "in part" response category. Percentages from 1996 and 1997 were also split into service provider and publisher categories, requiring a less-than-accurate average to put them back together at this time.

Clove Lynch, Translation Tools Manager
Jean-Michel Gouriou, Translation Tools Integration Specialist
Tom Cummins, Translation Tools Integration Specialist
Jim Renwick, Translation Tools Integration Specialist

The Translation Tools Department can be reached at . The
TM lab direct line is +1 (303) 245-7932.