This year, Linguistics Instructor Ali Saeedi  received an SLCL Summer Fellowship for Specialized Faculty because of his work “The Race for Quality: Comparing Neural Machine and AI-Powered Translation Performance in the American Translators Association Exam”.  Congratulations to Ali for this award!

To learn more about Ali's research, take a look at his answers to the questions below:

Can you tell us a little bit about the project?

The project, titled "The Comparison of Quality: Neural Machine Translation versus Large Language Models Powered Translation in the American Translators Association Exam," seeks to delve into the evolving role of artificial intelligence in translation. It meticulously evaluates the capabilities of leading-edge machine translation technologies, including neural networks and large language models (LLMs), against the stringent standards set by the American Translators Association (ATA) certification exam. This exam rigorously tests linguistic accuracy, stylistic integrity, and adherence to conventional language norms essential for professional translation tasks.

What are you working on this summer?
 

• Experimental Design: Developing a complex experimental framework that repurposes texts from past ATA exams, transforming them into a testbed for assessing AI translation capabilities across two high-resource language pairs (English-into-Chinese  and English-into-Arabic).

• Translation Execution: Deploying advanced machine translation platforms, such as Google's neural engine (Google Translate) and OpenAI's LLM (GPT-4), to conduct a series of translation tasks.

• Critical Evaluation: Integrating state-of-the-art automated metrics like COMET and BLEURT with in-depth manual assessments by ATA-certified graders. This dual evaluation strategy is designed to dissect the translations, assessing their fidelity to the demanding requirements of professional translation.

What can your research tell us about AI and Translation?
This research is poised to provide a granular analysis of how AI technologies perform according to the standards of the ATA, which serves as a benchmark for professional translation. By placing these technologies under the microscope in such a rigorous setting, the study aims to uncover patterns that could inform enhancements in machine translation tools. The objective is to potentially elevate the productivity of professional translators by providing them with more effective AI tools. This exploration is expected to yield observations that could potentially provide insights to the teaching and practice of translation in regard to the dynamics between human expertise and machine-assisted translation.

Congratulations once again to Ali!