Experienced ML Quality Process Lead - Italian, Customer Engagement Technologies

At arenaflex, we're revolutionizing the way customers interact with our services by developing innovative customer and CS Associate (CSA) facing products globally. Our Customer Engagement Technology (CET) organization is at the forefront of this transformation, empowering customers to effortlessly self-serve and automating solutions to make their lives easier. When customers need human interaction, we enable CSAs to elegantly solve issues using our associate-facing products, all powered through human-centered design. We're now seeking a highly skilled ML Quality Process Lead, fluent in Italian and English, to join our Omni Machine Learning Data Associate (MLDA) team within CET. As a key member of this team, you'll play a vital role in managing quality management processes to analyze, improve annotation, testing, and contact reading accuracy, supporting new feature and product launches for Customer Service Large Language Models (LLMs).

A Day in the Life

As an ML Quality Process Lead, you'll be responsible for reviewing annotations, performing root cause analysis, providing support for ML model training data annotations, and improving Standard Operating Procedures (SOPs). You'll also review LLM testing results, monitor annotation project accuracy, and proactively communicate any blockers or potential delays. Your attention to detail, analytical skills, and ability to collaborate with cross-functional teams will be essential in ensuring the highest quality data and model performance.

Key Responsibilities

* Review annotations by paying close attention to details, making necessary adjustments to ensure high-quality data that supports ongoing model improvement.

• Perform root cause analysis using basic data analysis in Excel and SageMaker (annotation tool) on annotations to identify opportunities for improving data accuracy.
• Provide support for ML model training data annotations, assisting annotators in maintaining high-quality work by enforcing best practices.
• Improve Standard Operating Procedures (SOPs) by sharing findings from reviews or deep dives, ensuring a consistent standard of excellence across the team.
• Review LLM testing results provided by testers, paying close attention to details, to ensure their accuracy and identify areas for improvement.
• Carefully monitor the accuracy of multiple annotation projects and proactively communicate any blockers or potential delays in the completion of quality checks.

About the Team

Our Omni MLDA team is a remote-working group that develops and fine-tunes generative AI machine learning models, executing testing plans to elevate the customer service experience. We work closely with Science and Product/Program Management teams to release localized customer-facing and agent-facing CS solutions, allowing them to focus on new product development. Our team also helps existing CS teams keep their solutions up-to-date and identifies technical defects to tech partner teams or defect elimination teams.

Basic Qualifications

* Language fluency in Italian (Native-level) and English.

• Experience in creating and managing ML annotation processes, testing models, and quality assurance methodologies.
• Analytical and problem-solving skills to identify patterns, inconsistencies, and areas for improvement.
• Ability to thoroughly investigate and identify misalignment between annotations and SOPs, as well as the root causes of inaccuracies.
• Strong ownership and accountability to meet SLAs, and proactive communication regarding blockers, and proposed solutions.
• Ability to collaborate closely with cross-functional teams, understand project/stakeholder requirements, and align annotation efforts and model testing accordingly.

Preferred Qualifications

* Bachelor's Degree

• Experience in annotation
• Familiarity in using Excel
• Experience with project management and stakeholder management

Benefits and Compensation

At arenaflex, we offer a comprehensive benefits package that supports your well-being and career growth. Our benefits include:

• Medical, Dental, and Vision Coverage
• Maternity and Parental Leave Options
• Paid Time Off (PTO)
• 401(k) Plan

Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $40,400/year in our lowest geographic market up to $86,500/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. arenaflex is a total compensation company, offering a range of benefits, including equity, sign-on payments, and other forms of compensation, in addition to a full range of medical, financial, and/or other benefits.

Why Join arenaflex?

arenaflex is committed to a diverse and inclusive workplace, where everyone has the opportunity to grow and succeed. We're an equal opportunity employer and do not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. Our inclusive culture empowers arenaflexians to deliver the best results for our customers. If you're passionate about machine learning, quality assurance, and customer service, and you're fluent in Italian and English, we encourage you to apply for this exciting opportunity. Join our team and be part of a dynamic organization that's revolutionizing the way customers interact with our services.

How to Apply

To apply for this position, please visit our career site and submit your application. We look forward to reviewing your qualifications and learning more about your experience and skills. Apply Now! Apply for this job