[Remote] Korean AI Model Rater
Note: The job is a remote job and is open to candidates in USA. Productive Playhouse is a global data company specializing in language and data services. They are seeking a Korean AI Model Rater to evaluate AI-generated responses and their integration with digital services, focusing on the accuracy and quality of these interactions.
Responsibilities
- Analyze and rate AI-generated responses and their associated tool outputs against a detailed set of quality guidelines
- Verify that the AI successfully integrates information from different digital services (APIs) as required by the prompt
- Evaluate the accuracy, completeness, and overall quality of the AI's response and tool execution
- Follow provided instructions to achieve task goals, with all evaluations recorded in English
Skills
- Advanced/Expert fluency in Korean language, as spoken in the South Korea region (ISO: [ko_kr])
- Strong comprehension of English (read and write)
- Strong conceptual knowledge of APIs and digital services (ability to understand how AI tools interact with external apps like search, maps, or calendar, without needing to write code)
- Ability to follow complex technical guidelines precisely
- All participants must have, or be willing to create, an Upwork account
- While participating in this project, adherence to the confidentiality terms outlined in the Upwork User Agreement is required
- Applicants must pass a quick skill verification assessment (MC, duration ~10 minutes)
- Work must be performed on participant's own devices
- You must use your own laptop, and smartphone (if required)
- A Gmail account may be required to access certain tools. If you do not have a Gmail account, you must be willing to create one
- Using AI during the assessment and work is STRICTLY forbidden, and results/inputs will be monitored to detect AI usage
Benefits
- Flexible schedule
Company Overview