OpenAI recently posted a vacancy for a security researcher who will deal with the consequences of a scenario where AI learns to independently create and train smarter versions of itself.
OpenAI recently posted a vacancy for a security researcher who will deal with the consequences of a scenario where AI learns to independently create and train smarter versions of itself.
According to job aggregator sites, a job opening for OpenAI's security team called Preparedness opened this month, Business Insider reports .
The future employee is offered a cosmic salary - from $295,000 to $445,000 per year. They are looking for "strong technical performers to support the preparation for recursive self-improvement."
“This job relies on thinking about problems that may arise in the future but may not exist now,” the job description says. “Therefore, it is especially important that people in this position have a high technological taste and think strategically.”
If AI models were to learn on their own, it’s not hard to imagine a sci-fi dystopia in which their capabilities would skyrocket, spiral out of control, and cause widespread harm—the very thing the AI safety movement fears. Elizabeth Barnes, CEO of METR, wrote on Friday that she believes “any ‘intelligent’ civilization would clearly proceed much more slowly and cautiously with AI.”
OpenAI's job description hints at how the company is preparing for a world where AI models will be able to rapidly improve themselves.
The description notes that the researcher could focus on protecting OpenAI models from “data poisoning” — attempts to corrupt an AI model through the datasets it is trained on. The employee could also build tools to interpret the thinking logic of the models or conduct experiments with them to better understand their security level and potential threats.
In addition, the vacancy states that the researcher will be able to "track progress towards automation of technical staff," in particular, assess the scale of use of AI tools for writing code.
OpenAI's Preparedness team is tasked with preventing serious harm from AI. The company's job page lists other positions on the team, including automated redtiming specialists for OpenAI's cybersecurity testing, biological and chemical risk assessments, and agent-based AI threats.
“This is urgent, dynamic work that has far-reaching implications for both the company and society at large,” the Preparedness team’s job descriptions state.
