OpenAI has a new Applied Evals team that will optimize complex business processes using AI, such as processing refund requests or code migrations.
It will also focus on voice AI and tasks that require multi-step thinking, according to Shyamal Anadkat, who leads the team at OpenAI. Salaries range from $255,000 to $325,000 per year, plus company stock, according to the announcement.
OpenAI has a new Applied Evals team that will optimize complex business processes using AI, such as processing refund requests or code migrations.
It will also focus on voice AI and tasks that require multi-step thinking, according to Shyamal Anadkat, who leads the team at OpenAI. Salaries range from $255,000 to $325,000 per year, plus company stock, according to the announcement.
The term "Evals" refers to evaluating the capabilities of AI models, which Anadkat called "the most important part of creating AI products," Business Insider writes .
The creation of this team reflects a shift in the AI talent market. While tech companies previously sought developers to build and train models, they are now looking for experts with deep knowledge in specific industries to help those models deliver real value.
Anadkat said his team will include engineers with “real and deep experience.” He explained that he will initially look for generalists, but will eventually bring in experts from different fields, from software developers to work with code to perhaps humanities specialists if the need arises.
“OpenAI started as a research lab, and then we built a product,” Anadkat said. “Now, I hope we’ll see a shift in how research and product can work closely together to jointly define what constitutes a quality outcome.”
The Anadkat team is starting small and will scale as demand demands. It is focused solely on business customers and operates separately from the teams working on OpenAI’s consumer products, such as apps or consulting services.
According to Anadkat, Applied Evals will work with OpenAI’s sales and business teams to prioritize projects based on customer needs and areas where their models are underperforming.
In the AI industry, engineers have always evaluated models, but their work has mostly been limited to checking whether the model works at all.
According to Michael Jacobides, a professor at the London Business School, the evaluation of AI models has evolved from simple approval or disapproval to deep analysis of context and asking the right questions.
“There are probably only a hundred people who can lead the development of such advanced models,” said Justin Farris, vice president of products at Read AI, “but there is still a tremendous amount of work to be done to make them useful.”
“AI is increasingly moving from general to specific use cases,” said Tanmay Gopal, CEO of PromptQL.
“For many applied tasks, defining what constitutes a good or bad result becomes quite subtle and multifaceted,” he added.
