[ad_1]
Among other things, this is what Gebru, Mitchell and five other scientists warned about in their paper, which called the LLM a “random parrot.” Emily Bender, a professor of linguistics at the University of Washington and one of the co-authors of the paper, said: “Language technology can be very, very useful in the appropriate scope, location and framework.” However, the general nature of the LL.M. and its imitative persuasion Power, prompting companies to use them in areas that are not necessarily equipped.
In a keynote speech at the biggest recent AI conference, Gebru linked this hasty deployment of LLM to the consequences of her own life experience. Gebru was born and raised in Ethiopia, where Escalating war Destroyed the northernmost Tigray area. Ethiopia is also a country that speaks 86 languages, and almost all mainstream language technologies cannot explain these languages.
When the Tigiri War first broke out in November, Gebru saw the platform struggling to deal with a lot of error messages. This is a symbol of a persistent pattern that researchers have observed in content reviews, and Facebook relies heavily on a master of law in this field. community Speak language Environments not prioritized by Silicon Valley suffer from the most unfavorable digital environment.
Gebru pointed out that the damage does not end here. When fake news, hate speech and even death threats are not mitigated, they will be used as training data to build the next generation of LLM. These models make the training they receive meaningless, and ultimately make the harmful language patterns on the Internet reflect.
In many cases, researchers have not conducted sufficiently thorough investigations to understand that this toxicity may manifest itself in downstream applications. But some scholarships do exist.In her 2018 book Compression algorithmSafiya Noble, an associate professor of information and African American studies at the University of California, Los Angeles, documented how the prejudices embedded in Google search perpetuate racism and, in extreme cases, may even contribute to racial violence.
She said: “The consequences are very serious and serious.” Google is not only the main knowledge portal for ordinary citizens. It also provides information infrastructure for institutions, universities, and state and federal governments.
Google has used the Master of Laws to optimize some of its search results.With the latest announcement of LaMDA and Recent suggestion The company has issued a statement on the preprint, clearly stating that it will only increase its reliance on this technology. The nobleman feared that this might make the problems she found more serious: “Google’s ethical AI team was fired for raising very important questions about the racism and sexist patterns embedded in the big language model. This is really a wake-up call.”
Big science
The beginning of the BigScience project directly responded to the growing demand for scientific review of the Master of Laws. While observing the rapid spread of this technology and Google’s censorship of Gebru and Mitchell, Wolf and several colleagues realized that it is time for the research community to solve the problem on its own.
Inspired by open scientific collaborations like CERN in the field of particle physics, they came up with the idea of an open source LLM that can be used for critical research independent of any company. In April of this year, the organization received a grant to use the French government’s supercomputer for construction.
In technology companies, the LL.M. usually consists of only six people with major technical expertise. BigScience hopes to attract hundreds of researchers from different countries and disciplines to participate in the real collaborative model building process. The Frenchman Wolf first came into contact with the French NLP community. Since then, the program has rapidly developed into a global business, covering more than 500 people.
Now, the collaborative organization is roughly divided into a dozen working groups and counted. Each working group deals with different aspects of model development and investigation. One group will assess the impact of the model on the environment, including the carbon footprint of training and running the LLM and the life cycle cost of the supercomputer. The other will focus on developing methods to obtain training data responsibly-looking for alternatives to simply grab data from the web, such as transcribing historical radio archives or podcasts. The goal here is to avoid the use of harmful language and unauthorized collection of private information.
[ad_2]
Source link
