.
Professor André Ponce de Leon Ferreira de Carvalho, dean of the Institute of Mathematics and Computational Sciences (ICMC) at USP in São Carlos, is one of the members of the group of 100 artificial intelligence experts from 30 countries who participated in preparing the International AI Safety Report. Released by the British government on January 29, the document unprecedentedly summarizes the scientific evidence on the main risks and capabilities of general-purpose AI – the technology behind tools such as ChatGPT, LLaMA, and PaLM.
The report was chaired by computer scientist Yoshua Bengio, winner of the Turing Award, and experts from the European Union, the United Nations (UN), and the Organization for Economic Cooperation and Development (OECD) participated in the writing of the report, divided into three committees: Expert Advisory Panel, Editors, and Senior Advisors. Carvalho was the Brazilian representative on the expert advisory panel, and Professor Teresa Ludermir, from the Federal University of Pernambuco (UFPE), participated in the group of Senior Advisors. They are the only Brazilians to participate in the document.
The invitation to join the Expert Advisory Panel came at the end of 2023 when the British government held a meeting with representatives from 29 countries, including Brazil. At the end of the meeting, the Bletchley Declaration was released, and each country was asked to appoint an expert to form the working group focused on AI safety. Due to his expertise in the area, Carvalho was invited by the Ministry of Science, Technology, and Innovation (MCTI). “The meeting was motivated by the growing concern about general-purpose AIs, which have increasingly sophisticated capabilities and pose significant challenges for society,” the dean points out.
AI safety is an issue that needs to be addressed globally, and the report’s strength was its diversity of perspectives. The professor compares this challenge to the spread of biological viruses: “If a country doesn’t adopt effective preventive measures, the threat can spread rapidly around the world. It’s the same with AI: without a coordinated global policy, there will always be a risk of a weak link in the chain, undermining everyone’s safety.”
How the report was organized
According to the professor, the team responsible for preparing the International AI Safety Report began work in 2024 and held regular meetings to review and improve the document. “The drafts were reviewed regularly, and each expert had to suggest adjustments, additions, and changes. In a second moment, two new committees were created: one of senior experts and another of editors. The committee of editors was in charge of the final wording, while the experts ensured technical accuracy. Afterward, a more extensive review ensured the coherence and clarity of the final report,” he explains.
The document was developed independently, without influence from industry or governments, and is exclusively advisory – in other words, it does not make policy recommendations to countries but only provides what is known about the current state of AI, its risks, and approaches to reducing them. “The report presents an overview of the state of knowledge and scientific discussions on AI – including uncertainties and divergences that still persist,” the Brazilian emphasizes.
Regarding security, the document highlights three main concerns:
- Empowerment of malicious individuals: AI can be used to boost cyberattacks, spread disinformation, and manipulate content, creating a more vulnerable digital environment.
- Facilitating the creation of chemical and biological weapons: the development of these weapons previously required a high level of technical expertise. Now, AI can make this information more accessible, amplifying global risks.
- Autonomous weapons and independent decisions: As AI advances, autonomous systems can make decisions without human intervention, which poses a significant risk to international security.
The report also addresses other important matters, such as:
- Impact on the labor market: AI could automate many functions, driving many workers into unemployment. However, the report argues that this impact will depend on the speed of technological advances, companies’ adherence, and how the demand for human labor adjusts. Experts differ on the matter: will the jobs created by AI be enough to cope with the losses? The report also warns of increasing socio-economic inequality, mainly benefiting highly qualified professionals.
- Environmental challenges: the high energy consumption of AI systems is a major concern. The document suggests prioritizing the production of clean energy and the development of more efficient algorithms. On the other hand, AI is seen as an ally in environmental preservation, helping to fight illegal deforestation and predicting extreme climate events.
- Disinformation and political influence: AI can be used to spread fake news and disinformation campaigns, undermining public trust and weakening democracies. The report points out that algorithms can amplify social and political biases, resulting in discriminatory decisions. To mitigate these risks, experts advocate the need for monitoring mechanisms and transparency of AI tools.
- Inequality in access to AI: emerging countries may struggle to keep up with the development of advanced AI due to barriers such as limited infrastructure, lack of specialized talent, and insufficient investment.
- Privacy and data protection: AI can be used for mass surveillance, data breaches, and the creation of deepfakes, increasing the challenges related to privacy.
- Market concentration: AI development is concentrated in a few countries, such as the United States and China, creating a global dependence on these nations. This can leave critical sectors such as health and finance vulnerable to external priorities and decisions.
Challenges and opportunities for Brazil
According to the ICMC dean, the country needs to invest heavily in education, research, and training to ensure the safe and ethical use of AI. “It is essential that people understand how it works, its risks, and its implications. Only with massive investment in education and research will it be possible to ensure that AI benefits society as a whole,” he says.
For him, training in AI should not be restricted to the technological area alone but should include all areas of knowledge. This way, professionals from different sectors will know how to strategically apply technology. The professor warns: “If Brazil doesn’t invest widely in this qualification, it runs the risk of creating a new professional chasm, in which only a portion of the population will have access to the opportunities generated by AI.”.
For the researcher, the impact on the labor market can already be felt more intensely in countries with lower-quality education systems, and Brazil needs to plan to avoid a scenario of unemployment and helplessness for its population. “The world is interconnected, and companies compete on a global stage. One of the determining factors in this international competition is worker productivity. If a country doesn’t qualify its workforce to keep up with this technological evolution, it will lose competitiveness, resulting in a loss of jobs and socio-economic opportunities,” he points out.
In addition to the impact on the labor market, there are also environmental issues to consider. Brazil has a relatively more efficient energy sector than many European countries, and this factor could be a strategic point for Brazil. The report proposes a balance when it comes to regulating AI, as excessively rigid rules can stifle innovation, while the absence of clear guidelines can generate unpredictable risks. In this sense, Carvalho explains that any legislation can become obsolete quickly since technology is advancing so quickly. “The ideal plan is to adopt a flexible model of regulation, capable of keeping up with technological advances,” he thinks.
For the ICMC dean, the release of the Brazilian Artificial Intelligence Plan (PBIA) in 2024 demonstrated the country’s intention not to be left behind in the global race. However, the plan supports infrastructure much more than research and the training of qualified personnel, the main reasons for advances in AI. Finally, the professor says that despite the challenges, general purpose AI can bring significant benefits to education, health, environment, and scientific research, as well as boost innovation and the country’s economic development.
Check out the International AI Safety Report neste link.
.
Text: Gabriele Maciel, ICMC Communications Office
