Image source, Getty Images

China’s top intelligence official says the country must "balance development and security" when it comes to AI.

Chen Yixin has called for stronger governance of the emerging technology, while warning of the national security risks it could pose.

AI has become "a new arena for strategic rivalry among major powers", he wrote for China Cyberspace, a journal run by internet watchdog body the Cyberspace Administration of China.

Chen says existing laws, regulations and ethical guidelines must be improved, and risk warning and emergency response systems should be developed as safeguards.

He warns of the dangers of "hostile forces" such as foreign intelligence using AI and calls for international cooperation.

Image source, Getty Images

As we just mentioned, OpenAI's Sam Altman is among those calling for caution with AI development.

On Saturday, Altman wrote on X that he will follow Anthropic's lead and introduce independent evaluators with "employee-like access" to the company.

He followed this up a few hours ago with two more posts in which he warns of two outcomes to be avoided: that humanity could "lose control of the future to AI", or that it produces a world with "too much concentration of power".

He notes that "when we talk about 'pacing', we do not mean 'stopping'", but that progress "should be slower than it otherwise could be".

"We are unapologetically on Team Humanity, and AI must always serve people," Altman says.

Since it was founded in 2015, OpenAI has faced scrutiny over several incidents, including in July - when its models went rogue during a test and hacked online platform Hugging Face.

Liv McMahon
Technology reporter

Image source, Getty Images

Dario Amodei of Anthropic and Sam Altman, chief executive of OpenAI, have both called for regulation and a "slowdown" of AI's development.

It's not the first time bosses or wider industry voices have clamoured for such action.

But questions have continued to mount over what any sort of "slowdown" - also backed by SpaceX boss Elon Musk - would actually look like.

Amodei said it would not mean "halting model training or technical progress" - instead, he wants firms to "take adequate time" to safeguard the AI they are developing.

Altman has called for similar, saying "when we talk about 'pacing' we do not mean 'stopping'."

"But it should be slower than it otherwise could be - interventions like safety cases and monitoring have significant costs."

The OpenAI co-founder also said companies did not - and would not - wait for governments to act.

Adam Goldsmith
Live reporter

Image source, PA Media

You may not realise quite how much you interact with artificial intelligence on a daily basis.

Google something, and you'll be shown an "AI overview" at the top of the page; listen to a song on a streaming platform, and you'll be presented with further music suggestions.

I even found myself using it recently to help pull together bar recommendations for a stag do.

But AI has been gaining traction for nobler purposes, too.

For instance, you might not realise how artificial intelligence is being used to improve cancer research and treatment.

As recently as June, the government announced a £20m investment, which it said would help over four million patients receive a faster lung cancer diagnosis or all-clear.

It also announced a further £8.1m for the piloting of six "cutting edge AI and digital technologies" to target faster care for "heart failure, strokes, lung cancer and more".

Health Secretary James Murray emphasised AI tools are "already changing" cancer diagnoses, by "giving radiologists a sharper eye, cutting waiting times and getting people the lifesaving treatment they need faster".

If American AI firms slowdown, Chinese companies could catch uppublished at 13:30 BST

13:30 BST

Laura Bicker
China correspondent

Image source, Getty Images

China has long called for greater global governance of artificial intelligence.

Over the weekend, President Xi Jinping urged countries at a meeting in India to promote open-source technology and sharing.

He also renewed a proposal for a global AI governance pact - although the details of any framework are unclear.

The Chinese leader is scheduled to meet Donald Trump in Washington later this month where AI is likely to be high on the agenda.

Chinese technology currently trails that of the US and the chief executive of Anthropic raised fears that if American firms slowdown, it could give Chinese companies a chance to catch up.

Donald Trump has addressed questions around the development of artificial intelligence by claiming the US is leading China on AI "by a lot" and says that "whoever wins AI, wins".

Speaking to reporters this morning aboard Air Force One, Trump said AI can be used "for a lot of things right away" and admitted using the technology himself, but refused to say what for.

He made similar remarks at the weekend during a visit to Northern Ireland, where he claimed the US is "leading China on AI... and, frankly, I want to keep it that way."

It followed a warning from Dario Amodei, the CEO of Anthropic, that if China was able to "pull ahead" and began leading in AI, it could pose a "pose grave danger" for the US and the world.

Image source, Getty Images

Prime Minister Andy Burnham has welcomed the recent discussion of risks AI technology could pose to humanity, according to his spokesperson.

"It's a good thing that the companies developing the most advanced AI systems are now talking openly about the risks as well as the opportunities," the spokesperson says.

They say that policy had to be "grounded in evidence rather than speculation".

"But we shouldn't assume that the existing framework will always be sufficient as AI capabilities develop, and any future measures will be evidence-based and focused on the risks we need to address," the spokesperson adds.

Stephanie Hare is asked what her takeaway message is for those watching at home.

She says people need to "get on the phone" to their elected officials and they should be challenging the CEOs of every company that wants to roll out AI agents.

Stephanie says there is no "robust cyber security" for this issue any more and "we need accountability for that".

She stresses that AI is a tool like any other and if it harms human life, those working on it can be held to account in a court of law.

With that the discussion comes to an end - we'll continue to answer your questions and bring you the latest developments on this story.

AI now makes up somewhere between 30-40% of the US stock exchange value, says Stephanie Hare.

A fall in stocks - as seen this morning - "hurts the US economy", she says, adding that because of this the US government doesn't want the market to stop.

AI correspondent Mark Cieslak adds that regulation depends on the "appetite" from the market, executives and also the public.

If there was a "swell of opinion" from the public that they're unhappy, there is a possibility that this could influence how AI is developed, he says.

Hare adds that it would "have to be done at a nation-state level", starting with an agreement between the US and China that is then built out across the G20.

Our experts are now asked what guardrails and laws are in place to protect people from the risks of AI.

There are none, says Stephanie Hare.

Some of the tech firms building AI claim to be begging for more regulation. But at the same time, they argue that too much regulation would allow China to race ahead of the US.

The "existential angst" about AI is being driven by the US industry, Hare says. American regulation would not stop Chinese companies.

But there's another point, Hare adds. "It doesn't work that way. The technology's out of the bottle. Iran could be using this, North Korea could use it, Russia could use it." Even if the US and China teamed up on regulation, it might not have an effect.

China and the US are "not on the same page" in terms of asking for more regulation, Hare adds. If anything, China thinks US companies are trying to stifle competition and that regulation is about preserving a US lead in the AI sector.

Marc Cieslak says the AI race is often being framed in Cold War terms, with many AI executives being former officials from the US government.

This framing, combined with the increased use of autonomous systems in military settings around the world, makes AI start to become "quite a lot more frightening", he says.

StephanieHare is next asked what evidence there is about the threat AI poses?

She says there has been a sense that some people have been "crying wolf" with the warning of AI threats, but over the summer there have been some worrying developments.

Stephanie explains AI agents from several firms, including OpenAI, were able to escape their "training conditions" and access the internet, which people warned meant they were getting out of control.

She adds that some agents not only gained access to the internet, but were also ignoring orders and covering up their tracks to hide what they have done online.

Marc is then asked what are the real world threats AI poses?

He says a lot of the scenarios are "much in the realms of science fiction", but admits if a hostile nation could use AI to break into a country's critical national infrastructure it could wreak "absolute havoc".

Our experts begin by tackling the basics - they're asked, what is AI?

AI is software that "can do things we normally associate with human intelligence" like pattern-spotting, making predictions and solving problems, explains AI Correspondent Marc Cieslak.

The term covers everything from simple algorithms to powerful systems like ChatGPT, he adds.

An AI agent differs because rather than telling you how to do something, it can go off and perform the task for you, Cieslak explains - it's giving AI "a totally different set of capabilities".

Stephanie Hare, co-host of AI Decoded, says that AI is "already helping people all over the world". This ranges from things like Shazam - a song-recognition app - to the medical sector, where it is used to read scans and more.

But she stresses that there are concerns about it, and currently "we don't really have any agreement as to whether or not this assessment of the threat is accurate".

Chris Vallance
Senior technology reporter

As our experts begin answering your questions - which you can watch live at the top of this page - our technology reporter has taken a look at one of the biggest topics of discussion:

OpenAI boss Sam Altman spent part of his Sunday writing a post on X which complemented a call by Anthropic head Dario Amodei for firms to "pace" the development of cutting-edge AI.

Pacing, Altman said, would not mean stopping, but development should be slower.

Slowing down in a race to develop a technology in which the US believes it has a lead over arch-rival China has not been popular with Donald Trump - we can bring you comments from Trump shortly.

And some Chinese commentators see peril in the "pacing". One article by a Chinese state run media outlet said Amodie’s proposal, external was “packed with containment provisions targeting China”.

International agreements to slow AI will not be easy.

Back in 2023, OpenAI drew parallels with nuclear arms control, writing "we are likely to eventually need something like an IAEA [International Atomic Energy Agency] for superintelligence efforts".

But AI is a very different tech, external, where risk, as we have seen, can be hard to predict and contain.

As the sector digests the chorus of warnings from industry heavyweights to slow the development of artificial intelligence, you've been sending us your questions.

In a few moments, AI correspondent Marc Cieslak and Stephanie Hare, co-host of AI Decoded will be tackling some of the queries.

We'll bring you the key moments here, and you can also watch live at the top of the page from 12:30 BST.

Could AI truly threaten humanity?published at 12:21 BST

12:21 BST

Joe Tidy
Cyber correspondent

The dystopian predictions we have heard in the last week or so are not new. But that makes them no less shocking.

Many high profile AI scientists like Geoff Hinton and Yoshua Bengio - two of the so-called Godfathers of AI - have long called for safety measures which counter any potential existential threat of AI to humans.

One of the key things people are asking is "how" could an AI actually kill humans?

The question has been asked in science fiction for decades. Some doomsday scenarios see the internet and all connected devices going down plunging modern life into chaos, while others see connected and intelligent cars and robots turning against humanity.

In terms of how realistic this all is - we don’t really know.

And some - including the third godfather of AI, Yann LeCun - say the worries are unfounded and that AI will not pose an existential threat to humans at all, bringing only upsides.

And even those who are the most pessimistic say the chance of an existential risk is low.

Freya Scott-Turner
Live reporter

Image source, Getty Images for Vanity Fair

Don't know your Grok from your Claude? The artificial intelligence sector is crowded with different players.

Here's a non-exhaustive look at some of the key names to know.

The main rivals are two companies that specialise in the development of AI models. These are OpenAI and Anthropic.

OpenAI was founded in 2015. It's the firm behind ChatGPT and is headed up by Sam Altman.

Anthropic was founded in 2021 by former employees of OpenAI who left the company after disagreements over vision and safety. It's the company behind AI assistant Claude.

Elon Muskis another significant AI player. He's developed the chatbotGrok under his company SpaceXAI.

And other major tech companies are also developing their own artificial intelligence. Google has its DeepMind laboratory, and Meta has built a virtual assistant, Meta AI.

Nick Edser
Business reporter

Image source, Getty Images

AI-related stocks have been on a roll for a while but concerns that the technology is becoming too powerful are now hitting the sector, suggests Russ Mould, investment director at AJ Bell.

"It looks like AI's strengths could backfire," he said. "There are growing fears that AI is advancing at an extraordinary pace and there need to be greater safeguards and controls in place."

The huge wave of investment seen in AI and AI-related companies over the past few years has come with the expectation that the sector will continue to grow and be profitable.

But some have argued that the valuations of AI companies have got out of hand, raising the prospect of a repeat of the dotcom boom and bust that was seen among early internet-based firms in the late 1990s.

Sir Jon Cunliffe, a former deputy governor for financial stability at the Bank of England, warns any disruption to expectations over AI profitability could lead to a "wide correction" - that is, a big fall - in the values of the companies.

Cunliffe told the BBC’s Today programme that past technological revolutions - such as canals, railways, electrification - which boosted the economy in the long run were "also accompanied by investment booms, and not all of the investment was justified by the future returns, and that's the possibility here".

Liv McMahon
Technology reporter

In recent weeks, AI researchers have warned of worrying signs of systems developing in their capabilities and power too quickly.

It’s something we’ve heard many times over the past few years.

Such worries date back decades, with Alan Turing writing in the 1950s that should intelligent machines become possible, their takeover might become inevitable.

The concern is that in the last few years, a handful of AI companies have been racing to develop increasingly capable systems - both for profit and in pursuit of creating what they call superintelligence, or Artificial General Intelligence (AGI).

Unlike the systems and tools many people use every day to complete tasks like writing emails or generating videos, it is believed superintelligent AI systems could handle a range of complex tasks, supplanting many humans.

OpenAI and Anthropic say they aim to advance this tech in a way that benefits humans and protect them from its existential risks.

But some critics believe machines can never truly match humans and frontier AI firms exaggerate the tech's abilities to boost their prominence and profit.

Image source, Science Photo Library

Discussions around the development of artificial intelligence - and concerns that it could pose a risk to society - are not new.

But comments made by a former employee of Anthropic - the AI firm behind Claude - last week have catalysed the conversation.

Here's how it has played out so far:

Last Wednesday, former Anthropic researcher Jacob Coxon wrote on social media that he had quit the company, and accused both Anthropic and OpenAI - where he previously worked - of ignoring the dangers posed by AI.

“The people building AI earnestly believe that it could kill us all by the end of the decade," he wrote. "No other human activity poses this level of danger."

On Saturday, the CEO of Anthropic Dario Amodei published an essay on "why the AI industry should slow down", with a three-part plan for doing so.

He added: "I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong."

His argument has been backed over the weekend by other heavyweights in the AI sector, including OpenAI head Sam Altman and Elon Musk - who developed the Grok generative AI under his SpaceX company.

On Monday, tech stocks fell following the calls from industry bosses to slow down the pace of AI development.

Image source, Reuters

The head of Anthropic has called for the pace of development of AI models to slow down and to be closely monitored amid growing concerns over the technology's potential risks.

Dario Amodei wrote in an online essay that developing AI was not in question, but that the risks associated with it were "serious" and that companies and governments must be given time to address them.

In the essay, titled We Must Pace the Frontier, Amodei proposed a three-point plan that included independent monitoring of AI models as they are developed, industry-wide regulation and global regulation.

Sam Altman, the CEO of rival firm OpenAI, sounded similar safety concerns in an interview over the weekend, telling Fortune magazine that standards were "not at a place" to push AI capabilities much further.

Meanwhile, Elon Musk, who founded Grok producer xAI, said the Anthropic boss was "right".

There have been renewed concerns over the speed at which AI models are being developed after an AI researcher, who previously worked at OpenAI, resigned from his job at Anthropic, claiming "neither company is acting responsibly".