Professor Stephen Hawking has warned that artificial intelligence could develop a will of its own that is in conflict with that of humanity.
It could herald dangers like powerful autonomous weapons and ways for the few to oppress the many, he said, as he called for more research in the area.
But if sufficient research is done to avoid the risks, it could help in humanity's aims to "finally eradicate disease and poverty", he added.
He was speaking in Cambridge on Wednesday at the launch of The Leverhulme Centre for the Future of Intelligence, which will explore the implications of the rapid development of artificial intelligence.
All great achievements of civilisation, from learning to master fire to learning to grow food to understanding the cosmos, were down to human intelligence, he said.
"I believe there is no deep difference between what can be achieved by a biological brain and what can be achieved by a computer.
"It therefore follows that computers can, in theory, emulate human intelligence - and exceed it."
Artificial intelligence was progressing rapidly and there were "enormous" levels of investment.
He said the potential benefits were great and the technological revolution could help undo some of the damage done to the natural world by industrialisation.
"In short, success in creating AI could be the biggest event in the history of our civilisation," said Prof Hawking. "But it could also be the last unless we learn how to avoid the risks.
"Alongside the benefits, AI will also bring dangers, like powerful autonomous weapons, or new ways for the few to oppress the many.
"It will bring great disruption to our economy.
"And in the future, AI could develop a will of its own - a will that is in conflict with ours.
"In short, the rise of powerful AI will be either the best, or the worst thing, ever to happen to humanity.
"We do not know which."
He continued: "That is why, in 2014, I and a few others called for more research to be done in this area.
"I am very glad that someone was listening to me!"
He welcomed the launch of the new centre, which is a collaboration between the University of Cambridge, the University of Oxford, Imperial College London, and the University of California, Berkeley.
"The research done by this centre is crucial to the future of our civilisation and of our species," he said.
CFI is funded by an unprecedented £10 million grant from the Leverhulme Trust.
Its mission is to create an interdisciplinary community of researchers that will work closely with industry and policy-makers.
It is the first centre of its kind that will examine both risks and benefits, short and long-term.
And Prof Hawking joked: "We spend a great deal of time studying history, which, let's face it, is mostly the history of stupidity.
"So it is a welcome change that people are studying instead the future of intelligence."