Mr Mandela, 85, one of the world's leading AIDS campaigners, joined 40,000 fans of all races who filled a stadium in South Africa's tourist mecca Cape Town under a cloudless sky for a fund-raising concert.
"AIDS sufferers are serving a prison term for life," Mr Mandela, who spent 27 years in jail during South Africa's apartheid era, told the crowd.
AIDS has hit South Africa harder than any other country, with more than five million of its 45 million people infected, and is seen as corroding an already fragile social fabric as it leaves an army of orphans in its wake.
Mr Mandela, who was given a thunderous reception, has launched a campaign called 46664 his prison number to mobilise governments to declare HIV/AIDS a global emergency and to get millions of infected people on life-prolonging anti-retrovirals.
"Millions of people infected with HIV/AIDS are in danger of being reduced to a number unless we act now," Mr Mandela, who stepped down as South Africa's first black president in 1999, said in a reference to his prison number.
"AIDS has ceased to be something to be ashamed of. It's just another medical condition," pop singer Bob Geldof, who organised the hugely successful Live Aid concert in London in the 1980s to help Ethiopian famine victims, told the crowd.
The South African Broadcasting Corporation televised the concert live on its Africa channel. A live webcast was put out on the internet on www.46664.com, organisers said.
The concert will be screened globally by MTV on World AIDS Day on December 1. The music channel has offered a 90-minute concert version to broadcasters rights-free and estimates the event could reach more than two billion viewers.
Earlier this month, the South African government approved a national drug treatment program to tackle AIDS, bowing to huge domestic and international pressure to act against the epidemic that kills an estimated 600 South Africans each day.
President Thabo Mbeki's cabinet had long resisted calls for free drug treatment for infected people, but in August it ordered officials to draw up a national treatment plan.
The question of treatment had threatened to dominate the run-up to next year's general election marking 10 years since the end of apartheid.
Mr Mbeki has long backed so-called "AIDS dissident" scientists who question the link between AIDS and HIV. He has since withdrawn from public discussion over the disease. The United Nations says more than three million people have died from AIDS in 2003.