Prison officers have called for a former jail which held some of the North's most notorious paramilitaries to be bulldozed after proposals to build a peace centre there stalled.