FORMER Junior Minister Ivor Callely could be forced to scrap guest accommodation he built at his holiday home if he loses an appeal to An Bord Pleanála.