A FARMER who secretly built a castle hidden behind walls of straw has vowed: “This house will never be knocked down.”