About 500 people have fled their homes in Cape Town as a wildfire defied rain and firefighters for a fourth day.