Children from disadvantaged backgrounds are spending less time reading and engaging in physical activity as they grow older, a new study reveals.