Periods of repeated isolation have “chipped away” at pupils’ progress pupil , says the head of Ofsted.