Shaun Harrad scored the goal which settled the first-round replay at the Pirelli Stadium and secured a tie with Burscough for Nigel Clough's men.
The game was scoreless until Harrad's intervention in the 73rd minute, when he picked the ball up on the edge of the area before advancing and beating Posh keeper Mark Tyler from six yards.
League Two sides Leyton Orient and Boston overcame League One opposition - but there was to be no fairytale success for non-league clubs Chasetown and Eastbourne.
Orient beat Chesterfield 2-1 at Saltergate. The League Two leaders went ahead when John Mackie sidefooted into an empty net from 20 yards after Barry Roche had punched out a free-kick.
Kevan Hurst replied for the hosts but just before half-time Shane Tudor hit the winner for the visitors to secure a second-round clash at Rushden & Diamonds.
Julian Joachim scored twice as Boston knocked Swindon out of the cup at York Street with a 4-1 victory.
The former Aston Villa striker set the Pilgrims on their way with a shot from 12 yards after 10 minutes.
Joachim claimed his second midway through the half before Bradley Maylett and Jason Lee put the tie out of the Robins' reach, although they pulled a goal back through a late penalty by Rory Fallon.
Oldham despatched Chasetown 4-0 to earn a home tie against Brentford. Goals from Paul Warne, Chris Porter (two) and Chris Hall broke the Midlands Alliance club's proud resistance at Boundary Park.
Steve Basham claimed a hat-trick as Oxford beat Eastbourne to earn a trip to Cheltenham.