Fourth seed Dementieva lost 5-7 6-4 6-3 to American Melanie Oudin, who survived a thigh injury in the third set to close out a famous victory.
Fifth seed Jankovic, a finalist last year, lost a thrilling encounter against impressive Kazakh Yaroslava Shvedova 6-3 6-7 (4-7) 7-6 (8-6).
Top seed Dinara Safina continued to struggle as she defeated Germany’s Kristina Barrois 6-7 (5-7) 6-2 6-3.
“I don’t even know what to say right now. I’m so excited. You have no idea,” said the 17-year-old Oudin after her victory.
Oudin defeated Jelena Jankovic in the third round of Wimbledon and showed great tenacity to add another scalp at the Arthur Ashe Stadium.
In a match that had 13 breaks of serve, Oudin led 2-0 in the third set but Olympic Champion Dementieva edged a lengthy third game.
Oudin took a medical timeout for treatment on an injury to her left thigh and despite returning with her leg heavily strapped broke her opponent to lead 3-1.
The American, clearly in pain and struggling to hold back the tears, then dropped her serve but went on to win the next three games.
Dementieva took the next game to trail 5-3 and saved two match points before failing to clear the net from an Oudin first serve.
“Going into that last point I was thinking I might not get another chance so I just had to go for it with a big serve and I was lucky it went in,” added Oudin.
“The crowd supporting me helped me get through.”
Safina survived a huge scare to beat Australian world number 167 Olivia Rogowska in her first-round match – and she again looked out of sorts against Barrois, particularly with her brittle serve.
The Russian, who has never won a Grand Slam, made 15 double faults during a match that lasted more than two hours.
Barrois looked set to pull off a real upset when she took a 2-0 lead in the third set but Safina showed great determination to win four of the next five games.
The German then had four break points to level at 4-4 but Safina held her nerve to move two games clear and broke Barrois to take the match.
Jankovic looked uncomfortable from the start and was broken early in the opening set by the aggressive Shvedova.
The Serbian did not offer much resistance at 5-3 down as she sent down a forehand that went long to allow Shvedova to break again and take the first set.
It was not looking good in the second set either as Jankovic went 4-2 down when Shvedova broke her serve but the Serbian broke back in the next game.
At 5-4 Jankovic served for the set, racing to 40-0 before Shvedova scrambled back into the game to break serve and stay in the set.
The see-saw nature of the set continued when Jankovic broke back again in the 11th game and at 6-5 up she once again had the opportunity to serve to level the match.
The Serb again gave herself set points at 40-15 before letting the 21-year-old back in and Shvedova again broke serve to force a tiebreak.
Jankovic moved to set point at 6-3 but again showed she was capable of throwing it away when she served a double fault. The next point, though, saw the fifth seed finally get it done and she squared the match at 1-1.
But Shvedova was not to be denied and won through in a tense tie-break, finishing it in style, sending Jankovic crashing out with an ace.
Ninth seed Caroline Wozniacki brushed aside Petra Martic of Croatia 6-1 6-0, ensuring the Dane has reached at least the third round of every major this year.
Thirteenth seed Nadia Petrova of Russia eased into the third round with a 6-4 7-6 (7-3) victory over France’s Julie Coin.
In the men’s singles, Wimbledon semi-finalist Tommy Haas breezed into the third round with a 6-4 6-4 7-6 (7-3) victory over American Robert Kendrick.
The German, seeded 20th, created 18 break point opportunities in the match, but it stretched on longer than it might have as he converted only two.
He will now face 10th seed Fernando Verdasco in the next round after he cruised past Florent Serra 6-3 6-0 6-3.
Verdasco, who reached his first Grand Slam semi-final when losing the Rafael Nadal at the Australian Open this year, dispatched the Frenchman in under 90 minutes.