Nadal could not contain a barrage of relentless, brutal groundstrokes from the Czech, 26, on his Wimbledon debut.
Rosol moved into a two-one lead but Nadal fought back to win the fourth.
Play was suspended while the roof was closed and it was Rosol who settled first to win the final set and the match 6-7 (9-11) 6-4 6-4 2-6 6-4.
The result now opens up the draw for Britain's world number four Andy Murray, who was seeded to meet the Spaniard in the semi-final and beat Ivo Karlovic in four sets earlier in the day.
Rosol, who is making his main draw debut at Wimbledon after losing in the first round of qualifying for the last five years, will play Philipp Kohlschreiber next after the German beat Malek Jaziri of Tunisia.
It was the first time Nadal had lost before the third round of a Grand Slam since he was beaten by Gilles Mueller in the second round at Wimbledon in 2005.
Nadal, who won the Wimbledon title in 2008 and 2010, arrived at the All England Club in good form having sealed his seventh French Open title.
Rosol, meanwhile, has never got past the third round of a Grand Slam and has remained a relative unknown since turning pro in 2004, although he was ranked as high as 65 last year.
"I don't know what to say," Rosol told BBC Sport. "I'm not just surprised but it's like a miracle. I never expected something like this. There are so many emotions - I don't know what to say.
Afterwards, the Spaniard said: "I played a great fourth set. He came back [after the delay to close the roof] and played unbelievable in the fifth.
"The only thing I can do is go back home and rest, and I need and deserve it."
He added: "I'm very, very disappointed [but] it's not a tragedy, it's only a tennis match."
Elsewhere, fifth seed Jo-Wilfried Tsonga overcame a first-set blip to beat Guillermo Garcia-Lopez 6-7 (3-7) 6-4 6-1 6-3 to set up a tie with Lukas Lacko.
Three-time Wimbledon runner-up Andy Roddick progressed to the last 32 with with a comprehensive 6-3 7-6 (7-1) 6-3 victory over Bjorn Phau of Germany. (BBC.co.uk)