First, Giroud squandered a golden chance by heading wide from inside the six-yard box, Sweden captain Andreas Granqvist doing just enough to put him off after great work from Benjamin Mendy down the left to create the chance.
However, the 30-year-old played his way into danger and rushed his kick, passing the ball straight to opposing attacker Ola Toivonen on the halfway line.
The Toulouse midfielder collected a low clearance from Lloris close to the halfway line and clinically lifted the ball into the unguarded net to earn a 2-1 victory that leaves Sweden top of Group A in qualifying for next year's World Cup in Russian Federation.
Sweden's forward Ola Toivonen celebrates after scoring a goal during the FIFA World Cup 2018 qualification football match between Sweden and France in Solna, Sweden. Earlier this year, it had beaten San Marino in a friendly for its first win after 86 consecutive matches. Then Durmaz equalised for Sweden in the second half.
It capped an absorbing match in Stockholm as Sweden secured three points to move above France to top Group A. The latest result for the Scandinavian side was a huge 4-0 win over Belarus, but before that did have a tough 2-1 loss to France away from home. If we didn't win this game, we should have got a draw.
Veteran midfielder Wesley Sneijder - making a record 131st global appearance on his 33rd birthday - slotted in a second after 35 minutes.
Hungary are now a distant eight points behind Portugal after they crashed to Andorra, a first-half header from Marc Rebes giving the minnows a rare triumph.
In Group H Dries Mertens and Nacer Chadli were on target for Belgium as they won 2-0 in Estonia. Just before half-time, Estonia midfielder Artjom Dmitrijev saw red for a foul on Marouane Fellaini.
90+3' - GOAL! Sweden 2-1 France (Toivonen).