Germany defender Josha Vagnoman's 81st-minute goal took Stuttgart to a 2-1 win at Cologne on Sunday.
Cologne, promoted in the summer after a season in the second division, went ahead after just four minutes when Jakub Kaminski slid the ball into the bottom corner.
Ermedin Demirovic, awarded a penalty despite not going down in the box, levelled things up from the spot after 28 minutes – scoring for the third straight game in succession.
With nine minutes remaining, Vagnoman latched onto an inch-perfect pass from Germany teammate Angelo Stiller and turned the ball into the far corner.
German Cup holders Stuttgart lost three of their first five matches in all competitions to open the season, but have now won three in a row.
Earlier, Freiburg were held to a 1-1 draw at home against Hoffenheim, missing out on a third consecutive Bundesliga win.
Freiburg stumbled out of the gates this season, after finishing fifth last term, with consecutive defeats, but bounced back with victories over Stuttgart and Werder Bremen.
Lukas Kuebler put the hosts in front early, tapping in from close range after a Vincenzo Grifo corner in the third minute.
Hoffenheim's Fisnik Asllani scored 10 minutes later, turning in from Vladimir Coufal's low cross after the Czech full-back snatched the ball from the Freiburg defence.
Asllani now has four goals in five Bundesliga games to start the season.
Freiburg, who opened their Europa League campaign with a 2-1 home win over neighbours Basle, play at Serie A side Bologna on Thursday.
Hamburg visit Union Berlin in Sunday's late game.