It pushed Texas 7 1/2 games off the Angels' American League West pace with 15 games left on the leaders' schedule.
"It's exciting, getting my first win, especially against a team we're fighting for a playoff spot," Kazmir said. "My fastball command for the most part got me through it."
Vladimir Guerrero's solo homer and Torii Hunter's RBI single were all the offense Kazmir and the relief triumvirate of Jason Bulger, Kevin Jepsen and Brian Fuentes needed to send the Rangers reeling to their fifth consecutive loss.
"[Kazmir] has a fastball that jumps," Angels catcher Mike Napoli said, "and he can get hitters to miss it. That's why he can be effective when he doesn't have his off-speed pitches."
Kazmir's first win in four starts shaved his ERA to 1.42 in 25 1/3 innings since departing Tampa Bay for the West Coast on Aug. 28.
Kazmir said he didn't have a handle on his slider or his changeup, forcing him to throw more pitches than he wanted early on and preventing him from getting deeper in the game.
"I wasn't able to put away guys, and they were getting a lot of foul balls and some base-hits on two-strike pitches," Kazmir said. "I felt I was getting ahead -- I just wasn't able to finish them off.
"I didn't have a feel for my slider or changeup; I was leaving the ball up. I relied on fastball command for the most part to get a couple rollovers. You want your best stuff, a couple of things working, so you don't have to battle every single time out there."
The Rangers had two hits in the second inning and two more in the fifth, but both times Kazmir reached back and found the action and location he needed from his heater to escape.
"I remember the first time I caught him, in Seattle," Napoli said. "He got in trouble the first inning, loading the bases, and I went out with [Mike Sweeney] hitting. He asked me what I wanted, and I told him, `How about a hard fastball in.' He said, `Let's go.'
"He's one of those guys who says, `Here it is. See if you can hit it.' I like that."
Guerrero, who has feasted on Texas pitching throughout his career, returned to the lineup after missing a game with a bruised rib cage and launched a solo homer, his 15th, in the third inning against young Tommy Hunter.
It landed deep in the left-field seats, an estimated 408 feet away.
Hunter's RBI single had given Kazmir a lead before he threw his first pitch. It followed a walk by Maicer Izturis, also back in the lineup after missing a game with hamstring tightness, and a strikeout by Guerrero that became a wild pitch, enabling him to reach first and Izturis to advance to second.
"It wasn't like we gave him a big cushion," Angels manager Mike Scioscia said. "Kaz was pitching with his back against the wall all night against a lineup that can drive the ball. He had to throw a lot of pitches the first couple innings, then settled in."
A spectacular catch at the wall in right by Nelson Cruz robbed Bobby Abreu of three RBIs in the second, leaving the bases loaded.
"That catch Cruz made off Abreu really was a momentum-changer," Scioscia said. "Off the bat, I thought that thing was going to definitely be over his head and off the wall."
Known for its offensive eruptions, the Rangers' offense has fallen silent with two of its big guns, Michael Young and Josh Hamilton, sidelined.
Kazmir got the ball to Bulger for a perfect seventh inning, and then Kevin Jepsen waded through a jam in the eighth. Juan Rivera made a fine running catch in left for the first out, but a pair of singles and Hunter's first error in 265 games -- a throw sailing over second base allowing Ian Kinsler to advance -- had runners at second and third with one out.
Elvis Andrus grounded to Chone Figgins at third, and Jepsen escaped by retiring Andruw Jones on a fly ball to right.
Claiming his 43rd save, Fuentes walked a man before striking out Julio Borbon for the final out.
Kazmir's biggest strikeout came in the fifth when he put away Esteban German after a pair of singles had runners at the corners with one out.
"Three straight fastballs," Napoli said of the German punchout. "The last one really had some jump on it."
Kinsler flied out to end the inning.
The Rangers had only one other baserunner reach scoring position against Kazmir. In the second, Ivan Rodriguez bounced into a double play started by Figgins at third base. Izturis turned a brilliant double play in the third on Andrus' grounder through the middle.
"That's a pretty good team over there," said Rangers president Nolan Ryan. "There aren't many holes on that team, although we've played them pretty good this year."
The Rangers are 9-4 against the Angels, but that's of little consolation now.