The FA Cup winners took the lead through Santi Cazorla's angled shot and then added to that when Aaron Ramsey drove in a strike to finish a counter-attack.
Premier League champions City came close when Stevan Jovetic had a header hit the post and a shot saved. But Olivier Giroud's dipping 25-yard strike secured Arsenal's win, The BBC reported.
The victory is the first for Arsene Wenger's side in the English season's curtain raiser since 2004, and should give them plenty of belief before they start their top-flight campaign against Crystal Palace on Saturday.
The Gunners fielded three of their four summer signings as forward Alexis Sanchez, right-back Mathieu Debuchy and defender Calum Chambers started the game.
City were missing eight key players and, while manager Manuel Pellegrini may play down the defeat, it will serve as a warning sign before they begin the defence of their Premier League title at Newcastle on Sunday.
Arsenal has won their second trophy in 85 days after going 3283 days without silverware prior to that, the report added.