Mark Cooper's men moved ahead shortly before the half-hour mark when Michael Smith kept his cool after being played in by Massimo Luongo, before Nathan Byrne extended the lead shortly after the restart.
Orient, who remain without a manager after Russell Slade's resignation last month, scored a consolation deep into injury time through Darius Henderson's header, but it was not enough to stop a Swindon side who have now gone eight games unbeaten in League One.
Peterborough United slip to third after being held 2-2 at home by Oldham Athletic.
Goals from Michael Bostwick and Marcus Maddison had cancelled out Jonathan Forte's opener for Oldham, but the visitors left London Road with a point when Genseric Kusunga headed home Mike Jones' corner.
Leaders Bristol City remain unbeaten despite Romaine Sawyers' well-taken 85th minute goal rescuing a 1-1 draw for Walsall.
Milton Keynes Dons triumphed 2-0 at strugglers Yeovil Town, who had Ben Nugent sent off, while Preston North End recovered from going a goal down to see off lowly Colchester United 4-2 at Deepdale.
At the Proact Stadium, Chesterfield held off a late Sheffield United rally to win 3-2 in a match that saw both teams finish with 10 men.
Jimmy Ryan put the hosts ahead, before United were hindered by Michael Higdon's dismissal for a hefty challenge on Daniel Jones, while Chesterfield goalkeeper Tommy Lee too saw red before the break.
However, further goals from Gary Roberts and Eoin Doyle - his 12th of an excellent League One season - put Chesterfield in control and they held on despite late goals from Mark McNulty and Neill Collins.
Crewe Alexandra's miserable campaign continued in a 2-0 loss at Bradford City, while beleaguered Scunthorpe United were beaten 2-1 by Doncaster Rovers.
Fleetwood Town ended an seven-match winless league run with a 1-0 victory over lowly Port Vale, Stephen Dobbie's 80th-minute penalty sinking the visitors who had Jordan Slew sent off.
Rochdale boss Keith Hill suffered disappointment against former club Barnsley, who moved out of the relegation zone as 1-0 win ended a stretch of five league games without a victory - Peter Ramage marking his return on loan from Crystal Palace with the only goal.
Elsewhere, Coventry City were held 2-2 by Crawley Town and Notts County beat Gillingham 1-0.
