Lea Valley conceded the advantage in the title race after a disappointing performance allowed Mayfield Athletic to go home with a 2-0 victory and a two-point lead at the top of the table with a game in hand.
The Swans lacked quality in midfield with two players missing through suspension and found themselves two behind after only 37 minutes following some chaotic defending.
The introduction of Michael Nwosu and Jamal Francis at the break gave the team a more balanced look and the game opened up with chances being created by both sides.
But after the Swans' Rhys Rossiter was handed his marching orders for a second yellow, the hosts were left chasing the game and rarely looked capable of pulling level.
It was a much happier afternoon for Lea Valley's Under-11 side who progressed to the final of the League Cup with a 7-2 victory over Woodford Youth United.
The talented Valley team, who also booked a place in the County Cup final only last week, made it two in a row thanks to a hat-trick from Harrison McGillicuddy and further goals from Nigel Yeboah, Jeremie Sivi, Steven Kabungwe and Abraham Ladoja.
Woodford were unable to match the young Swans’ movement and were constantly stretched by some excellent wing play from Liam Babb and Ethan Sealy.
The visitors were on level terms at half time but Valley strolled through the gears in the second half and eased to a comfortable win.