Dalton's Bond career was fundamentally harmed by Roger Moore not accepting the situation and moving more gracefully out of the way after Moonraker. They'd done Bond-as-sci-fi at that point, catching the late-Seventies wave, and it'd damn near ended up Carry On Moonraking, and really it was time for a change. That was done with scale and scope and feel with For Your Eyes Only -- which looks and 'reads' more like one of the Fleming short stories despite the slightly bizarre and clanky Bloefeld opening -- but at that point it was time for someone new. And depending on Dalton's filming schedule with Flash Gordon it should've been him right then. This was the peak of Ian Ogilvy's career, the late Seventies, but I think he would have seemed to much like a bargain-window Moore where Dalton would have brought a flavor to Bond that would've suited FYEO frankly better than Moore's. It leaves Moore at best batting .500 as bond (Live and Let Die despite its racist overtones was a solid effort and Spy Who Loved Me is the definitive Moore bond, but Golden Gun and Moonraker -- despite the game efforts of the always excellent Michael Lonsdale ["Look after Mister Bond. See that some harm comes to him."] -- was a hot mess. Besides being a genuinely skilled actor (especially if you've seen his early stuff from the late Sixties, just watch him in Lion in Winter) Dalton would've suited the Eighties milieu much better. And it would've gotten him a nice long run, probably three films (if not the exact same ones) before Living Daylights even shows up and a corpus that would allow him to be judged fairly among the Bonds. And I even have some time for Brosnan -- the first two were yeomanlike efforts, it's the drop off the cliff of the second two that puts him in a rather Roger Moore-like situation. Interestingly enough as a fan of the first (i.e. the good) season of Remington Steele, like Moore Pierce Brosnan always struck me as much better suited to play Simon Templar in a proper revival of The Saint, leaving Bond to actors like Dalton.
Setting aside the irony of replacing Sean Connery with an actor who was even older(!), I think Eon could probably defend keeping him for as late as For Your Eyes Only, which turned out to be one of his better efforts (and his last decent outing), even if we was getting a little past his sell-by date.
But otherwise, I agree. Of course, a new Bond was not enough; what was needed was better scripts than they were getting in the 80's. Dalton was/is an outstanding actor, and it was clear he wanted to take the character in a different direction from the detached one-liner-and-gadgets facade that Moore was comfortable living in; but he needed more to work with.
Brosnan was almost borne to play Bond. But he needed good scripts and good direction to thrive, and the only film he got that on was Goldeneye.