I know that under primogeniture rules (specifically, late-medieval and modern English rules of noble inheritance), inheritance first goes downstream to the senior surviving descendant of the guy who just died (for example, when the 3rd Duke of Earl dies, the Dukedom passes to the 3rd Duke's senior descendant). If the 3rd Duke has no surviving heirs of his body (i.e. he has no kids or all his descendants predecease him, leaving his branch of the family extinct), then the title reverts to the 2nd Duke of Earl and (since the 2nd Duke is already dead, or there wouldn't be a 3rd Duke) it's immediately inherited by the senior surviving descendant of the 2nd Duke (e.g. the 3rd Duke's younger brother). If the 2nd Duke also has no surviving descendants, the title reverts to the 1st Duke and passes to the 1st Duke's current heir. And if the 1st Duke has no surviving descendants, the title reverts back to whoever created the title and granted it to the 1st Duke (usually the King -- the title "reverts to the Crown").
Extending the same principle to early-medieval divided inheritance: suppose the father (Alfred) was Count of A, B, and C and passed the County of A to the eldest son (Bill), the County of B to the second son (Bob), and the Count of C to the youngest son (Brian). If Bob dies without issue before Bill and Brian, then the County of B reverts to the late Alfred, and then is divided, negotiated over, or fought over by Bill and Brian.
However, if Alfred were Duke of D (comprising the future Counties of A, B, and C) and during his lifetime creates Bill Count of A, Bob Count of B, and Brian Count of C, and on his deathbed leaves the senior title Duke of D to Bill, then when Bob dies the County of B reverts to the Duke of D, since Bob's the first Count of B and has no heirs of his body.
Of course, rules of inheritance were fuzzier in those days. In either case, if Bob named either Bill or Brian, or even someone who wasn't part of the family but who had the confidence of Bob's vassals and retainers, as his heir in default of heirs of his body, then that would probably stick. Likewise, if Bill, or Brian, or the King, or anyone else with a plausible pretext came in with enough military and political support, they'd probably be able to make a claim of their own stick.