Kennedy would've been involved in Vietnam unlike what the book puts forward. His reputation is worse than Nixon's long term because of the numerous mistresses he had and tried to cover up.
The matter at hand is that Vietnam as executed was Lyndon Johnson's way of doing it, which means an American War in Vietnam, and one on LBJ's terms and way of prosecution. Johnson assumed you beat someone into submission and dominated. Kennedy did not share that thinking, nor did he have full faith in the Joint Chiefs which pushed for war, nor did he commit the United States to a War in Vietnam when it was presented to him in 1961 that it was inarguable that the US needed to begin combat operations to save the country. Kennedy avoided every war that was put on his plate to be had, from Cuba to Berlin to Laos, for the simple reason that he thought war should be avoided if possible, there were other means of resolution to be exhausted first, these means to resolution should involve creative thinking, and that potential wars in those theaters were too uncertain. There really is no point in US combat operations in Vietnam, other than to avoid political fallout. You can argue that in the wake of Diem, it became inevitable, but I would also argue against that. Diem's Vietnam was a matter of a national situation not working, and looking grim. In the aftermath of the assassination, it was a worse situation, but still a bad situation nonetheless. The public also is not paying attention to Vietnam in Kennedy's administration; the stark minority that do have little faith in the situation anyway, thinking either Saigon will fall or there'll be a coalition government. There are other options on the table that it would be in Kennedy's behavior to pursue. There was a proposal to neutralize Cuba in exchange for neutralizing Vietnam between the US and USSR. The Soviets were tired of Castro, Castro had lost faith in the Soviets, and Kennedy was discussing renormalization of relations with the Castro government via back channels. For it's part, the United States was obviously tired of the situation in Indochina. Kennedy felt he could sell it to the American people. You can argue on the details, but the argument I see too often is that we did go into Vietnam, so of course we were always going into Vietnam. Or that pot smoking utopians think Kennedy was magic and everything would be perfect, so every positive decision he could have made or positive outcome of his living is just wishful thinking. I disagree with the argument that the man who avoided a nuclear war over Cuba could not avoid a direct American conflict in Vietnam, at a time when it is a third world nation that only has recognition in so far as the United States pays attention to it, when there are other options to be had, when it is not a pressing issue for the American public, and when the recent history of that nation has demonstrated that the United States should not so closely tie itself to that nation, and it's potential failure, as a war would do.
You also cannot take Kennedy's sexual escapades out of context. The reason you do so is because they were the first ones revealed. I believe it was in the Kennedy biography in 1968. Washington and every other center of power (political, economic or otherwise) were "Mad Men" dens of debauchery. Every man who could did have a string of affairs with everyone they could. Lyndon Johnson was as promiscuous as Kennedy. The fact that you did not see "I did not have sexual relations with that woman" on the CBS evening news in 1967 should indicate how likely it is to see JFK giving those speech in another world. Nelson Rockefeller divorced his wife for his mistress, which is why he failed to win the nomination in 1964. Rumor has it Rockefeller died of a heart attack mid coitus with his secretary trapped underneath. Again, anyone with power had a string of affairs and sexual escapades. There are people you know and people you don't know. So Kennedy is only special without context.