Obvious issue would be that in 1955, the Communists were running, nationwide, 17 Candidates with 33,144 Votes. So that makes up about an average of 1,946 votes per candidate. SNP were running 2 candidates with 12,112, with average of 6,056 votes per candidate. The Communists were spred out across the country and only took in 0.1% of the national vote, whilst the SNP were only in Scotland and 0.1%. The SNP saw an overall vote increase, but so did the Communist Party. But the Communists also increased the number of seats they were competing for, whilst the SNP remained static with 2.
Why did the SNP surge? Partially it was to do with the Unionists folding into the Conservatives. With a distinctly Scottish party no longer, well, Scottish, the SNP were able to expand into this empty space and make headway. Note that in 1966, following the merger, the SNP gained 0.5% of the popular vote and ran 23 Candidates, coming fourth in overall vote, and then in 1970 doubles their total votes and won a seat.
The SNP had an advantage thanks to the dissolution of the Unionist Party and merger into the Conservatives. They are a regional Party, so their lower vote share is explained by them running a campaign in which the number of needed voters is very small to make a big impact (as would be seen in Feb 1974, when they won 7 seats with 2%, and later in 2015 when they won 56 with 5.6%). The Communist Party, on the other hand, were party competing across the nation, not in Scotland and Scotland only.
The Communists couldn't replace the SNP, to be blunt. The SNP were localized and used this to their advantage. The Communists ran nationwide. The SNP had the advantage thanks to the Unionists leaving a void, whilst the Communists were often tarred by the actions of the Soviet Union and Warsaw Pact (notably the 1956 Hungarian Revolution, which saw membership drop to a third). Scotland is also not a communist nation waiting to happen; as a Scot myself, I find the notion a bit funny.
Can a Communist splinter in Scotland achive some success? Yeah, a Party like the SSP are reasonable to wank into Minority Governments in devolved Scotland and a significant amount of Westminster Seats. But to have the same unexpected and unprecedented success of the SNP? Yeah, no that's not happening without a major shift in Scottish, British, and global politics.