I'm a big History buff,and have thought along the same lines of why was the Dutch Empire in the long run not as successful as the Spanish,English,Portuguese,or even the French (another Empire,that while large,lacked the ability to populate their colonies with Frenchmen to any large degree).
Someone mentioned home country population as a cause.That would be part of it true,Spain and England had larger populations to work with.But Portugal in those days (18th century) only had maybe 3 million people,to the Dutch Republic's around 2.5 million.Yet they were able to send over 400 thousand settlers to Brazil alone in that century,not to count the Portuguese in other parts of their Empire.The French,which at that time had as many people as England and Spain combined,didn't manage to populate their colonies.So population isn't the main reason.As well as the Dutch home population,they had access to Germans and Scandinavians,that always showed themselves ready to immigrate to the Netherlands at the time.
Then it was mentioned that because they were a rich country,maybe people wouldn't immigrate to the colonies.But England,for the period was becoming a rich country as well,yet they were able to send people.And when they needed to,they used resources that would have been even easier for the Dutch to use,ie.German Palatines (most of them went though Holland before shipping to England),and French Huguenots(again,Holland was one of their best refuges).Not to mention,that even rich societies have problems,the late 17th and the whole 18th centuries,where times of economic troubles in the Netherlands,and the many wars,might have convinced a steady stream of people to immigrate to colonies.
So I think that leaves the most valid problem,and I think the crux of the whole matter.The fact that the Dutch leadership showed no long term vision.As someone said,they were only interested in profit.While that was true of all the Colonial powers as well,the others seemed to have a more long term outlook.The Spanish came for gold,but stayed for souls and land.The Portuguese,similar to the Spanish,but also wanted to stop others from freezing them out of the East,and they figured they had to settle to stop them.The English were a mixed bag.They hoped to find gold and silver like the Spanish.And when that failed,there was a feeling among some English that they could make a "New England" in the Americas and that started them coming in numbers.The French had good ideas in the beginning,but weren't able to keep to a consistent policy.So in the long run they failed (too simple an answer I know,but the subject is very complicated,and would take an essay to answer).
Then there were other important factors to consider about settlement colonies.Climate,native populations (large or small),ease of access (shipping),resource base.My theory is that there were 2 types of colonies in the world,settlement colonies and trading colonies.Sometimes they overlapped,and sometimes one was started for one purpose and ended up as the other.Colonies that I consider as settlement colonies are now,the U.S.,Canada,Australia,New Zealand,all the Spanish Latin American colonies (or Kingdoms,as the Spanish thought of them),Brazil,and South Africa.Other colonies would come under the heading of trading colonies.Sure they in time became regular colonies,but were never seriously thought of for mass European settlement.
Climate:
You will notice that all the colonies that made it as settlement colonies had climates that were temperate,sub-tropical,or at least large parts of them were.In the 16th,17th,18th centuries people didn't have the technology to live happily in either extreme cold climates or extreme hot climates (not that people like that today either,lol)Sometimes profit from resources would modify that,but not usually.The areas settled heavily in Canada,were the least cold at first,and in Latin America heavier settlement was away from the hottest areas.The U.S.,Australia,New Zealand and South Africa were not too cold or too hot (except in certain areas).
Native Populations:
In Canada,the U.S.,Australia and New Zealand,the native population wasn't large enough to be a long term hindrance to settlement.Short term yes,but never long term.In the Spanish Latin American colonies within the first century the Indian population was more than halved through disease.Something unintended by the Spanish,but peoples knowledge about disease was so limited then that it was probably unavailable.Between that and the Christianizing of the Indians,most of the heavier Indian population areas were left loyal to the Spanish and open for settlement.Portuguese Brazil was lightly settled by native Indians along the lines of the U.S. or the lightest Indian areas in the Spanish colonies,ie:the North of Mexico,or the Pampas of Argentina,etc. South Africa (which I think ultimately is the subject for this possible ATL on the Dutch Empire)was not heavily populated at that time.In the west of SA,very small native populations,and in the east,while heavier,were not any heavier than that,say of the U.S. Eastern Indian population at the start of settlement.Europeans were never able to settle in numbers in colonies with a population of high civilization similar or superior to their own,ie:India and China,Japan,etc.
Ease of access
We have to remember that in those days the only way to get to your colony was by sea.With the level of technology available then that was quite an effort.I think from England or Spain to the Northern Americas it took over a month on ship to get there,and that was without serious problems popping up.To reach say Argentina(or La Plata as it was known then) and the Cape in SA,was something like 3 or more months.One of the main reasons that the English were so successful and the Spanish starting in the 17th century were having problems in colonial development was the strength of the British Navy,and the weakness of the Spanish Navy.The Portuguese were able to keep up enough shipping and under somewhat of British protection make it work.France,while having a large navy,were in constant turmoil and wars in much of the period,and without profitable colonies didn't use there navies to their best abilities. The Dutch had probably the second (and for a time the first)largest navy of the period and would have been easily able to obtain access.As mentioned earlier their problem,is will,not ability.
Resource Base:
To make a settlement colony successful,you have to have a resource base to support the settlers and bring profit to the home country(even if only long term) or a resource that is hard to get somewhere else.In the U.S. and Canada,the fur trade,to which later was added lumber trade,was enough to keep the home countries interested until settlement could take off (especially in the U.S.),and other resources supplement them.In Spanish Latin America,silver and gold hooked Spain in their desire to hold the colonies until later agriculture,became even more profitable.The Portuguese in Brazil started out with large agricultural profits (sugar) and then in the late 17th and early 18th centuries discovered an "El Dorado" of gold and diamonds to supplement their resource base in the Colony,and almost support the home country as well.In South Africa,part of the Dutch failing was the lack of a resource base.Which again,was the fault of the Dutch powers that be.With a little effort they would have discovered the diamonds and gold that SA is noted for today.The Veld would have provided a paradise for a cattle and sheep industry (as the Spanish and Portuguese made in Mexico,Argentina and Brazil).And the varied climate would have allowed the production of tobacco,sugarcane,cotton,coffee,and various spices which had a ready market in Europe at the time.
Well,while trying to be as short as possible (believe me I was.There is oh so much more I could have said,lol).With this outline I hope I've helped anyone wanting to do an ATL for the Dutch Empire.I will try to post something on possibilities to change things if I have time later.