I think we need to untangle three things to get to good answers. There are separate questioons here:
- Could small states colonise?
- Could HRE states colonise
- Could German states colonise?
Well, I would say the question about small states is pretty unequivocally answered. Denmark may be a fluke in terms of the scale of its empire, but it does show the possibility. Nothing stops a minor European power from acquiring colonies (as lkong as acquiring colopnies is feasible as such - I am not advocating a TL where Hesse-Cassel occupies the Ryukyu Islands) And I think we can agree that there is nothing specific about Germannes that makes German states less likely to successfully colonise. In fact, while it is possible to be more German than early modern Denmark, you can't really be a lot more German. Large parts of Denmark's most prosperous territory, many of its larger cities and a big chunk of its upper class were (Low-)German-speaking.
The problem lies in the political setup of the HRE. Courland and Prussia both successfully acquired colonies, and Austria and Prussia ran East India Companies. Part of the difficulty lay in the fact that most coastal parts of the HRE were politically poorly suited to colonisation (its best Atlantic-side ports were in the Elbe and Weser estuary, both controlled by free cities without the military force or capital depth required, and the other coastal territories were poorly placed - East Frisia, Hadelnm and Jever too small and disunited, Bremen Swedish for a long time, Schleswig, Holstein and Oldenburg Danish. Its most prosperopus cities were content to trade up the Rhine in Dutch ships. And Flanders, while it was used by the Austrian East India Company, was always thought of as a war zone more than a trading port by its rulers. The government in Vienna had other things on its mind. When prussia acquired Ostfriesland, it set up a trading company, but the results were similar. Hanover, once constituted, could have colonised, had its government been interested, but ruling in personal union with Great Britain, the Elector already had colonies. The significant powers in the HRE were all oriented elsewhere.
In addition to this, we have a problem unique to the structure of the HRE. Its smaller states were effectively independent and could not count on eacxh other's suppoort for such ventures (combining the capital of the archbishoprics with the maritime capabilities of the North Sea ports could have gotten somewhere, but coordinating it would make an EU military look simple). However, diplomatically, they were still considered a unitary entity, and every time Austria was at war with anyone else (about ervery other year), they were techincally belligerents. Mostly, they needn't have cared much, but it made any ships or overseas possessions targets for enemy privateers. One of the big problems for Hamburg, Lübeck and Bremen between 1650 and 1770 was that in each and every conflict between France and Austria (or one involving the Netherlands and Britain) meant their trade suffered. Hamburg's navy, BTW, counted coup against Jean Barth, not that it helped much in the big picture). That is a big strike against a Holy Roman state successfully acquiring colonies.
