There is academic consensus that while India's economy would probably have suffered significantly before the sheer industrial might of the West even without colonization, colonial rule made India particularly worse off. I'll quote an earlier post of mine, adding some emphasis on the way to highlight how the "normal" economic shock of industrial-level imports was exacerbated by the British:
On how British rule "knocked the stuffing out" of India's economy and turned artisans and merchants into peasants, see "Progress and Problems: South Asian Economic and Social History c.1720-1860," p. 79-80:
[E]specially between the 1820s and 1850s, British rule restructured South Asian society and economy in ways meant to serve its own interests and which had the consequence of all but permanently precluding the transformation to modern industrialization. It was in this era that many of the social and economic features, understood by later generations to be the products of changeless tradition and taken by them to constitute the barriers of 'backwardness' to development, can be seen to have crystallized. Recent research has given particular attention to the nature and implications of the long-term price depression which descended on the South Asian economy toward the ends of the 1820s [...] In South Asia, it was exacerbated by three factors which can be directly associated with colonial rule: the export of large quantities of specie to service the China trade; the dismantling of many indigenous court, military, and religious centres, which had provided the main foci of internal demand, and the impact of Lancashire on South Asia's previous overseas and luxury textile markets. The depression, and these particular causes of it, can be seen as having knocked the stuffing out of a large part of South Asia's 'ancien regime' mercantile capitalist economy. Moreover, much of what was left was now taken out of the hands of the indigenous capitalists and passed, via the monopoly powers of the state, to British ones. [...] The principal process of economic change during these years has been described as "peasantization." Displaced soldiers, courtisans, priests and artisans found their way onto the land, which was fast becoming the only available base of subsistence.
On the impact of the British on South India's textile economy, "South India 1770-1840: The Colonial Transition", page 507-509, addressing your point directly:
While 'local' resistance may have kept Utilitarianism and Evangelicalism at arm's length, it could not do the same for the impact of Britain's Industrial Revolution. South India's overseas textile markets collapsed, staunching vital inflows of specie and precipitating a general price collapse which was to last for a generation. [...] Markets atrophied and a long depression sapped commercial vitality. By the time that the depression lifted, in the 1850s, what once had been one of the early modern world's great commercial economies had been turned into a 'backward' agricultural dependency.
How far the fact that South India was under colonial/Company rule directly contributed to this outcome remains a debatable question. The global impact of the British Industrial Revolution - which drastically reduced commodity prices - affected most of the world similarly, whether it was 'colonized' or not. While what might have happened had the supposedly 'modernizing' Mysore state of Hyder Ali and Tipu Sultan defeated the English in 1782 remains one of the teasing counterfactuals of Southern history, it is difficult to think that it could have made much difference by the late nineteeenth century. A 'reactively-modernizing' Mysore must surely have gone the same way as Mohammed Ali's Egypt and the China of the [Tongzhi] Restoration before the West's industrial power.
But in two ways, perhaps, it was important for South India and the wider world that an archaic form of colonial/Company rule was already established before British industrial supremacy became manifest - and ways which re-open questions on the transition(s) of the eighteenth century. In the first place, it guaranteed that the depression would be deep and prolonged and that many of its possible outcomes would be foreclosed. The Company's 'victory' did not only eliminate forces of competition in the market but, as Christopher Bayly argued, also forces of internal consumption and demand. As princely armies were cut back and elite supply trimmed, domestic markets tended to contract - promoting de-urbanization and de-industrialization. Further problems were created by the tendency of the Company to export specie to China and deplete an already constricted money supply.
Added to this, both the mercantilist and the historicist bases of the Company state combined to create a situation in which 'profit' would be sought much more readily through the pursuit of 'rent' than through the expansion of production. With the revenue system dominating the economy, energies were turned away from productive investment (which might attract penal taxation).
On how the Industrial Revolution made India a "colonial economy" which the EIC did nothing to stop,
A Concise History of Modern India, page 76-77
By 1815 Indian textiles and other artisanal commodities could no longer compete with Britain, or on the world market, with British machine-made goods. Within a few years British textiles began to penetrate the Indian market, initiating the development of a classically 'colonial' economy, importing manufactures and exporting raw materials, that was to last for a century, until the 1920s. [...] Although new opportunities for commercial agriculture brought advantage for some, the loss of overseas markets was devastating, especially for skilled weavers in the great weaving centres, such as Dacca and Murshidabad. In the countryside weavers managed to survive by taking advantage of cheap imported thread, but those who had relied on hand spinning for subsistence were often driven back into agriculture. At the same time the rapid decline in the number of Indian courts, lavish spenders on luxury goods and armaments, reduced demand for many commodities. The disbandment of these courts also forced on to the land large numbers of former militiamen and retainers, which in turn further adversely affected artisanal production.
[...] The East India Company during the early decades of the nineteenth century did little to set India on a path of economic growth [...] This 'drain' of wealth was complemented by the Company's withdrawal of funds to cover what it called the 'Home Charges,' including pensions, debt service, and the cost of maintaining the Company's offices. [...] The situation was exacerbated by the Company's forces of deflationary finance, as it sought to trim its budget deficits. Throughout, the heaviest burden India had to bear was that of the land revenue demand. Essential to the support of the army and the administration, these payments, rigorously collected in cash, lay at the heart of the British impact upon the Indian countryside.
From
India: A History, which also supports the theory that British rule drained India's economy, page 390-391:
Yet such was this superstructure of agents and rentiers, and such the extractive culture of the revenue system, that profits rarely found their way back into production other than as advances on the next crop. The actual cultivator thus became, if anything, even more indebted. Commercialisation only "led to differentiation without genuine growth." In effect India’s rural economy was already experiencing the down-side of plantation economics, in terms of labour exploitation, without the usual up-side of capital investment. "The point is not that so many peasants suffered (they would have suffered under capitalist modernisation, too) but that they suffered for nothing."
The British preferred to emphasise their investment in infrastructure, especially railways and irrigation works ("trains and drains"). They also pointed to the country’s generally favourable balance of payments. Critics, though, were less impressed by India’s theoretical prosperity and more exercised by Indians' actual poverty. As early as 1866 Dadabhai Naoroji, the future "Grand Old Man of Congress," had begun to wonder whom the trains actually benefited and whither the drains actually led. In fact he developed a "drain theory" which, with ramifications provided by his successors, would run like an undercurrent throughout the nationalist debate.
This ‘drain theory’ maintained that India’s surplus, instead of being invested so as to create the modernised and industrialised economy needed to support a growing population, was being drained away by the ruling power. The main drain emptied in London with a flood of what the government called "home charges." These included salaries and pensions for government and army officers, military purchases, India Office overheads, debt servicing, and the guaranteed interest payable to private investors in India’s railways. Calculated in sterling at an increasingly unfavourable rate of exchange, they came to something like a quarter of the government of India’s total revenue. With much of what remained being squandered on administrative extravagances and military adventures in Burma and Afghanistan, it was not surprising that Indians lived in such abject poverty or that famines were so frequent.
The theory also included an analysis of how the drain actually worked. The Secretary of State for India in London obtained sterling to meet his ‘home charges’ by selling bills of exchange to British importers. Presented in India, these bills could be converted into rupees out of government revenues and so used for the purchase of Indian produce. The private sector therefore played an important part in the drain since its exports from India constituted the drain’s flow. By the same token the export surplus was of little economic benefit to Indians; and worse still, since they consisted mostly of raw materials, exports gave no encouragement to India’s industrialisation. The classic case was cotton. In the days of the Company, British purchases had been mainly of finished piece-goods. Latterly, with Lancashire’s mills underselling India’s handloom weavers, British purchases switched to raw cotton and yarn. Now, when new and often Indian-owned mills in Bombay were at last in a position to compete, they were repeatedly frustrated by tariff policies which favoured British imports and by regulations which handicapped Indian production.
India’s embryonic industries – principally jute, cotton, coir and coal – needed protection; the British insisted on free trade. Their laissez faire attitudes extended even to the land revenue, where rising prices meant that fixed revenue assessments actually became somewhat less onerous during the latter half of the nineteenth century. But rather than adjust such assessments the government now preferred to explore other sources of revenue, like introducing an income tax.
