You'd need a different level of agricultural tech, for starters, in order to have what is non-necessary for subsistence be common and widely cultivated.
There were plenty of non-subsistence crops in medieval times, such as sugarcane, cotton, hops, mulberry (for silkworms), herbs and spices such as marjoram.
There were also staple crops that were grown enough expensively enough in a different climate to become cash crops instead (rice in the Middle East and North Africa, outside Egypt).
I have not heard of Sufis using Cannabis. I have heard claims that the Shia Ismaeli Nizari sect called the Assassins used hashish to inspire their warriors to carry out missions, but these stories are generally considered to be untrue urban legends / myth by academic historians.
The Assassins themselves probably didn't consume hashish, but hashish existed at the time. A few obscure Sufi orders such as the Qalandars probably used cannabis ritually. It gradually became more prevalent in Egypt from the Ayyubid period onward, though it was likely fairly uncommon.