You need to read Tooze. But basically, there was a lot of momentum in German air production and they were trying very hard to get numbers up.
Tooze had a flawed thesis. Overy's analysis is better, though using Tooze as a supplement is helpful.
German aircraft production was badly mismanaged and with more manpower, resources, and factory floor space the Germans produced less than the British aircraft industry. Things didn't get better until Udet killed himself, Milch took over and cleaned up the mess he left, and Speer took over the economy, which meant things were finally centralized and competing bureaucracies demolished. With the same amount of labor and raw materials aircraft production quadrupled by 1944. The problem was gross inefficiencies in production, even though German industry was fully leveraged into war production from at least 1938.
The Germans did try to upgrade aircraft production starting October 1940, Unfortunately (for the Nazi's) due to shortages of raw materials and the sheer bureaucratic inefficiency this effect was not really felt until 1942. And even then it still did not achieve the economies of scale that the americans and british could bring to the table. The Nazi's did outsource work especially to the french and benelux countries but again due to shortages of raw materials combined with low productivity and sabotage Western Europe contribution to the Luftwaffe was minimal (iirc the total contribution of occupied france between 40-44 was something pathetic like 712 fighters.) Also there is no chance of Germany capturing the middle east oilfields any time soon unless the royal navy can somehow be forced out of the eastern Mediterranean, even then the logistical challenges will be immense.
There was no problem with raw materials for aircraft production, but rather horrific mismanagement and inefficiencies caused by Udet. Overall there was very little direction and lots of hoarding of raw materials by factories for post war civilian orders. Some factories were producing goods off the books for sale on the black market and for stockpiling to meet the expected postwar demand, while others were using horrible machining methods that were wasting 1.5 tons of aluminum per engine! The problem didn't even begin to be solved until Udet committed suicide because he realized how badly he messed up, which let the professional Erhard Milch clean up the mess and quadruple aircraft production with the same manpower and raw material base from 1942-4.
Part of the problem with starting so late was that the Allies had started their bombing offensive, so the Germans could not have centralized factories of the scale the US or British achieved, as they would have been too easy a target. For those that were large scale they were used for types that Germany never used, like the He177 of which over 1200 units were produced and barely saw any service because of Udet's dive bombing order. Its not fair to talk about non-German production either, as they were mostly not used for combat models, but rather trainers and reconnaissance aircraft, so as to free up German factories for war models. France IIRC produced some 5k aircraft total for Germany, most were not combat models.