The German and Austrian Jewish population was about 750,000, of which three quarters were exterminated. Whereas the total German population was about 70 million. But 1941, when the extermination program began the number of Jewish forced labourers in German was 60,000, compared to the 2,000,000 foreign labourers (Fremdarbeiter) [source]. The Nazis decided that this was not an economic impediment to the Holocaust. Furthermore, in regions where the sudden absence of Jewish Labour would have been a problem they delayed the process to allow for their replacement.
It seems logical that the movement from forced labour to Extermination through Labour and the Holocaust of the Jewish population would have lead to costs (from the inefficiency of moving skilled workers to hard labour, to the expenses of diverting effort to commit to commit these awful crimes). These must have been the disruptions the Nazis had in mind and saw as no economic reason not to proceed. I will say the Jews brought from conquered lands for forced labour probably would have been an overall benefit to the German war effort, but I'm not including that as the "holocaust" as I'm assuming the alternative the questioner has in mind is assigning these foreign Jews to the Fremdarbeiter system anyway.