It should not be, no. But any leaks between the turbo outlet and the throttle body could cause this. Leaks on the pressure side are very bad for the turbo - you do not want to run this way or you risk overspeeding the turbo and causing severe damage. You could also have a problem with the vane control on the turbo or *possibly* an exhaust obstruction.

You need to determine whether you have a boost leak, a problem with vane actuation, or a blockage in the exhaust. Is the CEL illuminated? I would think a low boost issue would trigger a CEL.

No, the inlet to the turbo will not affect boost. Only plumbing after the turbo, not before.

Not entirely true, if the hose came apart and covered the inlet vanes with debris or worse yet blew stuff into the the Charge Air Cooler (CAC/intercooler) and stopped up the flues, it could very well negatively affect boost pressure especially when the turbo is calling for higher demand.
That is the way to stop a run-a-way diesel, you stuff something into the suction of the turbo real quick to choke off the air flow to the engine and thus shut it down.
Variable vane issues like sticking vanes that won't fully operate to max or a bad turbo modulator valve, or low vacuum to the controller, or a leaking or cracked pipe from anywhere in the vacuum system to the modular valve can also limit boost pressure in a big way.
Another possibility is a partly stopped up CAT which will raise backpressure so high on the outlet side of the turbo that it can not make full boost pressure.

If something went into the inlet of the turbo, that turbo is done for. The blades are extremely delicate - they will not survive any sort of impact. If something went in, and then through into the intercooler, that damage would be very obvious, and you're shopping for a new turbo. That looks like this:

Warranty won't cover abuse... If there's concern something went through the turbo, remove the inlet ducting and inspect!

If it looks fine it likely is. Why do you think you have low boost? A code? Live data? A boost gauge?

Best to get rid of the CAT and the choke point right in front of it if you can.
Replace all of it with a piece of 2.5" exhaust pipe and never look back.
Your engine and turbo will thank you for it!

As for short term do you think I can just hollow out the catalytic converter?

Absolutely! Have busted out the guts of many a CAT over the years and put them back on.