It is not your EGR, but it could be the intake throttle. These have plastic gears inside and will make a grinding noise if it fails. Try unplugging the wiring to the throttle and see if the noise disappears.

Is it the valve that cuts off the intake on shut down? Is it part of the EGR?

My Wife's TDI has a throttle plate to provide suction to suck in the exhaust from the EGR.

I wonder if removing the butterfly plate, removing the plastic gears, Let the motor run so the ECM is happy, and plating off the EGR inlet would get rid of the noise?

Some of the old '50s IDI diesels like MB and some IH tractors like the early B275D had a throttle, Vinturi, and a diaphragm that looked like a small brake booster. Vacuum on the booster from the engine sucking against the diaphragm would slow down the engine. Open up the throttle, the inline pump rack would go full bore until the suction in the venturi moved the rack to reduce fuel. Nissan had this system on their Diesel Pickups for a few years, I think Joe Romas had one and may be able to tell you more. I remember these type of diesels were not as responsive as our CRDs to say the least.