Having to ship the raw material halfway across the world. More stringent environmental protections--fabs generate many tons of hazardous waste. Higher healthcare costs for the workers. Plus they use lots of water which is more expensive in Arizona.
Considering how the low cost of shipping enabled globalization I doubt the shipping costs matter much. I'm also not sure if environmental protections are that much more stringent in the US than in Taiwan. Higher healthcare costs and water might make sense, but the bulk of the water cost in semiconductor manufacturing is probably due to the purification processes required than the volume of water used.
Shipping costs matter some when you go from being right next to the factory that makes your raw silicon wafers to having them be halfway around the world. In general it sounds like a big factor is cost - you've got to get talent that just doesn't exist (or isn't as widely available) in the US.
