Uncertainty surrounding the back-and-forth rollout of tariffs has eroded consumer confidence among would-be travelers.