Visitor spending was down accounting for a $20B loss in spending and tourism leaders say the economy is to blame.