Federal Reserve official Goolsby said that if tariffs persist, the Fed will have to consider their impact.