After right slams Yoder on immigration, he pledges to work with Trump to ‘fix this’