A federal judge in Georgia has denied Sen. Lindsey Graham's effort to delay his testimony before the Fulton County grand