Kennedy Jr. rejected characterizations of him as an anti-vaxxer in a Senate hearing Wednesday where senators will weigh his confirmation as Health and Human Services Secretary—as his former billionaire running mate threatens to fund primary challenges against lawmakers who vote against him.

Kennedy’s bid to serve as America’s top health official may be decided by a handful of Republicans, including several senators who questioned him Thursday.