The Hajj is purely of pagan origin adopted directly into Islam from Arab pagan polytheism.
"That Islam was conceived in idolatry is shown by the fact that many rituals performed in the name of Allah were connected with the pagan worship that existed before Islam. ... Because the rituals involved with the Islamic Pilgrimage are either identical or very close to the pre-Islamic pagan idol worship at Mecca. (Is Allah The Same God As The God Of The Bible?, M. J. Afshari, p 6, 8-9)
"Muslims are notoriously loathe to preserve traditions of earlier paganism and like to garble what pre-Islamic history they permit to survive in anachronistic terms" (Southern Arabia, 1944, p 398)