It has been hypothesized that the Vedda were probably the earliest inhabitants of Sri Lanka and have lived in the island before the arrival of Sinhalese from India. According to the 5th-century genesis chronicle of the Sinhala people, the Mahavamsa ("Great Chronicle"), the Vedda are descended from Prince Vijaya (6th–5th century BCE), the founding father of the nation, who originated from Eastern India, through Kuveni, a woman of the indigenous Yakkha (Odia/Pali for yaksha) whom he married. The Mahavansa relates that following the repudiation of Kuveni by Vijaya, in favour of a Kshatriya-caste princess from Pandya, their two children, a boy and a girl, departed to the region of Sumanakuta (Sri Pada or Adam's Peak in the Ratnapura District), where they multiplied, giving rise to the Veddas. Anthropologists such as Charles Gabriel Seligman believed the Veddas to be identical to the Yakkha.
Veddas are also mentioned in Robert Knox's history of his captivity by the King of Kandy in the 17th century. Knox described them as "wild men", but also said there was a "tamer sort", and that the latter sometimes served in the king's army. The Ratnapura District, which is part of the Sabaragamuwa Province, is known to have been inhabited by the Veddas in the distant past. This has been shown by scholars like Nandadeva Wijesekera. The very name Sabaragamuwa is believed to have meant the village of the Sabaras or "forest barbarians". Place-names such as Vedda-gala (Vedda Rock), Vedda-ela (Vedda Canal) and Vedi-Kanda (Vedda Mountain) in the Ratnapura District also bear testimony to this. As Wijesekera observes, a strong Vedda element is discernible in the population of Vedda-gala and its environs.