As the legend goes, on the island of Sveti Stefan, which later became cultural and historical center of Pastrovići, the fortress was built in 1442 and that is when it was first inhabited. The fortress was surrounded by walls to protect the families from nearby villages from Turkish invasion and pirates. The legend says that the settlement was set up after the Turkish siege of the town of Kotor, when Pastrovići tribe decided to help defend the town. Pastrovići and Kotor won the battle over the Turkish army. On their way back, in the vicinity of the Jaz beach and not far from the Old Town Budva, they noticed Turkish galleons, attacked them and captured a large amount of money. The money was used to build the fortress with a house for each of the twelve tribes of Pastrovići. This fortified settlement was named Sveti Stefan, after the church of Sveti Stefan. Thanks to its significant position, Sveti Stefan was commercial and transportation center of Pastrovići. At the time of the Venetian Republic and trade with Venice, it had great strategic and commercial importance.


There are four small churches on Sveti Stefan Island: Sveti Stefan Church, after which the island was named, situated on the highest part of the island; the Church of Aleksandar Nevski, the Church of the relics of St. Stephen the Archdeacon, and the smallest of them, dedicated to the Transfiguration, which lies right next to the town gates connected to the mainland by a narrow isthmus.


Vila Miločer was the summer residence of Queen Marija Karadordević in 20th centery. She enjoyed sunsets from the place then named Queen’s chair. This special place sits on the top of a rocky cliff emerging from the sea, as it was queens favorite place for pleasure and relaxation during her stay at royal summer residence as it gives a breathtaking view of the Old Town Budva, the island of Sveti Nikola and the peninsula of Sveti Stefan, thus being one of the most stunning viewpoints on the Budva Riviera

In 1960s island was transformed in hotel and it was a playground for the rich and famous until 1980s while Vila Miločer was summer residence for Yugoslavian President Josip Broz Tito.


Among the celebrities and statesmen who stayed on the island of Sveti Stefan are numerous Soviet Marshals: Timoshenko, Grechko, Kirilenko, Jakubovski, novelists Malro and Moravia, Soviet astronaut led by Gagarin, Greek composer Mikis Theodorakis, movie stars and other celebrities such as: Sophia Loren, Marina Valdi, Kirk Douglas, Sergei Bandarchuk, Sylvester Stallone, Carlo Ponti,, Richard Widmark, Klaus Kinski, Jeremy Irons, Monica Vitti, Boris Spassky and Bobby Fischer, Leonid Leonov, Sandro Pertini, Willy Brandt, Giulio Andreotti, Helmut Schmidt, Robert McNamara, Claudia Schifferand many more.