Introduction to Scott's Castle: Scotty`s Castle is located 3 miles east of Highway 267. It is the most famous building in Death Valley. It was built by the infamous little man named Scott. Built by merchants.