It is probably in the eleventh century that the castle is erected. It is a small square fortress perched on a rock and protected by a moat and a second wall above the valley. A reference dating from 1192 indicates that Wauthtier de Wiltz and Beaufort is the first lord. In 1348, the property passes to the House of Orley following the marriage of Adelaïde de Beaufort and Guillaume de Orley. The lords of Orley realize important enlargements. In 1477, Maximilian of Austria grants the castle to Jean Bayer of Boppard after Jean de Orley-Beaufort lost his confidence. In 1539, Bernard de Velbrück became the lord of Beaufort by marriage and added a wing of Renaissance style adorned with cross over the medieval walls.
The castle then goes to the hands of Gaspard de Heu, who married Velbrück's granddaughter. Partisan of the Dutch resistance and the House of Orange, de Heu is captured by the Spaniards, accused of heresy and betrayal, and executed on the public square of the Marché-aux-Pisces of Luxembourg in 1593.
Philip II of Spain confiscated the property and entrusted it to Pierre-Ernest I of Mansfeld, governor of Luxembourg. By marriage, the castle becomes the property of Henri de Chalon then Gaspard du Bost-Moulin, which is forced to sell it after it was reduced to ruins during the Thirty Years War. By order of the King of Spain, Jean de Beck, governor of Luxembourg, bought a large part of the property in 1639. He is at the origin of the construction of the Renaissance castle which begins in 1643. It is his son who will complete works in 1649, after John of Beck died in 1648 from his wounds received at the Battle of Lens.
After several changes of owners, including Pierre de Coumont (1774) and Jean Théodore Baron de Tornaco-Vervoy (1781), the castle is abandoned and falls into ruins. It is even used as a career in the early nineteenth century. In 1893, the new owner, Henri Even, restored the building and in 1928, Edmond Linkels cleared the rubble and opened the medieval castle to visitors. In 1981, the property is bought by the State and the whole site is classified national monument since 1988.
Today, the prestigious furnishings of the interior, the gardens, the stables and the distillery are preserved in the same states as during the days when the last owners lived in the castle.