If you want to create a new website, you don't have to immediately register a domain name and buy a website hosting plan.
1) You might change your plan later and find that the domain name not suitable to your content.
2) You can complete the website development and release to live server after fill up with substantial contents.
3) You can show to your customers from your PC how the website works without connection to Internet and etc.
Assuming you already have XAMPP (if not, you can find the installation guide here), you can easily install Joomla to your local Windows PC and develop your website before release to internet. Here, I am using Joomla 3.1.5 as example. You can actually install Wordpress, Drupal and many types of PHP/MySQL related program with the same steps. Remember to turn on XAMPP on your PC.
Go to Joomla website to get the latest version of Joomla CMS. Select Download button. The current version is 3.1.5 as of this point.
