Once you have used it for the first time, starting it up will be a lot quicker. The first time you open GIMP, it will take awhile because it needs to read all of the plugins, fonts, etc. Once you have downloaded the installer, click on it and it will, of course, install GIMP and GTK and create a program file for them. When you click on the download link, it should redirect you to the download site where you will need to click on the download tab.

First you need to download the file, which can be found here, the latest version will always be at the top of the page.