Discovering them was not so easy because they don't form chemical compounds. So they exist only as gases. And Helium disperses off into space.
Argon was the first to be found by density considerations because it is so abundant on Earth's atmosphere (~1%): Argon is abundant on Earth's atmosphere because it comes from the decay of Potassium-40.
Then basically all of the others were discovered by spectral lines. Helium notably was first found on the Sun like that, and only later on Earth! Thus its name. Pretty cool.