as the altitude gets higher, the temperature decreases
The reason it's actually colder is because as you go up in the atmosphere is that the Earth's atmosphere feels less pressure the higher up you go. So as the gas in the atmosphere rises it feels less pressure, which makes it expand. When the gas expands it does some work.
Hope that helps