Why is space travel so hard?

Because of the gravity well!

This is a gravity coin well.

One of these bad boys is placed in our city's zoo. I really loved to play with them when I was a kid. If you want to see one in action, click the picture!
Normally, when people think of "space" they think that we have to reach a certain heigth above the Earth's surface. This idea is flawed.
Just like this lovely coin game pictured above, you can't just send something out really far and expect it to stay there. The reason is obvious, the surface of the "coin well" is curved. It will always roll back to the middle.
Normally spacetime, the "space" our spaceships move in, is flat. With the gravity of massive objects, like Earth, a similar scenario to the "coin well" emerges. Spacetime becomes bent.

Now, just like the coins on the coin well roll back to the middle, so do spaceships. You might have heard about Newton's First Law - an object in motion will stay in motion, unless acted upon by an external force.
The spaceship, just like the coin, has some initial velocity. Be it by you tossing the coin or a fiery rocket sending the spaceship into space. Either object, wants to keep the direction of its initial stimulus but there is an external force! In most cases, when you toss a coin or send out a spaceship into space, this initial stimulus is much greater in value than the external force, otherwise the respective object would roll quickly into the well's deepest point.
With a low external force, you only have a slight deviation from the objects original direction acting upon it at all times. When you watch a spaceship take off from Earth it is hence rightly to assume that it simply goes up.
But it only does so for a very short time to leave our atmosphere. Our atmosphere is very thick. Many nitrogen and oxygen atoms are hitting the rocket and slowing it down.
Due to this atmospheric drag, satellites do not usually orbit our Earth below 300 km above the surface. But it doesn't matter how far your fuel can take you away from Earth.
The gravity well, due to the enormous mass of our Earth, might decrease with distance but it still is always pulling us back!
No matter how much fuel we take with us, when we run out of it, there is nothing to stop Earth from pulling us back eventually.
(More exactly: Because the gravitational attraction decreases with an inverse square of its distance, you could leave Earth forever with a certain amount of speed because the attraction is negligible at an infinite distance from Earth. But the required speed is significantly higher than the speed to orbit and is extremely expensive to achieve.)

How can we go to space then?

In Physics there is the term "equipotential lines". These indicate locations with equal potential energy on which one can stay, on certain conditions, without losing energy.
In a 2D-model they would be circularly aligned around the phenoma that creates the potential well, i.e. in our case - Earth.

That's right, we go in circles around Earth! This way we can make sure not to roll back.
Furthermore, because of the absence of a significant amount of particles at a certain altitude (above 300 km), there is no possibility of air resistance slowing us down.
Back to our original idea of "going to space". We are now at roughly 300km. At this point we will point our spaceship horizontally to the Earth's surface to gain horizontal velocity to "orbit" around Earth in circles.
Why do we need horizontal velocity? We are still falling towards Earth. We will now draw a circle at a certain heigth around Earth - the place where our spaceship now resides, for instance 300 km.
Additionally, we will give our spaceship enough horizontal velocity, so that it can move along the circle in an unchanged distance from Earth, although it is always failing! Remember Newton's First Law.
We only provide horizontal velocity in one consistent direction, Earth is acting as an external force. The combination of those two makes us orbit!
Isn't that a sweet notion? Satellites are always falling towards Earth but because of them moving so fast horizontally, they always stay at the same altitude.
Now, why is so hard to go "into space" or to orbit around our planet? Earth is really heavy, actually it is so heavy that it curves spacetime so much that we only have very few options of actually achieving orbit.
When we want to achieve orbit, the amount of horizontal speed we need is almost always equal. The decrease of gravitional pull at an altitude of 300 km is really tiny compared to you standing on the Earth's surface.
This speed is roughly 7,9 km/s or 28,000 km/h or 28 times as fast as an airplane or 14 times as fast as a bullet. To achieve this incredible speed we need a lot of fuel.
But you not only have to reach this unbelievable horizontal speed, you additionally have to move the spaceship to an altitude of 300 km or more while constantly consuming fuel carrying your fuel and the spaceship.
Therefore, approximately 90% of a spaceship's weight is made up of fuel. Certain types of fuel generate a variied specific impulse (I_SP), i.e. the amount of speed you can generate for each kilogram of fuel mass.
Because of this you can only reach orbital velocity with fuels like:

• LOX+H2 (liquid oxygen and hydrogen) - 3816 N-s/kg
• LOX+RP1 (liquid oxygen and kerosene) - 2941 N-s/kg
• N2O4+N2H4 (nitrous oxide and hydrazine) - 2862 N-s/kg

Other types of fuel won't make it. Especially can't you reach orbital velocity with solar or electrical energy from batteries.
Their specific impulse for each carried kilogram would be too low for our Earth's enormous curvature of spacetime.
If we were born on a planet with more mass like Jupiter and a gravitational of 24,8 m/s^2, which is roughly 2,5 times the one we perceive on Earth, we would be unable to leave our planet with our current technology.
Satellites, and therefore GPS navigation, would be impossible at out current stage of development.
Some spacecrafts like the voyager even used the curvature of spacetime of other planets to gain speed by exploiting their respective gravity wells. (see "Gravity assist")