Comets go around the Sun in a highly elliptical orbit. They can spend hundreds and thousands of years out in the depths of the solar system before they return to Sun at their perihelion. Like all orbiting bodies, comets follow Kepler's Laws - the closer they are to the Sun, the faster they move.
While a comet  is at a great distance from the Sun, its exists as a dirty snowball several kilmoeters across. But as it comes closer to the Sun, the warming of its surface causes its materials to melt and vapourise producing the comet's characteristic tail. Comet tails can be as long as the distance between the Earth and the Sun.

The image to the right shows the motion of a comet around the Sun. The red circle represents the orbit of one of terrestrial planets. As can be seen, the path of the comet is much more elliptical. In addition, the planes of revolution of comets are not limited to the plane of the ecliptic in which the planets rotate. Halley's comet is tilted at an angle of 18 degrees. Although the comet in the animation is orbiting the Sun in a counter clockwise direction, there are many comets that orbit in the opposite direction.
As can be seen from the diagram, the tail of the comet always point away from the Sun, so after a comet has passed the Sun it actually travels tail first.