It is, perhaps easier to understand when compared with a map:
Now, Caesar's meaning should be a little more clear. When he says:
"The Belgae rises from the extreme frontier of Gaul, extend to the lower part of the river Rhine; and look toward the north and the rising sun"
We can see that the territory of the Belgae extends from the frontier of Gaul furthest from Rome to the River Rhine in the South, and has a third border running northeast ("to the north and the rising sun").
It may seem a little confusing because the territory of the Belgae was roughly triangular, and so had only three borders.
Similarly, the territory of Aquitania, which
"extends from the river Garonne to the Pyrenaean mountains and to that part of the ocean which is near Spain: it looks between the setting of the sun, and the north star."
is bordered by the River Garonne, the Pyrenees and the ocean, with the fourth (northern) border running north-west ("between the setting of the sun, and the north star").
More generally, East is where the Sun rises, West is where it sets, and the North Star is in the north. Points between those on the compass would be North-east ("between the north star and the rising sun"), and North-west ("between the setting of the sun, and the north star").
It's also worth noting that translation from Latin to English (or, more generally, between any two languages) involves the translator making choices and interpretations.
My Latin copy of De bello Gallico has the following (emphasis mine):
"Belgae ab extremis Galliae finibus oriuntur; pertinent ad inferiorem partem fluminis Rheni; spectant in septentrionem et orientem solem."
"Aquitania a Garumna flumine ad Pyrenaeos montes et ad eam partem Oceani, quae est ad Hispaniam, pertinet; spectat inter occasum solis et septentriones."
Now, I quite like the 'standard' translation you quoted (it has a nice, poetic, ring), but alternate translations are certainly possible. However, whatever precise translation is preferred, the meaning of the text appears clear, as I described above.