Help me understand GROUP BY time performance

I was previously under the naive assumption that queries like

SELECT first(value) from… GROUP BY time(1w)

would be faster than e.g.

SELECT mean(value) from…

This since the latter query must visit every point(ignoring caching), but the first really only needs to visit (and collect) the first point in every week. But after having benchmarked some queries this really does not seem to be true, it looks like both queries will visit the same number of points, and take the same amount of time. Is there any way to make the queries perform as I envisioned group by did, essentially enabling sampling from arbitrary dense/long series as long as we pick correct time?