Thanks to @Sean Gillies.

Posting here the solution I also posted on gis.stackexchange.com (https://gis.stackexchange.com/a/353869/9518).

Maybe it would be useful for others in search of a solution for a similar problem.

Basically, it is not possible to simultaneously make a windowed AND decimated read, so the answer to my question is NO.

However, there is an undocumented (AFAIK) parameter called overview_level which can be passed to rasterio.open().

This parameter takes the 0-based index of an overview level. So, assuming overviews are present in a source raster, one can do

src = rasterio.open(raster, overview_level=0)

to create a Rasterio dataset from the first level of overview of source raster.

Then, simply do a windowed read in the common format:

for ji, src_window in src.block_windows(1):
    arr = src.read(1, window=src_window)