Thanks to @Sean Gillies.
Posting here the solution I also posted on gis.stackexchange.com (https://gis.stackexchange.com/a/353869/9518).
Maybe it would be useful for others in search of a solution for a similar problem.
Basically, it is not possible to simultaneously make a windowed AND decimated read, so the answer to my question is NO.
However, there is an undocumented (AFAIK) parameter called
This parameter takes the 0-based index of an overview level. So, assuming overviews are present in a source
to create a Rasterio dataset from the first level of overview of source raster.
Then, simply do a windowed read in the common format: