I don't think that assertions such as "...not all OA journals charge anything from either authors or readers..." or even "...the majority of OA journals do not charge anybody..." are very helpful for achieving widespread open access. One does come across them regularly, though. It seems more to do with the desire not to spend anything, or rather, to see that if any money is to be spent, it's done by 'someone else'. They may be mathematically correct, though.
The trouble is, 'journal' is in many respects the wrong entity in this regard. It may be a convenient one, but that doesn't make it right. Journals come in all different sizes. They range from publishing a few articles a year to publishing thousands. The variability is such, and the tail of minuscule journals so long, that I wouldn't even be surprised if it turns out that the smallest 50% of journals altogether represent less than 10% of articles published (I didn't do the calculation, but that's my sense).
I wonder, therefore, if the assertions above hold up if one looks at modal journals (i.e. journals with a modal number of peer-reviewed articles published per year; or perhaps journals with a modal impact factor).
Even if that should be the case, there is another issue. A while ago, I publicly pondered the question whether any of the non-charging OA journals (the ones that charge neither author nor reader) would be acceptable venues for articles that are the subject of funder mandates, such as the NIH or the Wellcome Trust. Not too many, I suspect. So far, I've heard or seen no answers to that question.
The non-charging OA journals are likely to operate on the fringe of scientific and scholarly publishing, and although they no-doubt have their function in the landscape, drawing this kind of attention to them at best takes away the focus from the mainstay of the academic peer-reviewed literature, and at worst, destroys these small journals, as there would be no way of coping with a flood of submissions without charging anyone.
It is relatively easy to sustain small fringe journals (some of them may be of very high quality, of course, though those are likely to cater to very small communities) on what the Dutch would call "charity and recycled paper" (liefdewerk oud papier). That's not scalable to the peer-review literature as a whole. Open access deserves to be taken more seriously.