I think the question of fair use is separate from the question of piracy, and probably separate from the question of intellectual property in general. Even if we were to protect fair use, that doesn't make it legal to wholesale copy books. Individual piracy from people who can't really afford it is one thing and largely harmless, even a net good. I know people who only started reading books from particular authors because they pirated one copy and bought others. That's very different from a company downloading entire libraries of books without paying. Shifting the question from piracy to fair use is just another way of making you think of the wrong question.

I'd like to live in a world that doesn't gatekeep property. But we live in a world where artists aren't paid for their work directly, and in that world intellectual property is necessary.