The main caveat is that this may violate a site's terms of use and thus website owners may feel called upon to sue you. Depending on circumstances, the legal situation here can be a long story.
Yes, it is complicated. That said, this is partly just because there aren't enough cases - and partly because the law hasn't stabilised (took a century to stabilise after invention of printing press). It isn't clear what rights society should grant yet, for maximising business.
The main caveat is that this may violate a site's terms of use and thus website owners may feel called upon to sue you. Depending on circumstances, the legal situation here can be a long story.