Here is a video showing you how to use free and paid proxies in Scrapy.
This may be particularly when you are downloading a number of files, and many other scenarios. One of the solutions shown in the videos works even when using Requests+BeautifulSoup instead of scrapy.
One of the solution covered is essentially a custom, open-source scrapy middleware.
The paid solution that I have covered is ScraperAPI. If you decide to make a purchase, use the special discount code coderecode10
Here the code. For a full walkthrough, view the embedded video on this page, or visit YouTube to watch it there.
