How to Use Proxy with Scrapy

Here is a video showing you how to use free and paid proxies in Scrapy.

This may be particularly when you are downloading a number of files, and many other scenarios. One of the solutions shown in the videos works even when using Requests+BeautifulSoup instead of scrapy.

One of the solution covered is essentially a custom, open-source scrapy middleware.

The paid solution that I have covered is ScraperAPI. If you decide to make a purchase, use the special discount code coderecode10

Here the code. For a full walkthrough, view the embedded video on this page, or visit YouTube to watch it there.

You May Also be Interested In These Blogs