How to Use Proxy with Scrapy

Here is a video showing you how to use free and paid proxies in Scrapy.

This may be particularly when you are downloading a number of files, and many other scenarios. One of the solutions shown in the videos works even when using Requests+BeautifulSoup instead of scrapy.

One of the solution covered is essentially a custom, open-source scrapy middleware.

The paid solution that I have covered is ScraperAPI. If you decide to make a purchase, use the special discount code coderecode10

Here the code. For a full walkthrough, view the embedded video on this page, or visit YouTube to watch it there.

4 4 votes
Article Rating
Notify of
Inline Feedbacks
View all comments

You May Also be Interested In These Blogs


Virtual Environments in Python

A Python virtual environment is a tool to isolate specific Python environments on a single machine, allowing for separate dependencies and packages for different projects.