Scrapy Pipeline - Unhashable Type List

I am trying to create a spider that fetches all the urls from one domain and create a record of the domain name and all the headers across the urls on this domain. This is a contin

Solution 1:

In python, a list cannot be used as key in a dict. The dict keys need to be hashable (which usually means that keys need to be immutable)

So, if there is any place where you are using lists, you can convert it into a tuple before adding to a dict. tuple(mylist) should be good enough to convert the list to a tuple.