[Solved] How to retrieve data from json response with scrapy?

by


You’ll want to access:

['hits']['hits'][x]['_source']['apply_url']

Where x is the number of items/nodes under hits. See https://jsoneditoronline.org/#left=cloud.22e871cf105e40a5ba32408f6aa5afeb&right=cloud.e1f56c3bd6824a3692bf3c80285ae727

As you can see, there are 10 items or nodes under hits -> hits. apply_url is under _source for each item.

def parse(self, response):
    jsonresponse = json.loads(response.body_as_unicode())
    print("============================================================================================================================")
    for x, node in enumerate(jsonresponse):
        print(jsonresponse['hits']['hits'][x]['_source']['apply_url'])

For example, print(jsonresponse['hits']['hits'][0]['_source']['apply_url']) would produce:

https://boards.greenhouse.io/mesosphere/jobs/1422922?gh_jid=1422922

0

solved How to retrieve data from json response with scrapy?