{"id":8924,"date":"2022-09-16T06:36:30","date_gmt":"2022-09-16T01:06:30","guid":{"rendered":"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/"},"modified":"2022-09-16T06:36:30","modified_gmt":"2022-09-16T01:06:30","slug":"solved-python-web-scraping-failed-to-extract-a-list-from-the-website","status":"publish","type":"post","link":"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/","title":{"rendered":"[Solved] Python Web Scraping &#8211; Failed to extract a list from the website"},"content":{"rendered":"<p> [ad_1]<br \/>\n<\/p>\n<div id=\"answer-70515769\" class=\"answer js-answer accepted-answer js-accepted-answer\" data-answerid=\"70515769\" data-parentid=\"70366821\" data-score=\"0\" data-position-on-page=\"1\" data-highest-scored=\"1\" data-question-has-accepted-highest-score=\"1\" itemprop=\"acceptedAnswer\" itemscope itemtype=\"https:\/\/schema.org\/Answer\">\n<div class=\"post-layout\">\n<div class=\"votecell post-layout--left\"><\/div>\n<div class=\"answercell post-layout--right\">\n<div class=\"s-prose js-post-body\" itemprop=\"text\">\n<p>It most likely dynamically writes the data using JavaScript.<br \/>\nYou could use libraries like Selenium or Dryscape to get the data.<\/p>\n<p>You might want to look into Web Scraping JavaScritp page with Python. Or, if you insist on using scrapy, look into <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/docs.scrapy.org\/en\/latest\/topics\/dynamic-content.html\">Selecting dynamically-loaded content<\/a>.<\/p>\n<\/p><\/div>\n<div class=\"mt24\"><\/div>\n<\/div>\n<p>            <span class=\"d-none\" itemprop=\"commentCount\"><\/span> <\/p><\/div>\n<\/div>\n<p>[ad_2]<\/p>\n<p>solved Python Web Scraping &#8211; Failed to extract a list from the website <\/p>\n","protected":false},"excerpt":{"rendered":"<p>[ad_1] It most likely dynamically writes the data using JavaScript. You could use libraries like Selenium or Dryscape to get the data. You might want to look into Web Scraping JavaScritp page with Python. Or, if you insist on using scrapy, look into Selecting dynamically-loaded content. [ad_2] solved Python Web Scraping &#8211; Failed to extract &#8230; <a title=\"[Solved] Python Web Scraping &#8211; Failed to extract a list from the website\" class=\"read-more\" href=\"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/\" aria-label=\"More on [Solved] Python Web Scraping &#8211; Failed to extract a list from the website\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[320],"tags":[2182,2511,760],"class_list":["post-8924","post","type-post","status-publish","format-standard","hentry","category-solved","tag-scrapy","tag-selector","tag-web-scraping"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>[Solved] Python Web Scraping - Failed to extract a list from the website - JassWeb<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"[Solved] Python Web Scraping - Failed to extract a list from the website - JassWeb\" \/>\n<meta property=\"og:description\" content=\"[ad_1] It most likely dynamically writes the data using JavaScript. You could use libraries like Selenium or Dryscape to get the data. You might want to look into Web Scraping JavaScritp page with Python. Or, if you insist on using scrapy, look into Selecting dynamically-loaded content. [ad_2] solved Python Web Scraping &#8211; Failed to extract ... Read more\" \/>\n<meta property=\"og:url\" content=\"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/\" \/>\n<meta property=\"og:site_name\" content=\"JassWeb\" \/>\n<meta property=\"article:published_time\" content=\"2022-09-16T01:06:30+00:00\" \/>\n<meta name=\"author\" content=\"Kirat\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kirat\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\\\/\"},\"author\":{\"name\":\"Kirat\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#\\\/schema\\\/person\\\/65c9c7b7958150c0dc8371fa35dd7c31\"},\"headline\":\"[Solved] Python Web Scraping &#8211; Failed to extract a list from the website\",\"datePublished\":\"2022-09-16T01:06:30+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\\\/\"},\"wordCount\":73,\"publisher\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#organization\"},\"keywords\":[\"scrapy\",\"selector\",\"web-scraping\"],\"articleSection\":[\"Solved\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\\\/\",\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\\\/\",\"name\":\"[Solved] Python Web Scraping - Failed to extract a list from the website - JassWeb\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#website\"},\"datePublished\":\"2022-09-16T01:06:30+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"[Solved] Python Web Scraping &#8211; Failed to extract a list from the website\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#website\",\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/\",\"name\":\"JassWeb\",\"description\":\"Build High-quality Websites\",\"publisher\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#organization\",\"name\":\"Jass Web\",\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/jassweb.com\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/jass-website-logo-1.png\",\"contentUrl\":\"https:\\\/\\\/jassweb.com\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/jass-website-logo-1.png\",\"width\":693,\"height\":132,\"caption\":\"Jass Web\"},\"image\":{\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/#\\\/schema\\\/person\\\/65c9c7b7958150c0dc8371fa35dd7c31\",\"name\":\"Kirat\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/wp-content\\\/litespeed\\\/avatar\\\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777613206\",\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/wp-content\\\/litespeed\\\/avatar\\\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777613206\",\"contentUrl\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/wp-content\\\/litespeed\\\/avatar\\\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777613206\",\"caption\":\"Kirat\"},\"sameAs\":[\"http:\\\/\\\/jassweb.com\"],\"url\":\"https:\\\/\\\/jassweb.com\\\/solved\\\/author\\\/jaspritsinghghumangmail-com\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"[Solved] Python Web Scraping - Failed to extract a list from the website - JassWeb","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/","og_locale":"en_US","og_type":"article","og_title":"[Solved] Python Web Scraping - Failed to extract a list from the website - JassWeb","og_description":"[ad_1] It most likely dynamically writes the data using JavaScript. You could use libraries like Selenium or Dryscape to get the data. You might want to look into Web Scraping JavaScritp page with Python. Or, if you insist on using scrapy, look into Selecting dynamically-loaded content. [ad_2] solved Python Web Scraping &#8211; Failed to extract ... Read more","og_url":"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/","og_site_name":"JassWeb","article_published_time":"2022-09-16T01:06:30+00:00","author":"Kirat","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Kirat"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/#article","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/"},"author":{"name":"Kirat","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31"},"headline":"[Solved] Python Web Scraping &#8211; Failed to extract a list from the website","datePublished":"2022-09-16T01:06:30+00:00","mainEntityOfPage":{"@id":"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/"},"wordCount":73,"publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"keywords":["scrapy","selector","web-scraping"],"articleSection":["Solved"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/","url":"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/","name":"[Solved] Python Web Scraping - Failed to extract a list from the website - JassWeb","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/#website"},"datePublished":"2022-09-16T01:06:30+00:00","breadcrumb":{"@id":"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/jassweb.com\/solved\/solved-python-web-scraping-failed-to-extract-a-list-from-the-website\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/jassweb.com\/solved\/"},{"@type":"ListItem","position":2,"name":"[Solved] Python Web Scraping &#8211; Failed to extract a list from the website"}]},{"@type":"WebSite","@id":"https:\/\/jassweb.com\/solved\/#website","url":"https:\/\/jassweb.com\/solved\/","name":"JassWeb","description":"Build High-quality Websites","publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/jassweb.com\/solved\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/jassweb.com\/solved\/#organization","name":"Jass Web","url":"https:\/\/jassweb.com\/solved\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/","url":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","contentUrl":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","width":693,"height":132,"caption":"Jass Web"},"image":{"@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31","name":"Kirat","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777613206","url":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777613206","contentUrl":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1777613206","caption":"Kirat"},"sameAs":["http:\/\/jassweb.com"],"url":"https:\/\/jassweb.com\/solved\/author\/jaspritsinghghumangmail-com\/"}]}},"_links":{"self":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/8924","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/comments?post=8924"}],"version-history":[{"count":0,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/8924\/revisions"}],"wp:attachment":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/media?parent=8924"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/categories?post=8924"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/tags?post=8924"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}