{"id":12205,"date":"2022-09-30T01:20:47","date_gmt":"2022-09-29T19:50:47","guid":{"rendered":"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/"},"modified":"2022-09-30T01:20:47","modified_gmt":"2022-09-29T19:50:47","slug":"solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays","status":"publish","type":"post","link":"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/","title":{"rendered":"[Solved] Cython interfaced with C++: segmentation fault for large arrays"},"content":{"rendered":"

[ad_1]
\n<\/p>\n

\n
\n
<\/div>\n
\n
\n

The memory is being managed by your numpy arrays. As soon as they go out of scope (most likely at the end of the PySparse<\/code> constructor) the arrays cease to exist, and all your pointers are invalid. This applies to both large and small arrays, but presumably you just get lucky with small arrays.<\/p>\n

You need to hold a reference to all the numpy arrays you use for the lifetime of your PySparse<\/code> object:<\/p>\n

cdef class PySparse:\n\n  # ----------------------------------------------------------------------------\n\n  cdef Sparse *ptr\n  cdef object _held_reference # added\n\n  # ----------------------------------------------------------------------------\n\n  def __cinit__(self,**kwargs):\n      # ....\n      # your constructor code code goes here, unchanged...\n      # ....\n\n      self._held_reference = [data] # add any other numpy arrays you use to this list\n<\/code><\/pre>\n
As a rule you need to be thinking quite hard about who owns what whenever you’re dealing with C\/C++ pointers, which is a big change from the normal Python approach. Getting a pointer from a numpy array does not<\/em> copy the data and it does not<\/em> give numpy any indication that you’re still using the data.<\/p>\n
\n
Edit note:<\/strong> In my original version I tried to use locals()<\/code> as a quick way of gathering a collection of all the arrays I wanted to keep. Unfortunately, that doesn’t seem to include to cdef<\/code>ed arrays so it didn’t manage to keep the ones you were actually using (note here that astype()<\/code> makes a copy unless you tell it otherwise, so you need to hold the reference to the copy, rather than the original passed in as an argument). <\/p>\n<\/p><\/div>\n
<\/div>\n<\/div>\n
            6<\/span> <\/p><\/div>\n<\/div>\n
[ad_2]<\/p>\n
solved Cython interfaced with C++: segmentation fault for large arrays <\/p>\n","protected":false},"excerpt":{"rendered":"
[ad_1] The memory is being managed by your numpy arrays. As soon as they go out of scope (most likely at the end of the PySparse constructor) the arrays cease to exist, and all your pointers are invalid. This applies to both large and small arrays, but presumably you just get lucky with small arrays. … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[320],"tags":[361,324,3326,581,349],"class_list":["post-12205","post","type-post","status-publish","format-standard","hentry","category-solved","tag-arrays","tag-c","tag-cython","tag-numpy","tag-python"],"yoast_head":"\n[Solved] Cython interfaced with C++: segmentation fault for large arrays - JassWeb<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"[Solved] Cython interfaced with C++: segmentation fault for large arrays - JassWeb\" \/>\n<meta property=\"og:description\" content=\"[ad_1] The memory is being managed by your numpy arrays. As soon as they go out of scope (most likely at the end of the PySparse constructor) the arrays cease to exist, and all your pointers are invalid. This applies to both large and small arrays, but presumably you just get lucky with small arrays. ... Read more\" \/>\n<meta property=\"og:url\" content=\"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/\" \/>\n<meta property=\"og:site_name\" content=\"JassWeb\" \/>\n<meta property=\"article:published_time\" content=\"2022-09-29T19:50:47+00:00\" \/>\n<meta name=\"author\" content=\"Kirat\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kirat\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/\"},\"author\":{\"name\":\"Kirat\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31\"},\"headline\":\"[Solved] Cython interfaced with C++: segmentation fault for large arrays\",\"datePublished\":\"2022-09-29T19:50:47+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/\"},\"wordCount\":232,\"publisher\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\"},\"keywords\":[\"arrays\",\"c++\",\"cython\",\"numpy\",\"python\"],\"articleSection\":[\"Solved\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/\",\"url\":\"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/\",\"name\":\"[Solved] Cython interfaced with C++: segmentation fault for large arrays - JassWeb\",\"isPartOf\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#website\"},\"datePublished\":\"2022-09-29T19:50:47+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/jassweb.com\/solved\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"[Solved] Cython interfaced with C++: segmentation fault for large arrays\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/jassweb.com\/solved\/#website\",\"url\":\"https:\/\/jassweb.com\/solved\/\",\"name\":\"JassWeb\",\"description\":\"Build High-quality Websites\",\"publisher\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/jassweb.com\/solved\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\",\"name\":\"Jass Web\",\"url\":\"https:\/\/jassweb.com\/solved\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png\",\"contentUrl\":\"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png\",\"width\":693,\"height\":132,\"caption\":\"Jass Web\"},\"image\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31\",\"name\":\"Kirat\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1775193939\",\"contentUrl\":\"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1775193939\",\"caption\":\"Kirat\"},\"sameAs\":[\"http:\/\/jassweb.com\"],\"url\":\"https:\/\/jassweb.com\/solved\/author\/jaspritsinghghumangmail-com\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"[Solved] Cython interfaced with C++: segmentation fault for large arrays - JassWeb","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/","og_locale":"en_US","og_type":"article","og_title":"[Solved] Cython interfaced with C++: segmentation fault for large arrays - JassWeb","og_description":"[ad_1] The memory is being managed by your numpy arrays. As soon as they go out of scope (most likely at the end of the PySparse constructor) the arrays cease to exist, and all your pointers are invalid. This applies to both large and small arrays, but presumably you just get lucky with small arrays. ... Read more","og_url":"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/","og_site_name":"JassWeb","article_published_time":"2022-09-29T19:50:47+00:00","author":"Kirat","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Kirat","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/#article","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/"},"author":{"name":"Kirat","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31"},"headline":"[Solved] Cython interfaced with C++: segmentation fault for large arrays","datePublished":"2022-09-29T19:50:47+00:00","mainEntityOfPage":{"@id":"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/"},"wordCount":232,"publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"keywords":["arrays","c++","cython","numpy","python"],"articleSection":["Solved"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/","url":"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/","name":"[Solved] Cython interfaced with C++: segmentation fault for large arrays - JassWeb","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/#website"},"datePublished":"2022-09-29T19:50:47+00:00","breadcrumb":{"@id":"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/jassweb.com\/solved\/solved-cython-interfaced-with-c-segmentation-fault-for-large-arrays\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/jassweb.com\/solved\/"},{"@type":"ListItem","position":2,"name":"[Solved] Cython interfaced with C++: segmentation fault for large arrays"}]},{"@type":"WebSite","@id":"https:\/\/jassweb.com\/solved\/#website","url":"https:\/\/jassweb.com\/solved\/","name":"JassWeb","description":"Build High-quality Websites","publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/jassweb.com\/solved\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/jassweb.com\/solved\/#organization","name":"Jass Web","url":"https:\/\/jassweb.com\/solved\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/","url":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","contentUrl":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","width":693,"height":132,"caption":"Jass Web"},"image":{"@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31","name":"Kirat","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/image\/","url":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1775193939","contentUrl":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1775193939","caption":"Kirat"},"sameAs":["http:\/\/jassweb.com"],"url":"https:\/\/jassweb.com\/solved\/author\/jaspritsinghghumangmail-com\/"}]}},"_links":{"self":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/12205","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/comments?post=12205"}],"version-history":[{"count":0,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/12205\/revisions"}],"wp:attachment":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/media?parent=12205"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/categories?post=12205"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/tags?post=12205"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}