[Solved] Can I store an iterator in a file which I can read from later? Will this reduce space consumption? [closed]

by


Building on larsmans answer, a custom iterator can be built to do this:

class my_large_num(object):

    def __init__(self):
        self.num_iterations = 0

    def __iter__(self):
        return self


    def next(self):
        if self.num_iterations < 1:
            self.num_iterations += 1
            return 10**200
        else:
            raise StopIteration()

You can then:

import pickle
pickled_repr = pickle.dumps(my_large_num())
restored_object = pickle.loads(pickled_repr)
sum(restored_object)

This works because underneath, iterable objects have a next() function which raises StopIteration when done. All we’re doing is creating a class that implements this functionality.

In this specific case, regardless of the fact you have stored the class in a file, you still need to perform the iteration, and thus store 10**200 in memory, so you gain no functionality except generating the number on demand, which you can do without serializing the object.

You might be thinking of mmap style space saving. This maps memory to a file – note however this still affects the usable memory of your program.

1

solved Can I store an iterator in a file which I can read from later? Will this reduce space consumption? [closed]