How do I read data into Python but not entirely into memory? -

i need parse through file of 100,000 records. there way without loading whole file memory? csv module (i.e., not load entire file memory)? if matters, plan on doing in idle.

i've never used cvs module, you'll want using generator, allow process record @ time without reading entire file in @ once. example, file, can like...

def read_file(some_file):     line in open(some_file):         yield line  all_lines = read_file("foo") results = process(all_lines) 

the all_lines generator , return 1 line each time referenced, in:

for line in all_lines:     ... 

i'd imagine can cvs module well.