A REALLY huge 3D array in C++ -

the point is: i'm writing kind of dumb program takes bmp image of handdrawn sound wave , converts actual sound. works fine, i'm searching way load whole image memory. used std::vector load 500 mb image see in task manager process has allocated of 8gb ram needed additional paging (stopped @ around 6 gb). guess it's kind of overhead...

and here's question: there more memory-efficient way allocate huge 3d arrays? , there speed-efficient way read huge files byte byte? (i'm writing while waiting load, it's 85% after ~50 minutes)

edit os 64-bit windows. , it's more 6gb, ram small it, windows started putting array in paging file (i made little percentage indicator , allocating around around 80% @ moment)..

and here's code i'm using allocation

vector <vector<vector<char> > > raster_data; //declaration raster_data.resize(width); for(int i=0; i<width; i++) {     raster_data[i].resize(height);      for(int j=0; j<height; j++)     {         raster_data[i][j].resize(3); //(24bpp=3bpp)     } } 

so i'm writing data not via push_back(), accessing vector normal array:

raster_data[i][j][k] // in 3d loop 

what call 3d array more 2d array containing rgb elements, , 500mb not 'really huge' days.

you approach of using std::vector store 3 bytes (char) should avoided as:

• the std::vector ifself consumes more 3 bytes (you may know how inspecting result of sizeof(std::vector<char>).
• the memory allocate fragmented

traditionnally, use case, loading 2d bitmap, memory allocated in single block, , then, values accessed using computed index.

overly simplified, :

struct rgb {            // should check sizeof(rgb) == 3*sizeof(char), sure   char channels[3]; }; std::vector<rgb> raster_data; raster_data.resize(width*height); // channel access : raster_data[j*width+i][k] = ...; 

you read entire file contents efficiently in buffer this, need consider alignment each rows, , off top of head, can't (i believe each row aligned on 4 bytes boundary).

for learn more handling bmp bitmap data in c/c++, suggest @ handmade hero, covers topic beginning, believe in episode : https://www.youtube.com/watch?v=ofmjuschxwo